[07:48:52] <_joe_>	 dduvall: I think it makes a lot of sense to keep one single build technology if we can
[07:49:11] <_joe_>	 so I just need the time to work on it, or maybe someone else could wink wink
[07:49:30] <_joe_>	 I think I've modularized the driver enough at this point :)
[08:29:51] <elukey>	 morning folks
[08:30:12] <elukey>	 I see again staged requestctl diffs in /var/lib/git/operations/private on puppetmaster1001
[08:36:48] <elukey>	 I filed https://gerrit.wikimedia.org/r/c/operations/puppet/+/1059253 to set DEBUG logging for the related systemd unit
[08:37:00] <elukey>	 (instead of warn)
[08:39:25] <elukey>	 if anybody has time for a quick code review --^
[08:41:14] <elukey>	 (correction - it is already INFO, 3 -vvv is DEBUG)
[08:44:50] <elukey>	 (testing now)
[10:07:02] <Emperor>	 Hi folks, could I get a review of https://gerrit.wikimedia.org/r/c/operations/puppet/+/1059259 please? We're a little short-handed in DP today. It's to remove a failed drive from the rings (docs at https://wikitech.wikimedia.org/wiki/Swift/Ring_Management#Removing_a_device ). It's a very small change :)
[10:17:54] <elukey>	 Emperor: happy to do it, but can you please run PCC so we know what's changing etc..?
[10:47:31] * Emperor awaits the PCC jobs
[10:51:07] <elukey>	 pcc is enqueued afaics, zuul is a bit under pressure https://integration.wikimedia.org/zuul/
[10:51:12] <elukey>	 but it seems climbing down
[11:31:33] <Emperor>	 elukey: PCC completed
[12:50:08] <elukey>	 ack I see that Manuel already reviewed it, super
[13:39:53] <ottomata>	 mutante: what do you think of https://phabricator.wikimedia.org/T370424 ?  Can we do it?
[13:42:49] <cdanis>	 sounds like a bitu pushbutton to me ;)
[13:46:14] <_joe_>	 ottomata: IIRC the process was tightened because someone who was granted access, an employee, did "leak" data involuntarily
[13:52:07] <ottomata>	 _joe_:  aye makes sense, but in practice we don't really do any more than just approve employees
[13:52:48] <ottomata>	 so in hindsight I don't think the tightened process will address that problem 
[13:52:51] <_joe_>	 ack, I was just recollecting why the approval step was introduced in the first place 
[13:52:54] <ottomata>	 aye cool
[13:52:59] <_joe_>	 a request by Nuria
[13:53:03] <ottomata>	 aye k
[13:53:18] <ottomata>	 _joe_: ! any luck on a rewrite rule?  should I just go back to /w in mw-config?
[13:53:35] <_joe_>	 heh sorry didn't have time
[13:53:40] <_joe_>	 it's KR reporting period
[13:54:08] <ottomata>	 ya, its okay.  do you have high hopes for one?  if not, i'll just start working on /w (i have few meetings today so can do some coding)
[13:54:28] <_joe_>	 I think it should be possible
[13:55:06] <ottomata>	 hm okay.  if you give me a pointer I could try?
[13:55:13] <ottomata>	 without mod proxy?
[13:56:35] <_joe_>	 we have mod proxy
[13:57:03] <_joe_>	 but without a <proxypass> stanza? not sure
[13:57:11] <ottomata>	 i tried a lot of things
[13:57:12] <ottomata>	 https://phabricator.wikimedia.org/T353817#10029516
[13:57:20] <ottomata>	  It looks like mod_proxy needs some other module enabled to handle proxying to https?
[13:57:20] <ottomata>	 I believe we'd need modules:
[13:57:20] <ottomata>	 proxy_http
[13:57:20] <ottomata>	 ssl
[13:57:21] <ottomata>	 SSLProxyEngine on
[14:20:51] <_joe_>	 ottomata: you don't need to proxy to https, you can proxy locally, but I'll take a look when I don't have paperwork to fill :)
[14:21:17] <_joe_>	 in theory ProxyPass /event/beacon /w/rest.php/... should work
[14:24:13] <andrewbogott>	 _joe_: I'm trying to find the initial email/blog post/whatever that you and yuvi wrote after you decided to adopt k8s as the future of mw + toolforge. I don't suppose you have that where you can find it?
[14:24:22] <andrewbogott>	 ^ very low priority question, obv :)
[14:27:14] <_joe_>	 andrewbogott: the closest thing I remember was the orchestrator evaluation spreadsheet 
[14:27:32] <_joe_>	 but it was owned by yuvi and IIRC all docs owned by someone who leaves used to vanish from gdocs
[14:27:41] <_joe_>	 so I can't find it
[14:27:48] <andrewbogott>	 ok :/ I'll see if google can turn it up for me but won't hold my breath
[14:28:08] <andrewbogott>	 thx
[14:30:28] <_joe_>	 I'm sure there's other stuff but that's 8 years ago, sorry
[14:30:55] <_joe_>	 maybe phab has some vestiges
[14:49:50] <taavi>	 andrewbogott: https://lists.wikimedia.org/pipermail/labs-announce/2015-September/000071.html
[14:50:11] <taavi>	 (from https://wikitech.wikimedia.org/wiki/User:Majavah/History_of_Toolforge)
[14:51:02] <_joe_>	 taavi: yeah the doc has been deleted sadly
[14:51:05] <_joe_>	 :/
[14:51:17] <_joe_>	 there was a period where every doc from a person that left was wiped out 
[14:53:02] <_joe_>	 same for the design doc that we wrote before the evaluation :/
[14:54:00] <andrewbogott>	 taavi: nice, thank you!
[14:56:10] <_joe_>	 andrewbogott: there's nothing about decisions there, sadly
[14:56:14] <_joe_>	 it's just announcements
[14:57:00] <andrewbogott>	 yeah but it provides more timeline specifics
[14:58:15] * andrewbogott realizes he forgot to eat breakfast, seeks breakfast
[15:44:27] <Lucas_WMDE>	 hi! could someone with sufficient access maybe peek at `journalctl -u wmde-analytics-minutely.service -e` on stat1011 for me?
[15:44:29] <Lucas_WMDE>	 (context: https://phabricator.wikimedia.org/T371706)
[15:49:25] <Emperor>	 looking
[15:49:41] <Emperor>	 Lucas_WMDE: I don't really know what I'm looking for but
[15:49:44] <Emperor>	 Aug 02 15:48:00 stat1011 minutely.sh[2012740]: PHP Parse error:  syntax error, unexpected ',' in /srv/analytics-wmde/graphite/src/scripts/lib/WikimediaDbSectionMapper.php(50) : eval()'d code on line 1
[15:51:50] <Lucas_WMDE>	 o_O
[15:51:55] * Lucas_WMDE looks
[15:52:16] <Emperor>	 journal only goes back as far as 14:17:20 today, so I can't tell you when that behaviour started
[15:52:43] <Lucas_WMDE>	 oh my GOD https://gerrit.wikimedia.org/g/analytics/wmde/scripts/+/d318d6fdf7/lib/WikimediaDbSectionMapper.php#41
[15:52:51] <Lucas_WMDE>	 we literally get a php file from noc.w.o and eval it?!
[15:53:57] <Lucas_WMDE>	 aha, and zabe moved the section mapping to a separate file https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/1057034
[15:53:59] <Lucas_WMDE>	 that’ll do it, I think
[15:54:07] <Emperor>	 I mean, at least that's curl|bashware from infra we own....
[15:54:12] <Lucas_WMDE>	 that should be enough to continue, thanks Emperor :)
[15:54:20] <Emperor>	 Lucas_WMDE: NP, happy to help
[22:00:07] <Krinkle>	 cdanis: I'm trying to understand where "Tags" and "Process" are decided on in Jeager UI e.g. at https://trace.wikimedia.org/trace/6f0557bb6db777bc3330d6e1742a0071 is this a Jeager-specific grouping? I've looked at https://github.com/open-telemetry/semantic-conventions/tree/01c542ccacd869ee8a329aa33c65fb2c47bf2815/docs/attributes-registry but could not find what makes the attributes show in different parts of the UI.
[22:01:27] <Krinkle>	 this one might be a better example as it has non-zero children: https://trace.wikimedia.org/trace/65e774a1a89d7ba0204afc4cca6e6b53