[10:00:34] <dcausse>	 lunch
[10:43:36] <pfischer>	 lunch
[12:31:23] <dcausse>	 inflatador: I have a conflict today and have to cancel our pairing session, sorry about that!
[12:45:51] <gehel>	 Trey314159: looks like there is one more comment on your blog post, for which you might have a reply: https://diff.wikimedia.org/2023/04/28/language-harmony-and-unpacking-a-year-in-the-life-of-a-search-nerd/#comment-74252
[12:54:22] <Trey314159>	 gehel—thanks, I'll reply!
[13:01:44] <inflatador>	 <o/
[13:01:50] <inflatador>	 dcausse np, see you tomorrow
[13:02:05] <dcausse>	 o/
[13:08:55] <ottomata>	 pfischer: o/ i'm skimming over CRs, but not closely, please ping me when you need me to look closely at something
[14:59:24] <dcausse>	 might be couple minutes late for the triage
[15:01:31] <gehel>	 inflatador, ryankemper, pfischer: triage: https://meet.google.com/eki-rafx-cxi
[16:56:03] <inflatador>	 workout/lunch, back in ~1h
[18:10:04] <ebernhardson>	 curious, i wonder how this is possible. The max age of titlesugget indices, only on this one cluster on one day, fluxuates back and forth between 1 and 2 days: https://grafana.wikimedia.org/goto/kjBjk-8Vz?orgId=1
[18:12:43] <inflatador>	 back
[18:20:47] <dcausse>	 ebernhardson: the exporter is getting the the max directly?
[18:22:59] <Trey314159>	 ebernhardson: that *is* weird
[18:23:40] <ebernhardson>	 dcausse: it's performing an aggregation query against *_titlesuggest once an hour
[18:26:30] <dcausse>	 not sure... perhaps deletion on the batch_id is not working as expected some times? 
[18:27:50] <ebernhardson>	 hmm, but then how would it show up and then go away? there shouln't be any maint scripts running since nothing resets back close to 0
[18:28:19] <ebernhardson>	 it's like deleted data becomes visible, then goes away.
[18:28:54] <ebernhardson>	 it's only once though, i suppose not worth significant investigation yet
[18:29:03] <ebernhardson>	 might indicate some underlying problem though
[18:32:30] <dcausse>	 yes it's weird, can't find a sane explanation
[19:29:24] <dcausse>	 dinner
[20:00:19] <ryankemper>	 Brian and I reviewed the workboard and moved several lower-priority tickets back to the backlog. We think what's in the SRE column now is achievable
[20:38:48] <inflatador>	 mpham ^^
[20:41:35] <inflatador>	  FWiW, our bullseye puppet config removes python2 by default. so we'll probably need to override that somewhere https://phabricator.wikimedia.org/T331300
[22:15:19] <inflatador>	 ryankemper just checked and it looks like cumin cmd failed on some wdqs hosts , they just need wdqs-categories restarted if you have time to do that . List of hosts here: https://phabricator.wikimedia.org/P48235
[22:17:16] <ryankemper>	 ack
[22:23:40] <ebernhardson>	 curious, creating an elasticsearch index after a cluster restart takes significantly longer than normally...maybe i need to wait for green after a restart before creating indices
[23:52:47] <ebernhardson>	 wow, with just 40 wikibase indices it takes 5 minutes to restart a cluster, and another 6 minutes before all the indices are green. and all these indices are empty