[12:55:56] <inflatador>	 Oh very nice, I do want to see that!
[12:56:07] <inflatador>	 Also, happy Monday all
[13:18:16] <inflatador>	 starting the ES65->68 upgrade in eqiad shortly
[13:37:25] <inflatador>	 gehel or anyone else, can you show me how to "revert the reversion" in gerrit? Re: https://gerrit.wikimedia.org/r/c/operations/puppet/+/763484/1 . The reversion is not needed
[13:37:55] <inflatador>	 the reversion is not in the git log in production
[13:38:08] <inflatador>	 so I think I just want to discard it?
[13:38:26] <inflatador>	 brb
[13:53:19] <inflatador>	 (oh yeah, he's off today)
[14:26:59] <inflatador>	 ebernhardson when you get in, could you take a look at https://gerrit.wikimedia.org/r/c/operations/puppet/+/783848 ? Just need to get puppet fixed up
[14:44:30] <inflatador>	 mpham are we still doing sprint planning today?
[14:55:39] <ebernhardson>	 \o
[14:57:03] <ebernhardson>	 inflatador: seems reasonable to me
[14:59:06] <cbogen_>	 mpham just messaged me that he's locked out and may not make it to planning
[14:59:21] <cbogen_>	 we have a pretty small group of folks in today, so thinking maybe we just cancel
[14:59:47] <inflatador>	 cbogen_ gotcha. I think most of our European associates are out today due to bank holiday
[14:59:58] <cbogen_>	 yep. I'm gonna cancel now!
[15:00:07] <inflatador>	 Cool.
[15:00:20] <cbogen_>	 actually I don't have access to cancel the event
[15:00:23] <inflatador>	 Thanks ebernhardson !
[15:00:26] <cbogen_>	 so if anyone shows up and is wondering why no one's there :) 
[15:11:56] <ebernhardson>	 aww, this wbsearchentities optimization has been working so well with minimal errors, but now running the step that actually does the optimization step it's failing while trying to read a dataset it built in previous round, some lockfile not found :( i guess i have to remember how tensroflow works :)
[15:16:29] <ebernhardson>	 actually not so bad, turns out i had a system specific tmp dir (/run/user/$(id -u)) when we ran this previously and of course hard coded that in
[15:24:46] <inflatador>	 brb
[15:32:26] <inflatador>	 back
[15:52:13] <ebernhardson>	 maybe i should reboot my desktop, it turns out pushing a laptop with all-cores to 100% gets throttled to 2.1Ghz :P
[16:13:34] <inflatador>	 back
[16:14:07] <inflatador>	 Disable those c-states and watch it cook ;)
[17:05:47] <inflatador>	 haircut/lunch, back in ~45
[17:18:29] <ebernhardson>	 so, this optimizers claims that the production-deployed ranking algorithm for item_en achieved a mean characters-typed of 6.68,  post optimization brings it to 6.09
[17:18:36] * ebernhardson doesn't remember how that compares to before
[17:19:15] <ebernhardson>	 std increases from 2.00 to 2.27, so less characters typed but slightly more variance 
[17:22:40] <ebernhardson>	 i guess i gotta find the notebooks from before that graphed improvements vs query percentiles (like https://phab.wmfusercontent.org/file/download/cvhhts7mfe4zcnr734nx/PHID-FILE-iytjtx5ahsna4z6fb5q3/wikidata_wbsearchentities_completion_search_analysis_-_oct18-jan19_-_item_-_de)
[17:23:25] <ebernhardson>	 based on that, last time de improved from 6.27 to 5.75, so same general scale of improvement (if the metric is legit)
[17:24:00] * ebernhardson wonders why he chose to use dec 25-jan 7 as a test dataset last time, thats going to be a biased dataset
[18:14:14] <inflatador>	 Back. Since most of us are out of office today, I'm going to postpone the ES upgrad until tomorrow
[19:04:20] <inflatador>	 ebernhardson you still running a job on cloudelastic? Was thinking of trying your PR ( https://gerrit.wikimedia.org/r/c/operations/software/spicerack/+/781009 ) in that env
[19:15:14] <ebernhardson>	 inflatador: should be all done
[19:15:18] <ebernhardson>	 err, s/should/is/
[19:15:35] <inflatador>	 Cool, will give your PR a try shortly
[20:11:59] <inflatador>	 Also created https://phabricator.wikimedia.org/T306389 to track the update
[20:21:41] <ebernhardson>	 makes sense, i should make more tickets for things i do. It's an ongoing problem and you're not the first to notice :)
[20:23:14] <inflatador>	 I have the opposite problem. I make tickets and never do them ;P
[20:27:55] <ebernhardson>	 forgot i scheduled something w/ the doc today, heading out a few hours
[20:32:54] <inflatador>	 OK, just started rolling restart of cloudelastic, will letcha know how it goes
[21:25:09] <inflatador>	 Rolling restart successful using ebernhardson 's patch; more details at https://phabricator.wikimedia.org/T306389
[21:40:21] <inflatador>	 Out a little early, my son has his 2nd recital in less than wk ;0
[22:15:54] <ebernhardson>	 back