[09:39:36] Early lunch [09:52:32] lunch 2 [12:59:36] o/ [13:03:38] sigh... 1500 wikis failed accross eqiad & codfw [13:03:50] *failed the reindex [13:06:33] that's not good...any idea why? [13:09:47] maybe a bug in how maint scripts are launched perhaps, seeing things like "tried path '/srv/mediawiki/php-1.41.0-wmf.2/maintenance/UpdateSearchIndexConfig.php' and class 'UpdateSearchIndexConfig\php" [13:12:18] Do we have or need alerts for this? I don't see any, but my email filters could be hiding them [13:13:03] no reindex are run manually from a tmux session by Trey or Erik and sometimes myself, it's not automated [13:21:01] Ah OK, if it needs to be automated LMK [13:38:10] ryankemper: any chance we can complete the WDQS SLO dashboard by April 19? [16:46:55] dcausse and inflatador—there was a message back in January on wikitech-l about deprecating running maint scripts directly: https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/thread/UFANXHGOD3FYEGQQFHECSO55UMXX4AMD/ [16:47:23] I was wondering at the time if that was going to break reindexing. I think maybe it did. [16:48:02] Trey314159: yes I saw this email but was not clear on how this will be affecting the mwscript wrapper [16:49:07] T326800 seems related [16:49:08] T326800: Make Wikimedia mwscript use run.php to run maintenance scripts - https://phabricator.wikimedia.org/T326800 [16:50:05] indeed [17:52:18] lunch, back in ~50 [17:53:32] dinner [19:16:03] had an eye appointment and unexpectedly got my eyes dilated (somehow I never expect it) and now I have a bad case of CSS (Can't See Squat), so I'm out for the rest of the day [19:19:33] Trey314159: and you think that eye doctors are there to help you see better :/