[00:37:57] PROBLEM - Check unit status of monitor_refine_eventlogging_legacy on an-launcher1002 is CRITICAL: CRITICAL: Status of the systemd unit monitor_refine_eventlogging_legacy https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_timers [03:01:14] hey Jo, thanks for the rerun, I just saw the email. I see mw history failed, investigating [03:14:58] !log rerunning mw history since the last failure just looked like a fluke [03:15:00] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [06:04:09] 10Data-Engineering, 10Platform Engineering: Deploy AQS service to codfw clusters - https://phabricator.wikimedia.org/T309808 (10Aklapper) [06:04:16] 10Data-Engineering-Radar, 10Cassandra, 10Generated Data Platform: AQS multi-datacenter cluster expansion - https://phabricator.wikimedia.org/T307641 (10Aklapper) [06:04:18] 10Data-Engineering, 10Platform Engineering: Deploy AQS service to codfw clusters - https://phabricator.wikimedia.org/T309808 (10Aklapper) [06:04:35] 10Data-Engineering, 10Platform Engineering: Deploy AQS service to codfw clusters - https://phabricator.wikimedia.org/T309808 (10Aklapper) [@Eevans: Please use subtasks for dependencies - thanks!] [06:35:45] 10Quarry, 10GitLab (Project Migration): Move quarry to gitlab - https://phabricator.wikimedia.org/T308978 (10Aklapper) Update links on https://meta.wikimedia.org/w/index.php?title=Research:Quarry etc once this has happened. [07:49:43] 10Quarry, 10Documentation: Landing page for Quarry - https://phabricator.wikimedia.org/T308783 (10Aklapper) [11:58:22] hm, looks like mw history is failing at the very end, looking at it closer [11:59:26] so it can't be any kind of schema data type thing, that would fail much earlier. It's gotta be this shuffle map thing that produces indeterminate results like it said last time [19:58:21] I think a good idea would be to run it for a small wiki, just to make sure. Because I don't understand the resource issue, it should have the whole cluster to play with