[00:14:21] PROBLEM - SSH on an-worker1140 is CRITICAL: CRITICAL - Socket timeout after 10 seconds https://wikitech.wikimedia.org/wiki/SSH/monitoring [00:35:01] PROBLEM - Check systemd state on an-worker1140 is CRITICAL: CRITICAL - degraded: The following units failed: systemd-timedated.service https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [00:49:31] PROBLEM - Host an-worker1140 is DOWN: PING CRITICAL - Packet loss = 100% [01:30:09] RECOVERY - Host an-worker1140 is UP: PING OK - Packet loss = 0%, RTA = 0.15 ms [01:30:17] PROBLEM - Check systemd state on an-worker1140 is CRITICAL: CRITICAL - degraded: The following units failed: systemd-timedated.service https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [01:49:13] PROBLEM - Host an-worker1140 is DOWN: PING CRITICAL - Packet loss = 100% [04:21:39] RECOVERY - Check systemd state on an-launcher1002 is OK: OK - running: The system is fully operational https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [05:59:03] 10Data-Engineering, 10Cloud-Services, 10Wikimedia Enterprise: Data request: make rendered HTML page dumps available on stats machines or labs - https://phabricator.wikimedia.org/T331018 (10Legoktm) @awight it's already on Toolforge and WMCS, see `/public/dumps/public/other/enterprise_html/` [06:46:23] 10Data-Engineering-Planning, 10DBA, 10Data-Persistence, 10Infrastructure-Foundations, and 11 others: eqiad row A switches upgrade - https://phabricator.wikimedia.org/T329073 (10Marostegui) [07:55:13] 10Data-Engineering, 10Data-Services, 10Wikimedia Enterprise: Data request: make rendered HTML page dumps available on stats machines or labs - https://phabricator.wikimedia.org/T331018 (10taavi) [11:35:13] PROBLEM - Check systemd state on an-launcher1002 is CRITICAL: CRITICAL - degraded: The following units failed: produce_canary_events.service https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [11:46:21] RECOVERY - Check systemd state on an-launcher1002 is OK: OK - running: The system is fully operational https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state