[00:01:30] <MacFanBot>	 [02statichelp] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/statichelp/commit/13ff65fb7b4c66311e271cb22fa509be3de0fc56
[00:01:30] <MacFanBot>	 02statichelp/03main 07WikiTideBot 0313ff65f Bot: Auto-update Tech namespace pages 2026-09-24 00:01:27
[00:19:39] <icinga-miraheze>	 PROBLEM - cp201 Disk Space on cp201 is WARNING: DISK WARNING - free space: / 45221MiB (10.0% inode=100%);
[00:30:29] <icinga-miraheze>	 RECOVERY - speedscope211 Check unit status of speedscope_daily_aggregation on speedscope211 is OK: OK: Status of the systemd unit speedscope_daily_aggregation
[00:35:58] <icinga-miraheze>	 PROBLEM - cp191 Disk Space on cp191 is WARNING: DISK WARNING - free space: / 45359MiB (10.0% inode=100%);
[00:37:36] <icinga-miraheze>	 RECOVERY - cp201 Disk Space on cp201 is OK: DISK OK - free space: / 86361MiB (19.0% inode=100%);
[00:49:28] <icinga-miraheze>	 PROBLEM - speedscope211 HTTPS on speedscope211 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Connection timed out after 10004 milliseconds
[00:49:39] <icinga-miraheze>	 PROBLEM - speedscope211 speedscope.wikitide.net HTTPS on speedscope211 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[00:51:01] <icinga-miraheze>	 PROBLEM - speedscope211 SSH on speedscope211 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[00:51:42] <icinga-miraheze>	 PROBLEM - speedscope211 APT on speedscope211 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds.
[00:52:10] <icinga-miraheze>	 PROBLEM - speedscope211 Puppet on speedscope211 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds.
[00:52:21] <icinga-miraheze>	 PROBLEM - speedscope211 chrony on speedscope211 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds.
[00:52:35] <icinga-miraheze>	 PROBLEM - speedscope211 speedscope 3000 check on speedscope211 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds.
[00:53:00] <icinga-miraheze>	 PROBLEM - speedscope211 nftables_active on speedscope211 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds.
[00:53:19] <icinga-miraheze>	 PROBLEM - speedscope211 Current Load on speedscope211 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds.
[00:53:23] <icinga-miraheze>	 PROBLEM - speedscope211 PowerDNS Recursor on speedscope211 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds.
[00:53:26] <icinga-miraheze>	 PROBLEM - speedscope211 conntrack_table_size on speedscope211 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds.
[00:53:29] <icinga-miraheze>	 PROBLEM - speedscope211 Check unit status of speedscope_daily_aggregation on speedscope211 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds.
[00:53:36] <icinga-miraheze>	 PROBLEM - speedscope211 Check unit status of speedscope_hourly_aggregation on speedscope211 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds.
[01:00:28] <MacFanBot>	 [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/697a0c7efc3b9a0472d6a2fbf5eadd36ff34bb0d
[01:00:28] <MacFanBot>	 02ssl/03main 07WikiTideBot 03697a0c7 Bot: Auto-update domain lists
[01:13:58] <icinga-miraheze>	 RECOVERY - cp191 Disk Space on cp191 is OK: DISK OK - free space: / 86870MiB (19.1% inode=100%);
[01:33:42] <icinga-miraheze>	 RECOVERY - speedscope211 speedscope 3000 check on speedscope211 is OK: TCP OK - 0.000 second response time on 127.0.0.1 port 3000
[01:33:45] <icinga-miraheze>	 RECOVERY - speedscope211 Check unit status of speedscope_hourly_aggregation on speedscope211 is OK: OK: Status of the systemd unit speedscope_hourly_aggregation
[01:33:49] <icinga-miraheze>	 RECOVERY - speedscope211 HTTPS on speedscope211 is OK: HTTP OK: HTTP/2 200 - 354 bytes in 0.013 second response time
[01:33:49] <icinga-miraheze>	 RECOVERY - speedscope211 conntrack_table_size on speedscope211 is OK: OK: nf_conntrack is 0 % full
[01:34:03] <icinga-miraheze>	 RECOVERY - speedscope211 PowerDNS Recursor on speedscope211 is OK: DNS OK: 0.055 seconds response time. speedscope211.fsslc.wtnet returns 10.0.21.172
[01:35:08] <icinga-miraheze>	 RECOVERY - speedscope211 SSH on speedscope211 is OK: SSH OK - OpenSSH_10.0p2 (protocol 2.0)
[01:35:12] <icinga-miraheze>	 RECOVERY - speedscope211 Puppet on speedscope211 is OK: OK: Puppet is currently enabled, last run 52 minutes ago with 0 failures
[01:35:17] <icinga-miraheze>	 RECOVERY - speedscope211 chrony on speedscope211 is OK: chrony OK: System Time 0.000003927 seconds slow of NTP time, Stratum 4
[01:35:24] <icinga-miraheze>	 RECOVERY - speedscope211 speedscope.wikitide.net HTTPS on speedscope211 is OK: HTTP OK: HTTP/1.1 200 OK - 26617 bytes in 0.042 second response time
[01:35:28] <icinga-miraheze>	 RECOVERY - speedscope211 APT on speedscope211 is OK: APT OK: 78 packages available for upgrade (0 critical updates).
[01:35:37] <icinga-miraheze>	 RECOVERY - speedscope211 nftables_active on speedscope211 is OK: OK nftables input default policy is set
[01:50:36] <MacFanBot>	 [02mw-config] 07alistair3149 drafted pull request #6565: Update Citizen config for 3.23 (07miraheze:03main...07alistair3149:03citizen-3.23-config) 13https://github.com/miraheze/mw-config/pull/6565
[01:51:05] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages on mw152: xdg-dbus-proxy
[01:51:11] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[01:51:25] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages on mw151: xdg-dbus-proxy
[01:51:31] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[01:51:38] <miraheze-github>	 miraheze/mw-config - alistair3149 the build passed.
[01:51:46] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages on mw153: xdg-dbus-proxy
[01:51:50] <icinga-miraheze>	 RECOVERY - mw152 APT on mw152 is OK: APT OK: 185 packages available for upgrade (0 critical updates).
[01:51:51] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[01:52:08] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages on mw161: xdg-dbus-proxy
[01:52:20] <icinga-miraheze>	 RECOVERY - mw151 APT on mw151 is OK: APT OK: 185 packages available for upgrade (0 critical updates).
[01:52:30] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[01:52:30] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages on mw163: xdg-dbus-proxy
[01:52:35] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[01:52:52] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages on mw171: xdg-dbus-proxy
[01:53:00] <icinga-miraheze>	 RECOVERY - mw163 APT on mw163 is OK: APT OK: 185 packages available for upgrade (0 critical updates).
[01:53:05] <MacFanBot>	 [02mw-config] 07alistair3149 readied pull request #6565: Update Citizen config for 3.23 (07miraheze:03main...07alistair3149:03citizen-3.23-config) 13https://github.com/miraheze/mw-config/pull/6565
[01:53:14] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages on mwtask161: xdg-dbus-proxy
[01:53:16] <icinga-miraheze>	 RECOVERY - mw153 APT on mw153 is OK: APT OK: 185 packages available for upgrade (0 critical updates).
[01:53:21] <icinga-miraheze>	 RECOVERY - mw171 APT on mw171 is OK: APT OK: 185 packages available for upgrade (0 critical updates).
[01:53:32] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[01:53:36] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages on mw173: xdg-dbus-proxy
[01:53:55] <icinga-miraheze>	 RECOVERY - mw161 APT on mw161 is OK: APT OK: 185 packages available for upgrade (0 critical updates).
[01:53:58] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages on mw193: xdg-dbus-proxy
[01:54:14] <icinga-miraheze>	 RECOVERY - mw193 APT on mw193 is OK: APT OK: 185 packages available for upgrade (0 critical updates).
[01:54:21] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages on mw162: xdg-dbus-proxy
[01:54:42] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages on mw201: xdg-dbus-proxy
[01:54:56] <icinga-miraheze>	 RECOVERY - mw162 APT on mw162 is OK: APT OK: 185 packages available for upgrade (0 critical updates).
[01:54:58] <icinga-miraheze>	 RECOVERY - mwtask161 APT on mwtask161 is OK: APT OK: 190 packages available for upgrade (0 critical updates).
[01:54:59] <icinga-miraheze>	 PROBLEM - cp201 Varnish Backends on cp201 is CRITICAL: 3 backends are down. mw161 mw202 mw203
[01:55:05] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages on mw172: xdg-dbus-proxy
[01:55:10] <icinga-miraheze>	 RECOVERY - mw173 APT on mw173 is OK: APT OK: 185 packages available for upgrade (0 critical updates).
[01:55:13] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[01:55:30] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages on test151: xdg-dbus-proxy
[01:55:33] <icinga-miraheze>	 RECOVERY - mw201 APT on mw201 is OK: APT OK: 185 packages available for upgrade (0 critical updates).
[01:55:43] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[01:55:52] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages on mw191: xdg-dbus-proxy
[01:56:02] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[01:56:07] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[01:56:12] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[01:56:14] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages on mw181: xdg-dbus-proxy
[01:56:18] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[01:56:20] <icinga-miraheze>	 RECOVERY - mw172 APT on mw172 is OK: APT OK: 186 packages available for upgrade (0 critical updates).
[01:56:22] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[01:56:27] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[01:56:36] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages on mwtask151: xdg-dbus-proxy
[01:56:41] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[01:56:43] <icinga-miraheze>	 RECOVERY - test151 APT on test151 is OK: APT OK: 195 packages available for upgrade (0 critical updates).
[01:56:46] <icinga-miraheze>	 RECOVERY - mwtask151 APT on mwtask151 is OK: APT OK: 190 packages available for upgrade (0 critical updates).
[01:56:57] <icinga-miraheze>	 RECOVERY - cp201 Varnish Backends on cp201 is OK: All 34 backends are healthy
[01:56:59] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages on mw192: xdg-dbus-proxy
[01:57:04] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[01:57:22] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages on mw203: xdg-dbus-proxy
[01:57:27] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[01:57:45] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages on mwtask171: xdg-dbus-proxy
[01:57:50] <icinga-miraheze>	 RECOVERY - mw191 APT on mw191 is OK: APT OK: 185 packages available for upgrade (0 critical updates).
[01:57:50] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[01:58:06] <icinga-miraheze>	 RECOVERY - mw181 APT on mw181 is OK: APT OK: 185 packages available for upgrade (0 critical updates).
[01:58:09] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages on mw182: xdg-dbus-proxy
[01:58:14] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[01:58:22] <icinga-miraheze>	 RECOVERY - mw203 APT on mw203 is OK: APT OK: 185 packages available for upgrade (0 critical updates).
[01:58:30] <icinga-miraheze>	 RECOVERY - mw192 APT on mw192 is OK: APT OK: 185 packages available for upgrade (0 critical updates).
[01:58:32] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages on mw183: xdg-dbus-proxy
[01:58:37] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[01:58:57] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages on mw202: xdg-dbus-proxy
[01:59:01] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[01:59:18] <icinga-miraheze>	 RECOVERY - mwtask171 APT on mwtask171 is OK: APT OK: 191 packages available for upgrade (0 critical updates).
[01:59:22] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages on mwtask181: xdg-dbus-proxy
[01:59:24] <icinga-miraheze>	 RECOVERY - mw202 APT on mw202 is OK: APT OK: 185 packages available for upgrade (0 critical updates).
[01:59:26] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[01:59:59] <icinga-miraheze>	 RECOVERY - mw182 APT on mw182 is OK: APT OK: 185 packages available for upgrade (0 critical updates).
[02:00:28] <icinga-miraheze>	 RECOVERY - mw183 APT on mw183 is OK: APT OK: 185 packages available for upgrade (0 critical updates).
[02:00:42] <icinga-miraheze>	 RECOVERY - mwtask181 APT on mwtask181 is OK: APT OK: 192 packages available for upgrade (0 critical updates).
[02:05:27] <icinga-miraheze>	 PROBLEM - speedscope211 Current Load on speedscope211 is WARNING: LOAD WARNING - total load average: 0.05, 0.28, 7.53
[02:07:23] <icinga-miraheze>	 RECOVERY - speedscope211 Current Load on speedscope211 is OK: LOAD OK - total load average: 0.04, 0.21, 6.66
[04:01:41] <icinga-miraheze>	 PROBLEM - cp171 Varnish Backends on cp171 is CRITICAL: 3 backends are down. mw181 mw182 mw153
[04:03:41] <icinga-miraheze>	 RECOVERY - cp171 Varnish Backends on cp171 is OK: All 34 backends are healthy
[04:20:42] <MirahezeLSBot>	 !log [universalomega@test151] starting deploy of {'force_upgrade': True, 'versions': '1.46', 'upgrade_extensions': 'ScheduledCacheExpiry'} to test151
[04:20:47] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[04:20:51] <MirahezeLSBot>	 !log [universalomega@test151] finished deploy of {'force_upgrade': True, 'versions': '1.46', 'upgrade_extensions': 'ScheduledCacheExpiry'} to test151 - SUCCESS in 9s
[04:20:56] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[04:21:07] <MirahezeLSBot>	 !log [universalomega@test151] starting deploy of {'extension_list': True, 'versions': ['1.45', '1.46']} to test151
[04:21:09] <MirahezeLSBot>	 !log [universalomega@test151] finished deploy of {'extension_list': True, 'versions': ['1.45', '1.46']} to test151 - SUCCESS in 2s
[04:21:11] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[04:21:16] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[04:22:39] <MirahezeLSBot>	 !log [universalomega@test151] starting deploy of {'l10n': True, 'versions': '1.46'} to test151
[04:22:44] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[04:23:35] <MirahezeLSBot>	 !log [universalomega@test151] finished deploy of {'l10n': True, 'versions': '1.46'} to test151 - SUCCESS in 56s
[04:23:40] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[04:24:10] <MirahezeLSBot>	 !log [universalomega@test151] starting deploy of {'config': True} to test151
[04:24:11] <MirahezeLSBot>	 !log [universalomega@test151] finished deploy of {'config': True} to test151 - SUCCESS in 0s
[04:24:14] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[04:24:19] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:11:41] <icinga-miraheze>	 PROBLEM - cp171 Varnish Backends on cp171 is CRITICAL: 7 backends are down. mw161 mw171 mw181 mw182 mw183 mw201 mw202
[07:12:52] <icinga-miraheze>	 PROBLEM - cp201 Varnish Backends on cp201 is CRITICAL: 7 backends are down. mw171 mw182 mw153 mw183 mw201 mw202 mw203
[07:17:41] <icinga-miraheze>	 RECOVERY - cp171 Varnish Backends on cp171 is OK: All 34 backends are healthy
[07:18:52] <icinga-miraheze>	 RECOVERY - cp201 Varnish Backends on cp201 is OK: All 34 backends are healthy
[07:30:27] <MacFanBot>	 [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/df182720c5e6e3073e18940be3d4ddb17626075d
[07:30:27] <MacFanBot>	 02ssl/03main 07WikiTideBot 03df18272 Bot: Auto-update domain lists
[12:00:24] <MacFanBot>	 [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/6a33d97ae49d5ede3e3eb5ccd6a926e6599932b5
[12:00:24] <MacFanBot>	 02ssl/03main 07WikiTideBot 036a33d97 Bot: Auto-update domain lists
[12:02:43] <MacFanBot>	 [02IncidentReporting] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/IncidentReporting/commit/9f23a695878c63995dfc50c3181706561dfdf7cd
[12:02:44] <MacFanBot>	 02IncidentReporting/03main 07translatewiki.net 039f23a69 Localisation updates from https://translatewiki.net.
[12:02:45] <MacFanBot>	 [02ManageWiki] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/ManageWiki/commit/37a186125a1043dda0cbd71ef039d6a051834a30
[12:02:47] <MacFanBot>	 02ManageWiki/03main 07translatewiki.net 0337a1861 Localisation updates from https://translatewiki.net.
[12:07:07] <miraheze-github>	 miraheze/IncidentReporting - translatewiki the build passed.
[12:08:02] <miraheze-github>	 miraheze/ManageWiki - translatewiki the build passed.
[13:30:24] <MacFanBot>	 [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/b21f057c8875d39111cbdffd9380c9696dfe2bc1
[13:30:24] <MacFanBot>	 02ssl/03main 07WikiTideBot 03b21f057 Bot: Auto-update domain lists
[13:52:45] <MirahezeLSBot>	 !log [wwr@mwtask171] MediaWiki\Extension\CentralAuth\User\CentralAuthUser::getInstance( User::newFromName( 'TruncateVirus99' ) )->attach( 'theleaguenswiki', 'admin' );
[13:52:51] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[14:00:26] <MacFanBot>	 [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/daefeb05470cc6d6fbd743fe9943a4d1d4aba6ad
[14:00:26] <MacFanBot>	 02ssl/03main 07WikiTideBot 03daefeb0 Bot: Auto-update domain lists
[15:34:29] <MacFanBot>	 [02.github] 07MacFan4000 opened pull request #102: CI: remove 1.45 and mark 1.46 as stable (07miraheze:03main...07MacFan4000:03patch-2) 13https://github.com/miraheze/.github/pull/102
[15:44:45] <MirahezeRelay>	 <tsybangd> wat
[15:46:09] <icinga-miraheze>	 PROBLEM - cp171 Varnish Backends on cp171 is CRITICAL: 1 backends are down. mw153
[15:46:23] <icinga-miraheze>	 PROBLEM - cp201 Varnish Backends on cp201 is CRITICAL: 1 backends are down. mw183
[15:48:09] <icinga-miraheze>	 RECOVERY - cp171 Varnish Backends on cp171 is OK: All 34 backends are healthy
[15:48:22] <icinga-miraheze>	 RECOVERY - cp201 Varnish Backends on cp201 is OK: All 34 backends are healthy
[17:24:08] <MirahezeLSBot>	 !log [universalomega@mwtask181] starting deploy of {'force_upgrade': True, 'versions': '1.46', 'upgrade_extensions': 'ScheduledCacheExpiry'} to all
[17:24:12] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[17:25:01] <MirahezeLSBot>	 !log [universalomega@mwtask181] starting deploy of {'force_upgrade': True, 'versions': '1.46', 'upgrade_extensions': 'ScheduledCacheExpiry'} to all
[17:25:07] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[17:26:08] <MirahezeLSBot>	 !log [universalomega@mwtask181] finished deploy of {'force_upgrade': True, 'versions': '1.46', 'upgrade_extensions': 'ScheduledCacheExpiry'} to all - SUCCESS in 66s
[17:26:12] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[17:26:19] <MirahezeLSBot>	 !log [universalomega@mwtask181] starting deploy of {'extension_list': True, 'versions': ['1.45', '1.46']} to all
[17:26:24] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[17:27:34] <MirahezeLSBot>	 !log [universalomega@mwtask181] starting deploy of {'extension_list': True, 'versions': ['1.45', '1.46']} to all
[17:27:39] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[17:28:31] <MirahezeLSBot>	 !log [universalomega@mwtask181] finished deploy of {'extension_list': True, 'versions': ['1.45', '1.46']} to all - SUCCESS in 56s
[17:28:36] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[17:28:39] <MirahezeLSBot>	 !log [universalomega@mwtask181] starting deploy of {'l10n': True, 'versions': '1.46'} to all
[17:28:44] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[17:30:35] <MirahezeLSBot>	 !log [universalomega@mwtask181] starting deploy of {'l10n': True, 'versions': '1.46'} to all
[17:30:40] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[17:31:20] <MirahezeLSBot>	 !log [universalomega@mwtask181] finished deploy of {'l10n': True, 'versions': '1.46'} to all - SUCCESS in 45s
[17:31:25] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[17:32:27] <MacFanBot>	 [02mw-config] 07Universal-Omega merged pull request #6564: Install ScheduledCacheExpiry on 1.46 and restrict MagicNoCache (03main...03ScheduledCacheExpiry-1.46) 13https://github.com/miraheze/mw-config/pull/6564
[17:32:28] <MacFanBot>	 [02mw-config] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/d3b8af1a588fef49a241d83624388b64da2728dc
[17:32:28] <MacFanBot>	 02mw-config/03main 07CosmicAlpha 03d3b8af1 Install ScheduledCacheExpiry on 1.46 and restrict MagicNoCache (#6564)
[17:32:30] <MacFanBot>	 [02mw-config] 07Universal-Omega 04deleted 03ScheduledCacheExpiry-1.46 at 03e5d2b7d 13https://api.github.com/repos/miraheze/mw-config/commit/e5d2b7d
[17:32:43] <MirahezeLSBot>	 !log [universalomega@test151] starting deploy of {'pull': 'config', 'config': True} to test151
[17:32:44] <MirahezeLSBot>	 !log [universalomega@test151] finished deploy of {'pull': 'config', 'config': True} to test151 - SUCCESS in 1s
[17:32:48] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[17:32:53] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[17:32:54] <MirahezeLSBot>	 !log [universalomega@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all
[17:32:59] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[17:33:22] <MirahezeLSBot>	 !log [universalomega@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 28s
[17:33:27] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[17:33:55] <miraheze-github>	 miraheze/mw-config - Universal-Omega the build passed.
[18:00:24] <MacFanBot>	 [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/932a3814ad72a2b4cda83f1d18a3ebb3335a1b18
[18:00:25] <MacFanBot>	 02ssl/03main 07WikiTideBot 03932a381 Bot: Auto-update domain lists
[18:03:06] <icinga-miraheze>	 PROBLEM - db161 Current Load on db161 is CRITICAL: LOAD CRITICAL - total load average: 66.01, 31.34, 14.73
[18:55:51] <icinga-miraheze>	 PROBLEM - db161 Current Load on db161 is WARNING: LOAD WARNING - total load average: 3.15, 3.98, 11.93
[18:59:39] <icinga-miraheze>	 RECOVERY - db161 Current Load on db161 is OK: LOAD OK - total load average: 3.09, 3.54, 10.01
[19:37:24] <icinga-miraheze>	 PROBLEM - db161 Current Load on db161 is CRITICAL: LOAD CRITICAL - total load average: 24.25, 18.30, 11.16
[19:41:24] <icinga-miraheze>	 PROBLEM - db161 Current Load on db161 is WARNING: LOAD WARNING - total load average: 5.81, 11.87, 10.18
[19:43:24] <icinga-miraheze>	 RECOVERY - db161 Current Load on db161 is OK: LOAD OK - total load average: 5.03, 9.58, 9.55
[19:47:51] <MirahezeLSBot>	 !log [universalomega@mwtask181] starting deploy of {'l10n': True, 'versions': '1.46'} to all
[19:47:57] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[19:48:35] <MirahezeLSBot>	 !log [universalomega@mwtask181] finished deploy of {'l10n': True, 'versions': '1.46'} to all - SUCCESS in 43s
[19:48:40] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[19:55:58] <MacFanBot>	 [02python-functions] 07renovate[bot] pushed 1 new commit to 03main 13https://github.com/miraheze/python-functions/commit/8d2bc915183f9f33aad656d6b945fadd82dc5819
[19:55:58] <MacFanBot>	 02python-functions/03main 07renovate[bot] 038d2bc91 chore(deps): update dependency flake8 to v7.4.1
[19:55:59] <MacFanBot>	 [02python-functions] 07renovate[bot] 04deleted 03renovate/all-deps at 038d2bc91 13https://api.github.com/repos/miraheze/python-functions/commit/8d2bc91
[19:59:45] <miraheze-github>	 miraheze/python-functions - renovate[bot] the build passed.
[20:51:24] <icinga-miraheze>	 PROBLEM - db161 Current Load on db161 is WARNING: LOAD WARNING - total load average: 10.35, 10.00, 8.13
[20:57:24] <icinga-miraheze>	 RECOVERY - db161 Current Load on db161 is OK: LOAD OK - total load average: 9.09, 9.68, 8.69
[21:26:00] <icinga-miraheze>	 PROBLEM - db161 Current Load on db161 is WARNING: LOAD WARNING - total load average: 8.95, 10.52, 9.72
[21:27:53] <icinga-miraheze>	 RECOVERY - db161 Current Load on db161 is OK: LOAD OK - total load average: 5.60, 8.84, 9.20
[21:47:24] <icinga-miraheze>	 PROBLEM - db161 Current Load on db161 is CRITICAL: LOAD CRITICAL - total load average: 13.39, 10.13, 8.35
[21:49:24] <icinga-miraheze>	 RECOVERY - db161 Current Load on db161 is OK: LOAD OK - total load average: 9.53, 9.68, 8.40
[22:00:06] <icinga-miraheze>	 PROBLEM - db161 Current Load on db161 is WARNING: LOAD WARNING - total load average: 9.76, 11.56, 9.87
[22:02:00] <icinga-miraheze>	 RECOVERY - db161 Current Load on db161 is OK: LOAD OK - total load average: 7.14, 10.05, 9.51
[23:29:24] <icinga-miraheze>	 PROBLEM - db161 Current Load on db161 is CRITICAL: LOAD CRITICAL - total load average: 87.84, 42.46, 19.91
[23:30:05] <icinga-miraheze>	 PROBLEM - cp161 Varnish Backends on cp161 is CRITICAL: 4 backends are down. mw161 mw181 mw153 mw202
[23:30:14] <icinga-miraheze>	 PROBLEM - cp191 Varnish Backends on cp191 is CRITICAL: 4 backends are down. mw182 mw153 mw183 mw202
[23:30:41] <icinga-miraheze>	 PROBLEM - mw151 MediaWiki Rendering on mw151 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.012 second response time
[23:30:47] <MacFanBot>	 [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/cacf011e72a546c571e110ac892456019e7e3a1c
[23:30:48] <MacFanBot>	 02ssl/03main 07WikiTideBot 03cacf011 Bot: Auto-update domain lists
[23:31:06] <icinga-miraheze>	 PROBLEM - cp171 Varnish Backends on cp171 is CRITICAL: 2 backends are down. mw171 mw201
[23:31:07] <icinga-miraheze>	 PROBLEM - cp201 Varnish Backends on cp201 is CRITICAL: 3 backends are down. mw171 mw182 mw201
[23:31:26] <icinga-miraheze>	 PROBLEM - mw192 HTTPS on mw192 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502
[23:32:02] <icinga-miraheze>	 PROBLEM - mw162 HTTPS on mw162 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502
[23:32:12] <icinga-miraheze>	 PROBLEM - mw203 MediaWiki Rendering on mw203 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.013 second response time
[23:32:20] <icinga-miraheze>	 PROBLEM - mw191 MediaWiki Rendering on mw191 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.011 second response time
[23:32:44] <icinga-miraheze>	 RECOVERY - mw151 MediaWiki Rendering on mw151 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 5.509 second response time
[23:33:25] <icinga-miraheze>	 PROBLEM - mw172 HTTPS on mw172 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10004 milliseconds with 0 bytes received
[23:33:32] <MirahezeRelay>	 <posix_memalign> !log Depool db161
[23:34:12] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[23:34:23] <icinga-miraheze>	 RECOVERY - mw191 MediaWiki Rendering on mw191 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 7.675 second response time
[23:34:36] <MirahezeLSBot>	 !log [petramagna@mwtask181] starting deploy of {'config': True, 'force': True} to all
[23:34:56] <MirahezeLSBot>	 !log [petramagna@mwtask181] finished deploy of {'config': True, 'force': True} to all - SUCCESS in 20s
[23:35:12] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[23:35:19] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[23:35:21] <icinga-miraheze>	 RECOVERY - mw172 HTTPS on mw172 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.078 second response time
[23:35:30] <icinga-miraheze>	 RECOVERY - mw192 HTTPS on mw192 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.081 second response time
[23:36:00] <icinga-miraheze>	 RECOVERY - mw162 HTTPS on mw162 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.074 second response time
[23:36:05] <icinga-miraheze>	 RECOVERY - cp161 Varnish Backends on cp161 is OK: All 34 backends are healthy
[23:36:12] <icinga-miraheze>	 RECOVERY - mw203 MediaWiki Rendering on mw203 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.237 second response time
[23:36:14] <icinga-miraheze>	 RECOVERY - cp191 Varnish Backends on cp191 is OK: All 34 backends are healthy
[23:37:06] <icinga-miraheze>	 RECOVERY - cp171 Varnish Backends on cp171 is OK: All 34 backends are healthy
[23:37:07] <icinga-miraheze>	 RECOVERY - cp201 Varnish Backends on cp201 is OK: All 34 backends are healthy
[23:39:58] <MirahezeLSBot>	 !log [petramagna@mwtask181] starting deploy of {'config': True, 'force': True} to all
[23:40:04] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[23:40:11] <MirahezeRelay>	 <posix_memalign> !log Repooling db161
[23:40:15] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[23:40:18] <MirahezeLSBot>	 !log [petramagna@mwtask181] finished deploy of {'config': True, 'force': True} to all - SUCCESS in 19s
[23:40:23] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[23:59:59] <icinga-miraheze>	 PROBLEM - cp171 Disk Space on cp171 is WARNING: DISK WARNING - free space: / 45315MiB (10.0% inode=100%);