[00:04:16] PROBLEM - db112 Current Load on db112 is CRITICAL: CRITICAL - load average: 18.00, 9.82, 4.48 [00:14:16] PROBLEM - db112 Current Load on db112 is WARNING: WARNING - load average: 5.00, 7.38, 6.32 [00:14:24] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 6.95, 6.60, 5.99 [00:16:16] RECOVERY - db112 Current Load on db112 is OK: OK - load average: 3.41, 6.05, 5.96 [00:16:20] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.43, 6.24, 5.94 [00:16:20] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.63, 3.70, 3.35 [00:20:10] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.27, 3.70, 3.41 [00:22:05] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 2.85, 3.50, 3.39 [00:24:00] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.75, 3.33, 3.34 [00:34:43] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 2.77, 3.47, 3.42 [00:36:43] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.07, 3.67, 3.50 [00:38:43] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.75, 3.25, 3.36 [00:42:45] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.79, 3.66, 3.51 [00:46:43] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.50, 3.21, 3.38 [00:50:52] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.53, 6.96, 6.38 [00:54:43] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.90, 3.77, 3.54 [00:54:44] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.50, 6.68, 6.44 [00:56:43] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 5.02, 4.23, 3.74 [01:00:43] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.85, 3.99, 3.74 [01:07:10] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.57, 3.22, 2.87 [01:08:43] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.40, 2.92, 3.34 [01:09:10] RECOVERY - matomo121 Current Load on matomo121 is OK: LOAD OK - total load average: 2.22, 2.87, 2.79 [01:10:13] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 4.85, 3.41, 2.84 [01:10:39] PROBLEM - os141 PowerDNS Recursor on os141 is CRITICAL: CRITICAL - Plugin timed out while executing system call [01:12:09] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.41, 3.32, 2.87 [01:12:33] RECOVERY - os141 PowerDNS Recursor on os141 is OK: DNS OK: 0.136 seconds response time. miraheze.org returns 2001:41d0:801:2000::3a18,2001:41d0:801:2000::5d68,51.195.201.140,51.89.139.24 [01:14:05] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 2.98, 3.24, 2.90 [01:21:58] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.39, 7.63, 6.82 [01:23:53] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.62, 6.86, 6.63 [01:25:49] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.62, 6.73, 6.61 [01:28:22] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/603be80c89a5...dfbd637bc92a [01:28:25] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [01:28:25] [02miraheze/mw-config] 07paladox 03dfbd637 - Enable warmup on wikis beginning with o and p [01:28:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:28:38] !log [paladox@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 13s [01:28:41] !log [paladox@mwtask141] starting deploy of {'config': True} to all [01:28:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:28:50] !log [paladox@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 9s [01:28:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:28:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:29:20] miraheze/mw-config - paladox the build passed. [01:35:10] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.60, 3.10, 2.87 [01:37:10] RECOVERY - matomo121 Current Load on matomo121 is OK: LOAD OK - total load average: 2.64, 2.86, 2.81 [01:56:18] !log [@test131] starting deploy of {'config': True} to all [01:56:19] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [01:56:22] PROBLEM - mw131 Current Load on mw131 is WARNING: WARNING - load average: 11.58, 10.63, 9.09 [01:56:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:56:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:58:18] RECOVERY - mw131 Current Load on mw131 is OK: OK - load average: 8.74, 10.19, 9.13 [02:02:45] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.43, 3.23, 3.03 [02:04:43] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 1.84, 2.78, 2.90 [02:16:50] PROBLEM - mw131 Current Load on mw131 is CRITICAL: CRITICAL - load average: 12.81, 10.49, 9.48 [02:17:29] PROBLEM - mw132 Current Load on mw132 is WARNING: WARNING - load average: 9.05, 10.41, 9.56 [02:18:50] PROBLEM - mw131 Current Load on mw131 is WARNING: WARNING - load average: 9.81, 10.31, 9.54 [02:19:14] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/afb2a1766415...a5abd81676c1 [02:19:15] [02miraheze/puppet] 07paladox 03a5abd81 - jobrunner-hi: increase parseroidcache runners to 8 [02:20:50] RECOVERY - mw131 Current Load on mw131 is OK: OK - load average: 9.04, 9.85, 9.47 [02:21:20] RECOVERY - mw132 Current Load on mw132 is OK: OK - load average: 7.95, 9.54, 9.43 [02:23:00] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 9.83, 7.82, 6.67 [02:43:20] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.77, 3.58, 3.21 [02:45:15] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.65, 3.13, 3.08 [02:51:18] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/d9e8b6cd7add [02:51:20] [02miraheze/mw-config] 07paladox 03d9e8b6c - Enable warmup on wikis beginning with q and r [02:51:20] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.14, 6.63, 5.96 [02:51:21] [02mw-config] 07paladox created branch 03paladox-patch-4 - 13https://github.com/miraheze/mw-config [02:51:22] [02mw-config] 07paladox opened pull request 03#5366: Enable warmup on wikis beginning with q and r - 13https://github.com/miraheze/mw-config/pull/5366 [02:51:50] [02mw-config] 07paladox closed pull request 03#5366: Enable warmup on wikis beginning with q and r - 13https://github.com/miraheze/mw-config/pull/5366 [02:51:53] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/dfbd637bc92a...d677be6b989e [02:51:55] [02miraheze/mw-config] 07paladox 03d677be6 - Enable warmup on wikis beginning with q and r (#5366) [02:51:56] [02miraheze/mw-config] 07paladox deleted branch 03paladox-patch-4 [02:51:59] [02mw-config] 07paladox deleted branch 03paladox-patch-4 - 13https://github.com/miraheze/mw-config [02:52:43] miraheze/mw-config - paladox the build passed. [02:53:20] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.43, 6.04, 5.82 [02:55:55] !log [@test131] starting deploy of {'config': True} to all [02:55:56] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 1s [02:56:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:56:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:05:05] [02miraheze/mw-config] 07The-Voidwalker pushed 031 commit to 03The-Voidwalker-patch-1 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/35a5a72f2884 [03:05:08] [02miraheze/mw-config] 07The-Voidwalker 0335a5a72 - GlobalCache: remove unused code + simplify expressions [03:05:11] [02mw-config] 07The-Voidwalker created branch 03The-Voidwalker-patch-1 - 13https://github.com/miraheze/mw-config [03:05:13] [02mw-config] 07The-Voidwalker opened pull request 03#5367: GlobalCache: remove unused code + simplify expressions - 13https://github.com/miraheze/mw-config/pull/5367 [03:06:10] miraheze/mw-config - The-Voidwalker the build passed. [03:09:15] !log [@mwtask141] starting deploy of {'config': True} to all [03:09:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:09:38] !log [@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 23s [03:09:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:23:48] PROBLEM - mw132 Current Load on mw132 is CRITICAL: CRITICAL - load average: 12.33, 10.37, 8.35 [03:24:43] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.08, 3.61, 3.19 [03:25:44] PROBLEM - mw131 Current Load on mw131 is CRITICAL: CRITICAL - load average: 13.38, 10.60, 8.24 [03:26:43] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.80, 3.34, 3.14 [03:27:39] PROBLEM - mw132 Current Load on mw132 is WARNING: WARNING - load average: 10.64, 10.59, 8.88 [03:29:35] RECOVERY - mw132 Current Load on mw132 is OK: OK - load average: 8.49, 9.97, 8.86 [03:29:36] RECOVERY - mw131 Current Load on mw131 is OK: OK - load average: 8.04, 10.13, 8.62 [04:19:10] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.60, 2.90, 2.41 [04:21:10] RECOVERY - matomo121 Current Load on matomo121 is OK: LOAD OK - total load average: 2.69, 2.79, 2.44 [04:26:30] PROBLEM - mail121 Current Load on mail121 is CRITICAL: LOAD CRITICAL - total load average: 3.19, 1.71, 0.78 [04:32:03] PROBLEM - encyclopediarobotica.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'encyclopediarobotica.org' expires in 15 day(s) (Wed 22 Nov 2023 04:12:47 GMT +0000). [04:32:17] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/27e670872cbb...040c37ba6c0e [04:32:20] [02miraheze/ssl] 07MirahezeSSLBot 03040c37b - Bot: Update SSL cert for encyclopediarobotica.org [04:42:30] PROBLEM - mail121 Current Load on mail121 is WARNING: LOAD WARNING - total load average: 0.82, 1.45, 1.86 [04:44:43] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.51, 3.13, 2.64 [04:46:31] RECOVERY - ns1 NTP time on ns1 is OK: NTP OK: Offset 0.00413364172 secs [04:46:43] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 3.40, 3.27, 2.75 [04:48:30] RECOVERY - mail121 Current Load on mail121 is OK: LOAD OK - total load average: 0.51, 0.94, 1.50 [05:00:35] RECOVERY - encyclopediarobotica.org - LetsEncrypt on sslhost is OK: OK - Certificate 'encyclopediarobotica.org' will expire on Sun 04 Feb 2024 03:32:10 GMT +0000. [05:02:32] PROBLEM - mw132 Current Load on mw132 is WARNING: WARNING - load average: 10.88, 9.51, 8.09 [05:04:28] RECOVERY - mw132 Current Load on mw132 is OK: OK - load average: 8.50, 8.92, 8.04 [05:05:19] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.75, 3.97, 3.30 [05:11:05] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.59, 3.79, 3.44 [05:14:05] PROBLEM - archives.themarkside.com - reverse DNS on sslhost is WARNING: NXDOMAIN: The DNS query name does not exist: archives.themarkside.com. [05:14:55] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 3.00, 3.33, 3.33 [05:42:40] RECOVERY - archives.themarkside.com - reverse DNS on sslhost is OK: SSL OK - archives.themarkside.com reverse DNS resolves to cp25.miraheze.org - CNAME OK [05:57:32] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importImages.php --wiki=gogiganticwiki /home/macfan/file --search-recursively (START) [05:57:33] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importImages.php --wiki=gogiganticwiki /home/macfan/file --search-recursively (END - exit=0) [05:57:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:57:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:58:02] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importImages.php --wiki=gogiganticwiki /home/macfan/file --search-recursively (START) [05:58:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:08:14] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.90, 6.58, 6.17 [06:10:10] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.56, 6.59, 6.23 [06:14:04] PROBLEM - archives.themarkside.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for archives.themarkside.com could not be found [06:31:58] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importImages.php --wiki=gogiganticwiki /home/macfan/file --search-recursively (END - exit=0) [06:32:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:33:29] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.69, 7.20, 6.82 [06:39:20] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.17, 6.59, 6.74 [07:03:06] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 8.49, 6.93, 6.12 [07:05:02] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.76, 6.45, 6.04 [07:12:06] RECOVERY - archives.themarkside.com - reverse DNS on sslhost is OK: SSL OK - archives.themarkside.com reverse DNS resolves to cp24.miraheze.org - CNAME OK [08:07:10] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.77, 2.95, 1.84 [08:09:10] RECOVERY - matomo121 Current Load on matomo121 is OK: LOAD OK - total load average: 2.58, 2.81, 1.93 [08:14:31] PROBLEM - ns1 NTP time on ns1 is WARNING: NTP WARNING: Offset 0.1013953984 secs [08:51:20] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.71, 6.22, 5.65 [08:53:20] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.77, 6.40, 5.79 [09:13:34] RECOVERY - cloud11 IPMI Sensors on cloud11 is OK: IPMI Status: OK [09:17:34] PROBLEM - cloud11 IPMI Sensors on cloud11 is CRITICAL: IPMI Status: Critical [Cntlr 2 Bay 8 = Critical] [10:11:02] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.99, 3.46, 1.86 [10:12:57] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [10:13:02] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 2.93, 3.14, 1.94 [10:19:58] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 6.13, 4.91, 3.02 [10:25:45] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 0.46, 2.78, 2.75 [10:35:50] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 8.10, 7.19, 6.20 [10:37:50] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.10, 6.40, 6.03 [10:39:28] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.47, 3.56, 2.92 [10:41:23] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.05, 3.02, 2.80 [10:42:58] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['ns.ankh.fr.eu.org.', 'ns1.eu.org.', 'ns1.eriomem.net.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [12:00:07] [02mw-config] 07paladox closed pull request 03#5367: GlobalCache: remove unused code + simplify expressions - 13https://github.com/miraheze/mw-config/pull/5367 [12:00:10] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/d677be6b989e...9865f2f3d3d7 [12:00:11] [02miraheze/mw-config] 07The-Voidwalker 039865f2f - GlobalCache: remove unused code + simplify expressions (#5367) [12:00:11] [02miraheze/mw-config] 07paladox deleted branch 03The-Voidwalker-patch-1 [12:00:14] [02mw-config] 07paladox deleted branch 03The-Voidwalker-patch-1 - 13https://github.com/miraheze/mw-config [12:00:19] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [12:00:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:00:45] !log [paladox@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 26s [12:00:47] !log [paladox@mwtask141] starting deploy of {'config': True} to all [12:00:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:00:59] !log [paladox@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 11s [12:01:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:01:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:01:18] miraheze/mw-config - paladox the build passed. [12:17:53] [02miraheze/IncidentReporting] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/IncidentReporting/compare/4ffa95037d93...4af656d0d2a6 [12:17:56] [02miraheze/IncidentReporting] 07translatewiki 034af656d - Localisation updates from https://translatewiki.net. [12:17:58] [02miraheze/MirahezeMagic] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/e3681532d2f4...034e1b3005ad [12:18:00] [02miraheze/MirahezeMagic] 07translatewiki 03034e1b3 - Localisation updates from https://translatewiki.net. [12:18:02] [02miraheze/PDFEmbed] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/PDFEmbed/compare/1d7e32538216...03a1ac64a408 [12:18:03] [02miraheze/PDFEmbed] 07translatewiki 0303a1ac6 - Localisation updates from https://translatewiki.net. [12:18:05] [02miraheze/RemovePII] 07translatewiki pushed 031 commit to 03master [+1/-0/±0] 13https://github.com/miraheze/RemovePII/compare/493c4efe1651...1011733468aa [12:18:07] [02miraheze/RemovePII] 07translatewiki 031011733 - Localisation updates from https://translatewiki.net. [12:18:10] [02miraheze/SpriteSheet] 07translatewiki pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/SpriteSheet/compare/182f2b01b6c0...83160b65a1b9 [12:18:11] [02miraheze/SpriteSheet] 07translatewiki 0383160b6 - Localisation updates from https://translatewiki.net. [12:18:13] [02miraheze/DataDump] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/DataDump/compare/1880734a061e...34677c88667b [12:18:15] [02miraheze/DataDump] 07translatewiki 0334677c8 - Localisation updates from https://translatewiki.net. [12:24:22] miraheze/SpriteSheet - translatewiki the build passed. [12:24:55] miraheze/IncidentReporting - translatewiki the build passed. [12:25:15] miraheze/PDFEmbed - translatewiki the build passed. [12:26:33] !log [@test131] starting deploy of {'config': True} to all [12:26:34] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [12:26:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:26:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:29:33] miraheze/DataDump - translatewiki the build passed. [12:32:03] miraheze/MirahezeMagic - translatewiki the build passed. [12:33:48] miraheze/RemovePII - translatewiki the build passed. [12:34:09] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/9865f2f3d3d7...89a49228d9ba [12:34:10] [02miraheze/mw-config] 07paladox 0389a4922 - Remove useMicrosecondTimestamps(false) calls [12:35:10] miraheze/mw-config - paladox the build passed. [12:39:35] !log [@mwtask141] starting deploy of {'config': True} to all [12:39:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:39:45] !log [@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 10s [12:39:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:46:04] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/89a49228d9ba...3a987cb1cf1b [12:46:07] [02miraheze/mw-config] 07paladox 033a987cb - Set LoginNotify to info [12:46:26] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/3a987cb1cf1b...90b15de4a903 [12:46:29] [02miraheze/mw-config] 07paladox 0390b15de - Set LockManager to warning [12:46:58] miraheze/mw-config - paladox the build passed. [12:47:32] miraheze/mw-config - paladox the build passed. [12:48:11] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [12:48:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:48:22] !log [paladox@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 10s [12:48:26] !log [paladox@mwtask141] starting deploy of {'config': True} to all [12:48:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:48:35] !log [paladox@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 9s [12:48:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:48:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:51:07] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/90b15de4a903...91cfd9bf6334 [12:51:09] [02miraheze/mw-config] 07paladox 0391cfd9b - Set SwiftBackend to info [12:52:07] miraheze/mw-config - paladox the build has errored. [12:52:32] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/91cfd9bf6334...4ebd33bd94fb [12:52:34] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [12:52:35] [02miraheze/mw-config] 07paladox 034ebd33b - Fix [12:52:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:52:44] !log [paladox@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 10s [12:52:46] !log [paladox@mwtask141] starting deploy of {'config': True} to all [12:52:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:52:52] !log [paladox@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 6s [12:53:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:53:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:53:31] miraheze/mw-config - paladox the build passed. [12:56:23] !log [@test131] starting deploy of {'config': True} to all [12:56:24] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [12:56:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:56:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [13:04:16] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 4.00, 3.58, 3.17 [13:06:11] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 5.32, 4.21, 3.45 [13:09:31] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.67, 7.28, 6.56 [13:10:02] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 2.98, 3.74, 3.45 [13:10:26] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/a5abd81676c1...a42fca095b17 [13:10:28] [02miraheze/puppet] 07paladox 03a42fca0 - Revert "jobrunner-hi: increase parseroidcache runners to 8" [13:11:56] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.35, 3.24, 3.30 [13:15:20] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.73, 6.51, 6.45 [13:21:20] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.23, 7.11, 6.72 [13:21:37] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.27, 6.80, 6.33 [13:23:20] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.89, 6.33, 6.49 [13:23:33] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.78, 6.50, 6.28 [13:30:38] [Grafana] !sre FIRING: The mediawiki job queue has more than 2500 unclaimed jobs https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [13:33:00] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 6.09, 6.58, 7.96 [13:48:00] !log [paladox@mwtask141] starting deploy of {'world': True} to all [13:48:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [13:49:00] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 9.69, 7.73, 7.49 [13:49:34] !log [paladox@mwtask141] finished deploy of {'world': True} to all - SUCCESS in 93s [13:49:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [13:53:00] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 7.76, 7.93, 7.64 [13:57:34] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.51, 7.32, 6.71 [13:59:30] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.68, 6.61, 6.52 [14:05:36] PROBLEM - os141 PowerDNS Recursor on os141 is CRITICAL: CRITICAL - Plugin timed out while executing system call [14:06:50] Error 502/503 again [14:09:32] Uh oh [14:09:41] PROBLEM - wiki.iurevar.world - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:09:41] PROBLEM - txt-dex.com - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.404 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out. [14:09:41] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.34, 5.24, 6.56 [14:09:41] RECOVERY - os141 PowerDNS Recursor on os141 is OK: DNS OK: 0.095 seconds response time. miraheze.org returns 2001:41d0:801:2000::3a18,2001:41d0:801:2000::5d68,51.195.201.140,51.89.139.24 [14:09:41] PROBLEM - wiki.qadrishattari.xyz - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.401 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out. [14:10:34] PROBLEM - cp24 Puppet on cp24 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [14:22:23] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.93, 7.07, 6.63 [14:24:19] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.92, 6.37, 6.42 [14:37:09] PROBLEM - txt-dex.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - txt-dex.com reverse DNS resolves to any-in-2001-4860-4802-36--15.1e100.net [14:37:19] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/a1c026612554 [14:37:20] [02miraheze/mw-config] 07paladox 03a1c0266 - Set wgTemporaryParsoidHandlerParserCacheWriteRatio and disable warmup for commonswiki [14:37:21] [02mw-config] 07paladox created branch 03paladox-patch-4 - 13https://github.com/miraheze/mw-config [14:37:22] [02mw-config] 07paladox opened pull request 03#5368: Set wgTemporaryParsoidHandlerParserCacheWriteRatio and disable warmup… - 13https://github.com/miraheze/mw-config/pull/5368 [14:37:29] [02mw-config] 07paladox edited pull request 03#5368: Set wgTemporaryParsoidHandlerParserCacheWriteRatio and disable warmup for commonswiki - 13https://github.com/miraheze/mw-config/pull/5368 [14:37:32] [02mw-config] 07paladox edited pull request 03#5368: Set wgTemporaryParsoidHandlerParserCacheWriteRatio and disable warmup for commonswiki - 13https://github.com/miraheze/mw-config/pull/5368 [14:37:46] [02mw-config] 07paladox synchronize pull request 03#5368: Set wgTemporaryParsoidHandlerParserCacheWriteRatio and disable warmup for commonswiki - 13https://github.com/miraheze/mw-config/pull/5368 [14:37:47] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/a1c026612554...b39d7e0d8d55 [14:37:49] [02miraheze/mw-config] 07paladox 03b39d7e0 - Update LocalSettings.php [14:38:25] RECOVERY - wiki.qadrishattari.xyz - reverse DNS on sslhost is OK: SSL OK - wiki.qadrishattari.xyz reverse DNS resolves to cp24.miraheze.org - CNAME OK [14:38:28] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/b39d7e0d8d55...866d23e50755 [14:38:30] [02miraheze/mw-config] 07paladox 03866d23e - Update GlobalCache.php [14:38:31] [02mw-config] 07paladox synchronize pull request 03#5368: Set wgTemporaryParsoidHandlerParserCacheWriteRatio and disable warmup for commonswiki - 13https://github.com/miraheze/mw-config/pull/5368 [14:38:33] RECOVERY - cp24 Puppet on cp24 is OK: OK: Puppet is currently enabled, last run 39 seconds ago with 0 failures [14:38:53] miraheze/mw-config - paladox the build passed. [14:38:54] [02mw-config] 07paladox synchronize pull request 03#5368: Set wgTemporaryParsoidHandlerParserCacheWriteRatio and disable warmup for commonswiki - 13https://github.com/miraheze/mw-config/pull/5368 [14:38:56] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/866d23e50755...87c7c5d32950 [14:38:58] [02miraheze/mw-config] 07paladox 0387c7c5d - Update GlobalCache.php [14:39:42] miraheze/mw-config - paladox the build passed. [14:39:56] miraheze/mw-config - paladox the build passed. [14:39:56] [02mw-config] 07paladox closed pull request 03#5368: Set wgTemporaryParsoidHandlerParserCacheWriteRatio and disable warmup for commonswiki - 13https://github.com/miraheze/mw-config/pull/5368 [14:39:57] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/mw-config/compare/4ebd33bd94fb...fc63cacf8854 [14:40:00] [02miraheze/mw-config] 07paladox 03fc63cac - Set wgTemporaryParsoidHandlerParserCacheWriteRatio and disable warmup for commonswiki (#5368) [14:40:03] [02miraheze/mw-config] 07paladox deleted branch 03paladox-patch-4 [14:40:04] [02mw-config] 07paladox deleted branch 03paladox-patch-4 - 13https://github.com/miraheze/mw-config [14:40:13] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [14:40:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:40:25] !log [paladox@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 12s [14:40:29] !log [paladox@mwtask141] starting deploy of {'config': True} to all [14:40:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:40:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:40:37] !log [paladox@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 8s [14:40:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:41:03] miraheze/mw-config - paladox the build passed. [14:41:53] PROBLEM - www.pyramidgames.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'pyramidgames.wiki' expires in 15 day(s) (Wed 22 Nov 2023 14:28:59 GMT +0000). [14:53:29] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.85, 3.66, 3.15 [14:55:53] PROBLEM - os131 Current Load on os131 is WARNING: LOAD WARNING - total load average: 3.72, 3.23, 2.48 [14:56:08] !log [@test131] starting deploy of {'config': True} to all [14:56:09] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [14:56:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:56:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:57:51] PROBLEM - os131 Current Load on os131 is CRITICAL: LOAD CRITICAL - total load average: 4.28, 3.74, 2.77 [14:58:07] PROBLEM - pyramidgames.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'pyramidgames.wiki' expires in 15 day(s) (Wed 22 Nov 2023 14:28:59 GMT +0000). [14:58:21] PROBLEM - db131 Current Load on db131 is WARNING: WARNING - load average: 6.91, 5.42, 4.42 [14:59:13] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.26, 3.81, 3.36 [14:59:46] PROBLEM - os131 Current Load on os131 is WARNING: LOAD WARNING - total load average: 3.84, 3.80, 2.91 [15:00:21] RECOVERY - db131 Current Load on db131 is OK: OK - load average: 5.42, 5.43, 4.55 [15:01:08] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.88, 3.78, 3.40 [15:03:04] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.24, 4.00, 3.52 [15:03:38] RECOVERY - os131 Current Load on os131 is OK: LOAD OK - total load average: 2.28, 3.21, 2.88 [15:05:51] RECOVERY - wiki.iurevar.world - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.iurevar.world' will expire on Tue 19 Dec 2023 18:31:35 GMT +0000. [15:06:02] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.81, 7.41, 6.72 [15:09:05] PROBLEM - thenintendowiki.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'thenintendowiki.com' expires in 7 day(s) (Tue 14 Nov 2023 15:08:52 GMT +0000). [15:09:54] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.07, 6.79, 6.64 [15:10:43] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.26, 3.88, 3.71 [15:14:43] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.06, 3.85, 3.72 [15:16:43] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 2.78, 3.48, 3.60 [15:22:43] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.62, 2.87, 3.30 [15:39:02] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.84, 3.32, 2.39 [15:41:02] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 2.13, 2.91, 2.36 [15:41:51] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.34, 6.77, 6.39 [15:47:39] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.32, 8.22, 7.13 [15:49:35] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.74, 7.78, 7.10 [15:52:30] PROBLEM - swiftobject121 NTP time on swiftobject121 is CRITICAL: NTP CRITICAL: Offset 0.8708985448 secs [15:55:23] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.08, 6.60, 6.80 [15:59:20] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.27, 7.04, 6.93 [16:01:20] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.42, 6.36, 6.69 [16:01:47] !log [paladox@mwtask141] starting deploy of {'world': True} to all [16:01:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:02:58] !log [paladox@mwtask141] finished deploy of {'world': True} to all - SUCCESS in 71s [16:03:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:06:51] RECOVERY - swiftobject121 NTP time on swiftobject121 is OK: NTP OK: Offset -0.0004870295525 secs [16:10:06] [Grafana] !sre RESOLVED: High Job Queue Backlog https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [16:21:52] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.90, 6.83, 6.70 [16:23:48] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.54, 6.52, 6.62 [16:26:43] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 6.18, 7.03, 6.47 [16:28:39] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 4.75, 6.18, 6.22 [16:32:32] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.63, 6.78, 6.42 [16:36:23] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 4.70, 6.00, 6.22 [16:51:10] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.82, 3.46, 3.12 [16:53:10] RECOVERY - matomo121 Current Load on matomo121 is OK: LOAD OK - total load average: 2.66, 3.10, 3.02 [16:56:07] PROBLEM - pyramidgames.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address pyramidgames.wiki and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [17:05:35] [Grafana] !sre FIRING: The mediawiki job queue has more than 2500 unclaimed jobs https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [17:24:50] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.andreijiroh.uk.eu.org All nameservers failed to answer the query. [17:26:07] PROBLEM - pyramidgames.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'pyramidgames.wiki' expires in 15 day(s) (Wed 22 Nov 2023 14:28:59 GMT +0000). [17:51:33] PROBLEM - swiftobject121 NTP time on swiftobject121 is CRITICAL: NTP CRITICAL: Offset 0.6628784537 secs [17:53:38] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.eu.org. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [18:05:55] RECOVERY - swiftobject121 NTP time on swiftobject121 is OK: NTP OK: Offset -0.0006979703903 secs [18:12:58] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [18:22:49] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp24.miraheze.org - CNAME OK [18:42:59] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['ns.ankh.fr.eu.org.', 'ns1.eu.org.', 'ns1.eriomem.net.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [18:44:43] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.65, 3.07, 2.81 [18:46:43] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.64, 2.78, 2.73 [18:58:34] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.06, 6.89, 6.50 [19:00:30] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.14, 6.46, 6.37 [19:02:31] PROBLEM - ns1 NTP time on ns1 is CRITICAL: NTP CRITICAL: Offset 0.5006664097 secs [19:21:16] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importImages.php --wiki=gtwiki /home/macfan/file --search-recursively (START) [19:21:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:23:34] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.55, 3.29, 2.95 [19:23:43] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/fc63cacf8854...c3ec870c0726 [19:23:46] [02miraheze/mw-config] 07paladox 03c3ec870 - Disable warmup [19:24:44] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [19:24:51] miraheze/mw-config - paladox the build passed. [19:24:51] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:24:55] !log [paladox@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 10s [19:25:00] !log [paladox@mwtask141] starting deploy of {'config': True} to all [19:25:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:25:09] !log [paladox@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 8s [19:25:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:25:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:26:21] !log [@test131] starting deploy of {'config': True} to all [19:26:22] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [19:26:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:26:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:27:23] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.21, 3.68, 3.17 [19:29:18] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.02, 3.49, 3.17 [19:33:08] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.98, 3.19, 3.12 [19:35:21] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importImages.php --wiki=gtwiki /home/macfan/file --search-recursively (END - exit=256) [19:35:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:36:59] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.46, 3.44, 3.24 [19:38:53] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.59, 3.18, 3.17 [19:40:35] [Grafana] !sre RESOLVED: High Job Queue Backlog https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [19:46:21] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/3cfc6e10edbc [19:46:24] [02miraheze/mw-config] 07paladox 033cfc6e1 - Set $wgPoolCounterConf and use redis [19:46:27] [02mw-config] 07paladox created branch 03paladox-patch-4 - 13https://github.com/miraheze/mw-config [19:46:30] [02mw-config] 07paladox opened pull request 03#5369: Set $wgPoolCounterConf and use redis - 13https://github.com/miraheze/mw-config/pull/5369 [19:46:52] [02miraheze/mw-config] 07github-actions[bot] pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/3cfc6e10edbc...f6dfb90e4536 [19:46:54] [02miraheze/mw-config] 07github-actions 03f6dfb90 - CI: lint code to MediaWiki standards [19:46:57] [02mw-config] 07github-actions[bot] synchronize pull request 03#5369: Set $wgPoolCounterConf and use redis - 13https://github.com/miraheze/mw-config/pull/5369 [19:47:36] miraheze/mw-config - paladox the build passed. [19:47:45] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importImages.php --wiki=gqwiki /home/macfan/file/GQ_Audio --search-recursively (START) [19:47:46] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importImages.php --wiki=gqwiki /home/macfan/file/GQ_Audio --search-recursively (END - exit=0) [19:47:51] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:47:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:48:09] [02mw-config] 07paladox synchronize pull request 03#5369: Set $wgPoolCounterConf and use redis - 13https://github.com/miraheze/mw-config/pull/5369 [19:48:12] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/f6dfb90e4536...99b52b04af4d [19:48:15] [02miraheze/mw-config] 07paladox 0399b52b0 - Update GlobalSettings.php [19:48:17] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importImages.php --wiki=gqwiki /home/macfan/file/GQ Audio --search-recursively (START) [19:48:18] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importImages.php --wiki=gqwiki /home/macfan/file/GQ Audio --search-recursively (END - exit=0) [19:48:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:48:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:48:43] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importImages.php --wiki=gqwiki /home/macfan/file/'GQ Audio' --search-recursively (START) [19:48:44] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importImages.php --wiki=gqwiki /home/macfan/file/'GQ Audio' --search-recursively (END - exit=0) [19:48:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:48:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:49:02] PROBLEM - mw132 Current Load on mw132 is WARNING: WARNING - load average: 10.70, 9.17, 8.47 [19:49:10]