[00:02:14] PROBLEM - cp23 Current Load on cp23 is CRITICAL: LOAD CRITICAL - total load average: 7.07, 4.65, 2.16 [00:03:14] PROBLEM - cp22 Current Load on cp22 is CRITICAL: LOAD CRITICAL - total load average: 6.70, 5.67, 2.79 [00:04:14] PROBLEM - cp23 Current Load on cp23 is WARNING: LOAD WARNING - total load average: 1.67, 3.46, 2.03 [00:05:14] PROBLEM - cp22 Current Load on cp22 is WARNING: LOAD WARNING - total load average: 1.10, 3.86, 2.47 [00:06:14] RECOVERY - cp23 Current Load on cp23 is OK: LOAD OK - total load average: 1.84, 2.86, 1.97 [00:07:14] RECOVERY - cp22 Current Load on cp22 is OK: LOAD OK - total load average: 1.68, 3.29, 2.44 [00:13:14] PROBLEM - cp22 Current Load on cp22 is CRITICAL: LOAD CRITICAL - total load average: 6.80, 6.65, 4.12 [00:15:18] PROBLEM - os131 Current Load on os131 is WARNING: WARNING - load average: 3.53, 2.87, 1.99 [00:16:18] RECOVERY - cloud11 IPMI Sensors on cloud11 is OK: IPMI Status: OK [00:17:18] RECOVERY - os131 Current Load on os131 is OK: OK - load average: 3.39, 3.04, 2.16 [00:20:13] PROBLEM - cloud11 IPMI Sensors on cloud11 is CRITICAL: IPMI Status: Critical [Cntlr 2 Bay 8 = Critical] [00:23:30] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: CRITICAL - load average: 4.27, 3.33, 2.52 [00:25:14] PROBLEM - cp22 Current Load on cp22 is WARNING: LOAD WARNING - total load average: 0.63, 2.42, 3.81 [00:25:26] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.49, 3.18, 2.56 [00:27:14] RECOVERY - cp22 Current Load on cp22 is OK: LOAD OK - total load average: 0.40, 1.73, 3.38 [00:29:20] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 3.35, 3.37, 2.78 [00:33:06] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/5c6fc4b45e87...cd5de0261ee1 [00:33:07] [02miraheze/puppet] 07paladox 03cd5de02 - mediawiki::nutcracker: update ferm rules [00:35:38] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.63, 3.44, 2.46 [00:37:38] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.95, 3.45, 2.58 [00:45:15] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/cd5de0261ee1...59eeca39e1df [00:45:16] [02miraheze/puppet] 07paladox 0359eeca3 - nutcracker: lower timeout [00:45:42] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/59eeca39e1df...d2b9a477d7df [00:45:44] [02miraheze/puppet] 07paladox 03d2b9a47 - nutcracker: remove duplicates [00:49:38] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.10, 3.32, 3.03 [00:55:14] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.eu.org. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [00:58:33] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.67, 3.61, 3.29 [01:01:29] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/377d9749071a [01:01:30] [02miraheze/puppet] 07paladox 03377d974 - mediawiki::nutcracker: set 2 server connections [01:01:31] [02puppet] 07paladox created branch 03paladox-patch-4 - 13https://github.com/miraheze/puppet [01:01:34] [02puppet] 07paladox opened pull request 03#3387: mediawiki::nutcracker: set 2 server connections - 13https://github.com/miraheze/puppet/pull/3387 [01:01:41] [02puppet] 07paladox closed pull request 03#3387: mediawiki::nutcracker: set 2 server connections - 13https://github.com/miraheze/puppet/pull/3387 [01:01:43] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-4 [01:01:46] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/d2b9a477d7df...686bb0bf84d0 [01:01:47] [02miraheze/puppet] 07paladox 03686bb0b - mediawiki::nutcracker: set 2 server connections (#3387) [01:01:50] [02puppet] 07paladox deleted branch 03paladox-patch-4 - 13https://github.com/miraheze/puppet [01:09:43] PROBLEM - test131 nutcracker process on test131 is CRITICAL: PROCS CRITICAL: 0 processes with UID = 114 (nutcracker), command name 'nutcracker' [01:10:10] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.33, 3.87, 3.54 [01:12:07] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.32, 3.66, 3.50 [01:13:43] RECOVERY - test131 nutcracker process on test131 is OK: PROCS OK: 1 process with UID = 114 (nutcracker), command name 'nutcracker' [01:19:52] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.74, 3.27, 3.40 [01:23:46] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.77, 3.47, 3.44 [01:24:28] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp22.miraheze.org - CNAME OK [01:31:38] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.05, 3.34, 3.38 [01:35:38] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.72, 3.49, 3.42 [01:37:38] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.18, 3.25, 3.34 [01:39:16] PROBLEM - cloud11 NTP time on cloud11 is WARNING: NTP WARNING: Offset 0.1082511842 secs [01:41:38] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.37, 3.41, 3.39 [01:55:42] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.andreijiroh.uk.eu.org All nameservers failed to answer the query. [01:57:38] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.93, 3.29, 3.40 [02:01:16] RECOVERY - cloud11 NTP time on cloud11 is OK: NTP OK: Offset 0.07767516375 secs [02:23:38] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.95, 3.87, 3.50 [02:45:45] PROBLEM - cp32 NTP time on cp32 is CRITICAL: NTP CRITICAL: Offset 0.6986022294 secs [03:23:26] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.eu.org. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [03:38:09] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 9.98, 6.56, 4.08 [03:38:21] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 23.38, 15.05, 8.14 [03:38:28] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 12.22, 8.43, 5.01 [03:38:37] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 18.91, 12.12, 6.65 [03:38:38] PROBLEM - swiftproxy131 Current Load on swiftproxy131 is CRITICAL: CRITICAL - load average: 13.98, 6.68, 3.18 [03:38:42] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: CRITICAL - load average: 4.18, 3.14, 2.18 [03:40:38] RECOVERY - swiftproxy131 Current Load on swiftproxy131 is OK: OK - load average: 4.79, 5.51, 3.17 [03:40:42] PROBLEM - graylog131 Current Load on graylog131 is WARNING: WARNING - load average: 3.52, 3.32, 2.37 [03:42:10] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.61, 7.28, 4.96 [03:42:28] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 5.80, 7.38, 5.38 [03:42:42] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: CRITICAL - load average: 4.04, 3.52, 2.55 [03:44:09] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 10.42, 8.04, 5.49 [03:44:28] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.11, 7.39, 5.60 [03:44:42] RECOVERY - graylog131 Current Load on graylog131 is OK: OK - load average: 3.26, 3.40, 2.63 [03:46:10] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 6.14, 7.16, 5.48 [03:46:28] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 6.33, 7.30, 5.81 [03:46:39] PROBLEM - swiftproxy131 Current Load on swiftproxy131 is WARNING: WARNING - load average: 8.00, 6.89, 4.49 [03:48:09] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 6.13, 6.73, 5.51 [03:48:38] RECOVERY - swiftproxy131 Current Load on swiftproxy131 is OK: OK - load average: 3.76, 5.84, 4.40 [03:48:42] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: CRITICAL - load average: 4.24, 3.71, 2.92 [03:50:28] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.22, 7.68, 6.27 [03:50:42] PROBLEM - graylog131 Current Load on graylog131 is WARNING: WARNING - load average: 3.54, 3.78, 3.06 [03:52:28] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 6.73, 6.94, 6.15 [03:52:42] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: CRITICAL - load average: 4.03, 3.84, 3.16 [03:54:28] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 5.20, 6.12, 5.93 [03:54:42] PROBLEM - graylog131 Current Load on graylog131 is WARNING: WARNING - load average: 2.89, 3.46, 3.10 [03:58:37] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 3.86, 6.83, 7.92 [03:58:42] RECOVERY - graylog131 Current Load on graylog131 is OK: OK - load average: 2.25, 3.08, 3.06 [04:12:37] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 10.33, 7.68, 7.38 [04:14:38] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 6.08, 6.82, 7.10 [04:18:48] PROBLEM - wiki.zulaclub.net - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.zulaclub.net All nameservers failed to answer the query. [04:19:38] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.56, 3.54, 3.97 [04:20:37] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 11.61, 8.10, 7.44 [04:21:56] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.andreijiroh.uk.eu.org All nameservers failed to answer the query. [04:22:37] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.50, 7.64, 7.34 [04:24:22] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 4.74, 6.92, 7.77 [04:26:37] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 4.86, 6.16, 6.79 [04:38:21] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.70, 6.14, 6.75 [04:47:38] PROBLEM - wiki.zulaclub.net - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.zulaclub.net. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [04:51:38] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.35, 3.26, 3.40 [05:00:33] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.55, 3.36, 3.37 [05:02:29] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.28, 3.35, 3.37 [05:16:49] RECOVERY - wiki.zulaclub.net - reverse DNS on sslhost is OK: SSL OK - wiki.zulaclub.net reverse DNS resolves to cp22.miraheze.org - CNAME OK [05:20:27] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.eu.org. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [05:33:38] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.76, 3.45, 3.28 [05:43:38] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.11, 3.22, 3.28 [05:47:38] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.48, 3.69, 3.44 [05:49:38] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.51, 3.61, 3.44 [05:53:38] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.50, 3.21, 3.32 [05:57:31] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [06:05:38] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.82, 3.76, 3.44 [06:20:15] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.andreijiroh.uk.eu.org All nameservers failed to answer the query. [06:25:02] PROBLEM - cloud13 Puppet on cloud13 is UNKNOWN: NRPE: Unable to read output [06:26:02] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [06:28:14] PROBLEM - cp23 Current Load on cp23 is CRITICAL: LOAD CRITICAL - total load average: 13.26, 11.40, 5.16 [06:29:14] PROBLEM - cp22 Current Load on cp22 is CRITICAL: LOAD CRITICAL - total load average: 13.05, 13.18, 6.20 [06:34:14] PROBLEM - cp23 Current Load on cp23 is WARNING: LOAD WARNING - total load average: 0.50, 3.89, 3.71 [06:36:14] RECOVERY - cp23 Current Load on cp23 is OK: LOAD OK - total load average: 0.29, 2.68, 3.29 [06:37:14] PROBLEM - cp22 Current Load on cp22 is WARNING: LOAD WARNING - total load average: 0.34, 2.90, 3.81 [06:39:14] RECOVERY - cp22 Current Load on cp22 is OK: LOAD OK - total load average: 0.47, 2.09, 3.40 [06:50:15] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.eu.org. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [06:51:02] RECOVERY - cloud13 Puppet on cloud13 is OK: OK: Puppet is currently enabled, last run 41 seconds ago with 0 failures [07:17:38] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.70, 3.63, 3.98 [07:20:15] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.andreijiroh.uk.eu.org All nameservers failed to answer the query. [07:29:38] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.19, 3.61, 3.71 [07:33:38] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.06, 3.60, 3.71 [07:39:38] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.40, 3.69, 3.68 [07:41:38] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.35, 3.62, 3.67 [07:49:38] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.74, 3.95, 3.73 [07:50:15] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.eu.org. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [07:51:38] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.80, 3.83, 3.71 [07:59:38] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.87, 3.08, 3.40 [08:20:15] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.andreijiroh.uk.eu.org All nameservers failed to answer the query. [08:21:10] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.49, 3.24, 3.23 [08:23:06] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.45, 3.65, 3.38 [08:25:02] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.74, 3.71, 3.44 [08:28:55] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.01, 3.82, 3.54 [08:30:51] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.04, 3.55, 3.48 [08:32:47] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.57, 3.29, 3.40 [08:40:34] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.52, 3.34, 3.35 [08:42:31] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.17, 3.26, 3.32 [08:50:15] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp22.miraheze.org - CNAME OK [08:50:18] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.90, 3.66, 3.47 [08:54:10] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.35, 3.37, 3.39 [08:58:05] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.63, 3.51, 3.45 [09:01:57] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.12, 3.23, 3.35 [09:22:15] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.andreijiroh.uk.eu.org. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [09:57:19] RECOVERY - ns1 NTP time on ns1 is OK: NTP OK: Offset 0.007826298475 secs [10:03:38] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.28, 3.63, 3.07 [10:05:38] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.78, 3.36, 3.04 [10:11:38] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.57, 3.68, 3.28 [10:13:38] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.12, 3.94, 3.43 [10:23:38] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 2.98, 3.84, 3.75 [10:27:38] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.17, 3.87, 3.77 [10:43:38] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 2.45, 3.63, 3.85 [10:49:38] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 1.74, 2.51, 3.26 [10:50:15] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.andreijiroh.uk.eu.org All nameservers failed to answer the query. [11:15:43] [02miraheze/landing] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/landing/compare/a1446c765124...0ec72e79af31 [11:15:44] [02miraheze/landing] 07translatewiki 030ec72e7 - Localisation updates from https://translatewiki.net. [11:15:47] [02miraheze/ManageWiki] 07translatewiki pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/ManageWiki/compare/a5e33fe580be...67ef731aaf07 [11:15:49] [02miraheze/ManageWiki] 07translatewiki 0367ef731 - Localisation updates from https://translatewiki.net. [11:15:51] [02miraheze/SpriteSheet] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/SpriteSheet/compare/5ed5810cfd6e...97082c1e662a [11:15:53] [02miraheze/SpriteSheet] 07translatewiki 0397082c1 - Localisation updates from https://translatewiki.net. [11:15:54] [02miraheze/WikiDiscover] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/WikiDiscover/compare/fe67d07093cd...75f449917a5a [11:15:57] [02miraheze/WikiDiscover] 07translatewiki 0375f4499 - Localisation updates from https://translatewiki.net. [11:16:00] [02miraheze/MirahezeMagic] 07translatewiki pushed 031 commit to 03master [+0/-0/±3] 13https://github.com/miraheze/MirahezeMagic/compare/9136a84d955f...59f3d75ab76f [11:16:02] [02miraheze/MirahezeMagic] 07translatewiki 0359f3d75 - Localisation updates from https://translatewiki.net. [11:16:32] miraheze/landing - translatewiki the build passed. [11:20:16] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp23.miraheze.org - CNAME OK [11:21:22] miraheze/MirahezeMagic - translatewiki the build has errored. [11:22:27] miraheze/SpriteSheet - translatewiki the build passed. [11:22:46] miraheze/ManageWiki - translatewiki the build passed. [11:43:09] PROBLEM - db131 Backups SQL on db131 is WARNING: FILE_AGE WARNING: /var/log/sql-backup.log is 864142 seconds old and 100642 bytes [12:09:38] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 5.11, 3.53, 2.69 [12:15:38] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.90, 3.94, 3.17 [12:21:38] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.73, 3.97, 3.38 [12:27:38] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.82, 3.92, 3.58 [12:31:38] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.26, 3.91, 3.64 [12:33:38] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.87, 3.94, 3.68 [12:39:38] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.92, 4.02, 3.75 [12:47:38] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 2.78, 3.72, 3.79 [12:49:38] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.90, 4.18, 3.95 [12:51:38] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.37, 3.98, 3.92 [12:53:38] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.18, 4.13, 3.98 [12:55:38] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.77, 3.94, 3.93 [12:59:38] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.18, 3.96, 3.93 [13:05:38] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 2.39, 3.71, 3.90 [13:07:48] PROBLEM - cp22 NTP time on cp22 is CRITICAL: NTP CRITICAL: Offset 0.500050962 secs [13:11:38] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.71, 2.83, 3.40 [13:13:33] PROBLEM - wiki.altilunium.my.id - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.altilunium.my.id All nameservers failed to answer the query. [13:19:19] PROBLEM - ns1 NTP time on ns1 is WARNING: NTP WARNING: Offset 0.101472944 secs [13:43:33] PROBLEM - wiki.altilunium.my.id - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.altilunium.my.id could not be found [14:04:00] PROBLEM - wiki.gab.pt.eu.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.gab.pt.eu.org' expires in 15 day(s) (Wed 30 Aug 2023 13:58:38 GMT +0000). [14:04:23] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/ee48fd16b551...d47ab9dd1c20 [14:04:24] [02miraheze/ssl] 07MirahezeSSLBot 03d47ab9d - Bot: Update SSL cert for wiki.gab.pt.eu.org [15:52:59] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_40 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/e13cee9ab0c4...ebf3524bd4dc [15:53:02] [02miraheze/mediawiki] 07paladox 03ebf3524 - Update TitleKey [15:53:49] [02mw-config] 07paladox closed pull request 03#5298: Use HookContainer - 13https://github.com/miraheze/mw-config/pull/5298 [15:53:50] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/3805505a9a6e...87d4aa301935 [15:53:51] [02miraheze/mw-config] 07paladox 0387d4aa3 - Use HookContainer (#5298) [15:53:53] [02miraheze/mw-config] 07paladox deleted branch 03paladox-patch-1 [15:53:56] [02mw-config] 07paladox deleted branch 03paladox-patch-1 - 13https://github.com/miraheze/mw-config [15:54:14] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [15:54:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:54:24] !log [paladox@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 10s [15:54:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:54:32] !log [paladox@test131] starting deploy of {'pull': 'config', 'config': True} to all [15:54:33] !log [paladox@test131] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 1s [15:54:42] !log [paladox@mwtask141] starting deploy of {'config': True} to all [15:54:43] !log [paladox@test131] starting deploy of {'config': True} to all [15:54:44] !log [paladox@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [15:54:51] !log [paladox@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 8s [15:54:55] miraheze/mw-config - paladox the build passed. [15:55:11] !log [paladox@mwtask141] starting deploy of {'config': True} to all [15:55:11] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/87d4aa301935...8304b6798ca9 [15:55:12] [02miraheze/mw-config] 07paladox 038304b67 - Revert "Use HookContainer (#5298)" [15:55:15] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [15:55:23] !log [paladox@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 7s [15:55:36] !log [paladox@mwtask141] starting deploy of {'config': True} to all [15:55:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:55:43] !log [paladox@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 7s [15:55:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:56:13] miraheze/mw-config - paladox the build passed. [15:56:16] !log [@test131] starting deploy of {'config': True} to all [15:56:17] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [15:56:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:56:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:58:04] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/8304b6798ca9...c203da2812aa [15:58:06] [02miraheze/mw-config] 07paladox 03c203da2 - Revert "Revert "Use HookContainer (#5298)"" [15:58:08] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [15:58:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:58:16] !log [paladox@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 7s [15:58:20] !log [paladox@mwtask141] starting deploy of {'config': True} to all [15:58:28] !log [paladox@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 7s [15:58:38] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [15:58:38] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/c203da2812aa...831bfeedfe7d [15:58:39] [02miraheze/mw-config] 07paladox 03831bfee - Revert "Revert "Revert "Use HookContainer (#5298)""" [15:58:46] !log [paladox@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 8s [15:58:48] !log [paladox@mwtask141] starting deploy of {'config': True} to all [15:58:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:58:55] !log [paladox@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 7s [15:59:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:59:14] miraheze/mw-config - paladox the build passed. [15:59:33] miraheze/mw-config - paladox the build passed. [16:04:38] !log [paladox@test131] starting deploy of {'pull': 'config', 'config': True} to all [16:04:40] !log [paladox@test131] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 1s [16:04:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:04:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:05:09] !log [paladox@test131] starting deploy of {'world': True, 'l10n': True} to all [16:05:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:07:32] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/d10f9eac6240 [16:07:34] [02miraheze/mw-config] 07paladox 03d10f9ea - Make sure $databases is an array [16:07:35] [02mw-config] 07paladox created branch 03paladox-patch-1 - 13https://github.com/miraheze/mw-config [16:07:38] [02mw-config] 07paladox opened pull request 03#5302: Make sure $databases is an array - 13https://github.com/miraheze/mw-config/pull/5302 [16:07:51] [02mw-config] 07paladox closed pull request 03#5302: Make sure $databases is an array - 13https://github.com/miraheze/mw-config/pull/5302 [16:07:54] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/831bfeedfe7d...23fbeb5afff1 [16:07:55] [02miraheze/mw-config] 07paladox 0323fbeb5 - Make sure $databases is an array (#5302) [16:07:58] [02miraheze/mw-config] 07paladox deleted branch 03paladox-patch-1 [16:08:01] [02mw-config] 07paladox deleted branch 03paladox-patch-1 - 13https://github.com/miraheze/mw-config [16:08:47] miraheze/mw-config - paladox the build passed. [16:08:56] miraheze/mw-config - paladox the build passed. [16:09:21] !log [@mwtask141] starting deploy of {'config': True} to all [16:09:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:09:31] !log [@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 9s [16:09:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:12:52] !log [paladox@test131] finished deploy of {'world': True, 'l10n': True} to all - SUCCESS in 463s [16:12:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:14:03] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: CRITICAL - load average: 4.35, 3.67, 2.63 [16:16:03] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.07, 3.43, 2.66 [16:23:31] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [16:26:03] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 3.37, 3.38, 3.04 [16:26:40] !log [@test131] starting deploy of {'config': True} to all [16:26:41] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [16:26:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:26:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:32:33] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.45, 3.38, 2.92 [16:34:37] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_40 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/ebf3524bd4dc...e4d1d7b88db7 [16:34:38] [02miraheze/mediawiki] 07paladox 03e4d1d7b - Update Flatbox to bd6851996d23db005c64aebb64dda7651de6df5b [16:34:48] !log [paladox@test131] starting deploy of {'world': True} to all [16:34:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:36:03] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.41, 3.22, 3.03 [16:36:26] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.14, 3.39, 3.03 [16:38:03] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 2.95, 3.16, 3.03 [16:39:22] !log [paladox@test131] finished deploy of {'world': True} to all - SUCCESS in 274s [16:39:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:49:15] PROBLEM - www.pomologia.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.pomologia.org' expires in 15 day(s) (Wed 30 Aug 2023 16:40:21 GMT +0000). [16:52:08] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/d47ab9dd1c20...6ff9e01c439e [16:52:11] [02miraheze/ssl] 07MirahezeSSLBot 036ff9e01 - Bot: Update SSL cert for www.pomologia.org [16:52:32] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [17:02:46] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+16/-3/±40] 13https://github.com/miraheze/puppet/compare/686bb0bf84d0...02f0555d8bf7 [17:02:48] [02miraheze/puppet] 07paladox 0302f0555 - Update java module [17:10:05] PROBLEM - db131 Puppet on db131 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 9 minutes ago with 0 failures [17:10:08] PROBLEM - mwtask141 Puppet on mwtask141 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 31 minutes ago with 0 failures [17:10:08] PROBLEM - mem131 Puppet on mem131 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 15 minutes ago with 0 failures [17:10:11] PROBLEM - db112 Puppet on db112 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 8 minutes ago with 0 failures [17:10:11] PROBLEM - mem141 Puppet on mem141 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 27 minutes ago with 0 failures [17:10:12] PROBLEM - mw134 Puppet on mw134 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 4 minutes ago with 0 failures [17:10:20] PROBLEM - os131 Puppet on os131 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 5 minutes ago with 0 failures [17:10:30] PROBLEM - cloud11 Puppet on cloud11 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 20 minutes ago with 0 failures [17:10:33] PROBLEM - bast121 Puppet on bast121 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 30 minutes ago with 0 failures [17:10:33] PROBLEM - bast141 Puppet on bast141 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 12 minutes ago with 0 failures [17:10:34] PROBLEM - prometheus131 Puppet on prometheus131 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 17 minutes ago with 0 failures [17:10:34] PROBLEM - swiftobject101 Puppet on swiftobject101 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 9 minutes ago with 0 failures [17:10:35] PROBLEM - ns2 Puppet on ns2 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 22 minutes ago with 0 failures [17:10:35] PROBLEM - ldap141 Puppet on ldap141 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 7 minutes ago with 0 failures [17:10:40] PROBLEM - test131 Puppet on test131 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 13 minutes ago with 0 failures [17:10:41] PROBLEM - cp33 Puppet on cp33 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 18 minutes ago with 0 failures [17:10:41] PROBLEM - mw141 Puppet on mw141 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 5 minutes ago with 0 failures [17:10:46] PROBLEM - swiftproxy131 Puppet on swiftproxy131 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 3 minutes ago with 0 failures [17:10:46] PROBLEM - swiftobject113 Puppet on swiftobject113 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 27 minutes ago with 0 failures [17:10:50] PROBLEM - swiftproxy111 Puppet on swiftproxy111 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 28 minutes ago with 0 failures [17:10:52] PROBLEM - mw143 Puppet on mw143 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 7 minutes ago with 0 failures [17:10:53] PROBLEM - jobchron121 Puppet on jobchron121 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 24 minutes ago with 0 failures [17:10:57] PROBLEM - mw142 Puppet on mw142 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 10 minutes ago with 0 failures [17:11:02] PROBLEM - cloud13 Puppet on cloud13 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 21 minutes ago with 0 failures [17:11:04] PROBLEM - os141 Puppet on os141 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 4 minutes ago with 0 failures [17:11:05] PROBLEM - db121 Puppet on db121 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 22 minutes ago with 0 failures [17:11:06] PROBLEM - mw133 Puppet on mw133 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 12 minutes ago with 0 failures [17:11:07] PROBLEM - mw131 Puppet on mw131 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 30 minutes ago with 0 failures [17:11:09] PROBLEM - db142 Puppet on db142 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 31 minutes ago with 0 failures [17:11:11] PROBLEM - cloud10 Puppet on cloud10 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 32 minutes ago with 0 failures [17:11:13] PROBLEM - ns1 Puppet on ns1 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 16 minutes ago with 0 failures [17:11:13] PROBLEM - mail121 Puppet on mail121 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 28 minutes ago with 0 failures [17:11:16] PROBLEM - matomo121 Puppet on matomo121 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 5 minutes ago with 0 failures [17:11:19] PROBLEM - graylog131 Puppet on graylog131 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 6 minutes ago with 0 failures [17:11:21] PROBLEM - db101 Puppet on db101 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 20 minutes ago with 0 failures [17:11:23] PROBLEM - swiftac111 Puppet on swiftac111 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 18 minutes ago with 0 failures [17:11:29] PROBLEM - cp23 Puppet on cp23 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 15 minutes ago with 0 failures [17:11:32] PROBLEM - cp32 Puppet on cp32 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 7 minutes ago with 0 failures [17:11:34] PROBLEM - puppet141 Puppet on puppet141 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 3 minutes ago with 0 failures [17:11:37] PROBLEM - swiftobject122 Puppet on swiftobject122 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 21 minutes ago with 0 failures [17:11:38] PROBLEM - cloud14 Puppet on cloud14 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 20 minutes ago with 0 failures [17:11:38] PROBLEM - mon141 Puppet on mon141 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 18 minutes ago with 0 failures [17:11:41] PROBLEM - cloud12 Puppet on cloud12 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 32 minutes ago with 0 failures [17:11:45] PROBLEM - phab121 Puppet on phab121 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 3 minutes ago with 0 failures [17:11:48] PROBLEM - mw132 Puppet on mw132 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 6 minutes ago with 0 failures [17:11:48] PROBLEM - cp22 Puppet on cp22 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 29 minutes ago with 0 failures [17:11:50] PROBLEM - reports121 Puppet on reports121 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 14 minutes ago with 0 failures [17:11:57] PROBLEM - swiftobject121 Puppet on swiftobject121 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 22 minutes ago with 0 failures [17:12:07] PROBLEM - swiftobject112 Puppet on swiftobject112 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 26 minutes ago with 0 failures [17:12:36] PROBLEM - swiftobject111 Puppet on swiftobject111 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 7 minutes ago with 0 failures [17:14:07] RECOVERY - Host puppetdb121 is UP: PING OK - Packet loss = 0%, RTA = 1.04 ms [17:14:27] RECOVERY - puppetdb121 ferm_active on puppetdb121 is OK: OK ferm input default policy is set [17:15:34] RECOVERY - puppetdb121 SSH on puppetdb121 is OK: SSH OK - OpenSSH_8.4p1 Debian-5+deb11u1 (protocol 2.0) [17:16:44] miraheze/WikiDiscover - translatewiki the build has errored. [17:23:20] RECOVERY - puppetdb121 APT on puppetdb121 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [17:23:31] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - uk.religiononfire.mar.in.ua All nameservers failed to answer the query. [17:39:33] PROBLEM - puppetdb121 Puppet on puppetdb121 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [17:40:14] PROBLEM - puppetdb121 APT on puppetdb121 is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. [17:42:16] PROBLEM - puppetdb121 APT on puppetdb121 is CRITICAL: APT CRITICAL: 167 packages available for upgrade (3 critical updates). [17:44:18] RECOVERY - puppetdb121 APT on puppetdb121 is OK: APT OK: 93 packages available for upgrade (0 critical updates). [17:48:32] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/02f0555d8bf7...20ec7b3b968e [17:48:35] [02miraheze/puppet] 07paladox 0320ec7b3 - Update init.pp [17:52:18] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_40 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/e4d1d7b88db7...3a79cca08e65 [17:52:21] [02miraheze/mediawiki] 07paladox 033a79cca - Update Interwiki [17:52:32] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [17:53:12] !log [paladox@test131] starting deploy of {'world': True} to all [17:53:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:57:34] !log [paladox@test131] finished deploy of {'world': True} to all - SUCCESS in 262s [17:57:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:01:03] [02miraheze/CreateWiki] 07MacFan4000 pushed 031 commit to 03revert-410-revert-409-MacFan4000-patch-1 [+0/-0/±3] 13https://github.com/miraheze/CreateWiki/commit/ffe50245f840 [18:01:06] [02miraheze/CreateWiki] 07MacFan4000 03ffe5024 - Revert "Revert "Logging: respect $this->plaintext" (#410)" [18:01:09] [02CreateWiki] 07MacFan4000 created branch 03revert-410-revert-409-MacFan4000-patch-1 - 13https://github.com/miraheze/CreateWiki [18:01:56] [02CreateWiki] 07MacFan4000 opened pull request 03#433: Revert "Revert "Logging: respect $this->plaintext" (#410)" - 13https://github.com/miraheze/CreateWiki/pull/433 [18:09:19] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/20ec7b3b968e...cc299e7c278f [18:09:21] [02miraheze/puppet] 07paladox 03cc299e7 - Update master.pp [18:09:57] miraheze/CreateWiki - MacFan4000 the build has errored. [18:11:43] [02CreateWiki] 07MacFan4000 closed pull request 03#433: Revert "Revert "Logging: respect $this->plaintext" (#410)" - 13https://github.com/miraheze/CreateWiki/pull/433 [18:11:43] PROBLEM - puppetdb121 Puppet on puppetdb121 is UNKNOWN: NRPE: Unable to read output [18:11:46] [02miraheze/CreateWiki] 07MacFan4000 deleted branch 03revert-410-revert-409-MacFan4000-patch-1 [18:11:48] [02miraheze/CreateWiki] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±3] 13https://github.com/miraheze/CreateWiki/compare/14dba174f5b4...dfdd08deb438 [18:11:50] [02miraheze/CreateWiki] 07MacFan4000 03dfdd08d - Revert "Revert "Logging: respect $this->plaintext" (#410)" (#433) [18:11:52] [02CreateWiki] 07MacFan4000 deleted branch 03revert-410-revert-409-MacFan4000-patch-1 - 13https://github.com/miraheze/CreateWiki [18:15:38] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.23, 3.78, 2.87 [18:17:38] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.61, 3.69, 2.95 [18:18:51] miraheze/CreateWiki - MacFan4000 the build has errored. [18:25:38] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.73, 3.32, 3.10 [18:32:33] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.16, 3.42, 3.25 [18:38:42] PROBLEM - puppetdb121 APT on puppetdb121 is CRITICAL: APT CRITICAL: 10 packages available for upgrade (10 critical updates). [18:40:18] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.17, 3.75, 3.46 [18:40:58] RECOVERY - puppetdb121 APT on puppetdb121 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [18:44:10] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.70, 3.87, 3.59 [18:49:59] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.14, 3.90, 3.66 [18:55:48] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 2.82, 3.79, 3.75 [18:56:38] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/45f1b29caca6 [18:56:41] [02miraheze/puppet] 07paladox 0345f1b29 - Set postgresql::master::locale to 'en_GB.UTF-8' [18:56:43] [02puppet] 07paladox created branch 03paladox-patch-4 - 13https://github.com/miraheze/puppet [18:56:45] [02puppet] 07paladox opened pull request 03#3388: Set postgresql::master::locale to 'en_GB.UTF-8' - 13https://github.com/miraheze/puppet/pull/3388 [18:56:57] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/45f1b29caca6...b58f3ca91388 [18:57:00] [02miraheze/puppet] 07paladox 03b58f3ca - Update puppet141.yaml [18:57:03] [02puppet] 07paladox synchronize pull request 03#3388: Set postgresql::master::locale to 'en_GB.UTF-8' - 13https://github.com/miraheze/puppet/pull/3388 [18:57:06] [02puppet] 07paladox closed pull request 03#3388: Set postgresql::master::locale to 'en_GB.UTF-8' - 13https://github.com/miraheze/puppet/pull/3388 [18:57:08] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/puppet/compare/cc299e7c278f...6aa8a4826845 [18:57:09] [02miraheze/puppet] 07paladox 036aa8a48 - Set postgresql::master::locale to 'en_GB.UTF-8' (#3388) [18:57:12] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-4 [18:57:15] [02puppet] 07paladox deleted branch 03paladox-patch-4 - 13https://github.com/miraheze/puppet [18:59:40] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.42, 3.95, 3.81 [19:01:38] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 2.04, 3.45, 3.66 [19:03:40] PROBLEM - puppetdb121 puppetdb on puppetdb121 is CRITICAL: connect to address 2a10:6740::6:313 and port 8081: Connection refused [19:07:35] RECOVERY - puppetdb121 puppetdb on puppetdb121 is OK: TCP OK - 0.001 second response time on 2a10:6740::6:313 port 8081 [19:09:38] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.04, 3.11, 3.38 [19:11:50] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/6aa8a4826845...eeada1ac9253 [19:11:53] [02miraheze/puppet] 07paladox 03eeada1a - postgresql: partially fix master [19:15:38] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.78, 3.58, 3.51 [19:16:52] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/33cc7915659d [19:16:53] [02miraheze/puppet] 07paladox 0333cc791 - postgresql: add support for pg 15 in master.conf [19:16:54] [02puppet] 07paladox opened pull request 03#3389: postgresql: add support for pg 15 in master.conf - 13https://github.com/miraheze/puppet/pull/3389 [19:16:56] [02puppet] 07paladox created branch 03paladox-patch-4 - 13https://github.com/miraheze/puppet [19:16:58] [02puppet] 07paladox closed pull request 03#3389: postgresql: add support for pg 15 in master.conf - 13https://github.com/miraheze/puppet/pull/3389 [19:17:01] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/eeada1ac9253...bd7e472fa0ec [19:17:02] [02miraheze/puppet] 07paladox 03bd7e472 - postgresql: add support for pg 15 in master.conf (#3389) [19:17:03] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-4 [19:17:06] [02puppet] 07paladox deleted branch 03paladox-patch-4 - 13https://github.com/miraheze/puppet [19:21:31] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [19:23:33] PROBLEM - puppetdb121 Puppet on puppetdb121 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [19:26:18] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/bd27f1b9025a [19:26:20] [02miraheze/puppet] 07paladox 03bd27f1b - postgresql: Fix "Detail: comparison of String with 13 failed" [19:26:23] [02puppet] 07paladox created branch 03paladox-patch-4 - 13https://github.com/miraheze/puppet [19:26:26] [02puppet] 07paladox opened pull request 03#3390: postgresql: Fix "Detail: comparison of String with 13 failed" - 13https://github.com/miraheze/puppet/pull/3390 [19:27:12] [02puppet] 07paladox synchronize pull request 03#3390: postgresql: Fix "Detail: comparison of String with 13 failed" - 13https://github.com/miraheze/puppet/pull/3390 [19:27:15] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/bd27f1b9025a...b881b8f06ff0 [19:27:16] [02miraheze/puppet] 07paladox 03b881b8f - Update master.pp [19:27:38] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.34, 3.30, 3.39 [19:30:26] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/b881b8f06ff0...fae6bd616021 [19:30:29] [02miraheze/puppet] 07paladox 03fae6bd6 - Update server.pp [19:30:30] [02puppet] 07paladox synchronize pull request 03#3390: postgresql: Fix "Detail: comparison of String with 13 failed" - 13https://github.com/miraheze/puppet/pull/3390 [19:31:22] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/fae6bd616021...10f6c889f60d [19:31:24] [02miraheze/puppet] 07paladox 0310f6c88 - Update dirs.pp [19:31:27] [02puppet] 07paladox synchronize pull request 03#3390: postgresql: Fix "Detail: comparison of String with 13 failed" - 13https://github.com/miraheze/puppet/pull/3390 [19:31:37] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/10f6c889f60d...ef84106c9f10 [19:31:38] [02miraheze/puppet] 07paladox 03ef84106 - Update master.conf.erb [19:31:40] [02puppet] 07paladox synchronize pull request 03#3390: postgresql: Fix "Detail: comparison of String with 13 failed" - 13https://github.com/miraheze/puppet/pull/3390 [19:31:47] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/ef84106c9f10...7aba3b8f613b [19:31:48] [02miraheze/puppet] 07paladox 037aba3b8 - Update ssl.conf.erb [19:31:49] [02puppet] 07paladox synchronize pull request 03#3390: postgresql: Fix "Detail: comparison of String with 13 failed" - 13https://github.com/miraheze/puppet/pull/3390 [19:32:09] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/7aba3b8f613b...ee42b1cda132 [19:32:11] [02miraheze/puppet] 07paladox 03ee42b1c - Update postgresql.conf.erb [19:32:14] [02puppet] 07paladox synchronize pull request 03#3390: postgresql: Fix "Detail: comparison of String with 13 failed" - 13https://github.com/miraheze/puppet/pull/3390 [19:35:04] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/ee42b1cda132...3c08fdba6817 [19:35:07] [02miraheze/puppet] 07paladox 033c08fdb - Update user.pp [19:35:10] [02puppet] 07paladox synchronize pull request 03#3390: postgresql: Fix "Detail: comparison of String with 13 failed" - 13https://github.com/miraheze/puppet/pull/3390 [19:36:28] [02puppet] 07paladox closed pull request 03#3390: postgresql: Fix "Detail: comparison of String with 13 failed" - 13https://github.com/miraheze/puppet/pull/3390 [19:36:31] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±7] 13https://github.com/miraheze/puppet/compare/bd7e472fa0ec...402602172c60 [19:36:32] [02miraheze/puppet] 07paladox 034026021 - postgresql: Fix "Detail: comparison of String with 13 failed" (#3390) [19:36:33] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-4 [19:36:34] [02puppet] 07paladox deleted branch 03paladox-patch-4 - 13https://github.com/miraheze/puppet [19:37:33] PROBLEM - puppetdb121 Puppet on puppetdb121 is UNKNOWN: NRPE: Unable to read output [19:41:38] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.42, 3.37, 3.34 [19:43:38] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.67, 3.18, 3.28 [19:51:45] PROBLEM - cp32 NTP time on cp32 is WARNING: NTP WARNING: Offset 0.4998770952 secs [19:53:31] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [20:22:32] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [20:23:04] [02miraheze/puppet] 07paladox pushed 031 commit to 03update-augeas-core [+53/-31/±12] 13https://github.com/miraheze/puppet/commit/2adb81233b17 [20:23:07] [02miraheze/puppet] 07paladox 032adb812 - Update augeas_core to 4.0.1 [20:23:08] [02puppet] 07paladox created branch 03update-augeas-core - 13https://github.com/miraheze/puppet [20:23:13] [02puppet] 07paladox opened pull request 03#3391: Update augeas_core to 4.0.1 - 13https://github.com/miraheze/puppet/pull/3391 [20:25:31] [02puppet] 07paladox closed pull request 03#3391: Update augeas_core to 4.0.1 - 13https://github.com/miraheze/puppet/pull/3391 [20:25:34] [02miraheze/puppet] 07paladox deleted branch 03update-augeas-core [20:25:36] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+53/-31/±12] 13https://github.com/miraheze/puppet/compare/402602172c60...48e33bc95fff [20:25:38] [02miraheze/puppet] 07paladox 0348e33bc - Update augeas_core to 4.0.1 (#3391) [20:25:40] [02puppet] 07paladox deleted branch 03update-augeas-core - 13https://github.com/miraheze/puppet [20:27:31] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [20:49:33] RECOVERY - puppetdb121 Puppet on puppetdb121 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [20:56:02] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [21:33:31] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.04037797451 secs [21:49:21] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+31/-53/±12] 13https://github.com/miraheze/puppet/compare/48e33bc95fff...aa2689d01a60 [21:49:24] [02miraheze/puppet] 07paladox 03aa2689d - Revert "Update augeas_core to 4.0.1" [21:52:27] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+8/-2/±22] 13https://github.com/miraheze/puppet/compare/aa2689d01a60...52a492900c0f [21:52:28] [02miraheze/puppet] 07paladox 0352a4929 - Update augeas_core to 1.3.0 [21:55:38] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+19/-6/±19] 13https://github.com/miraheze/puppet/compare/52a492900c0f...bdd5ffcb02a6 [21:55:41] [02miraheze/puppet] 07paladox 03bdd5ffc - Update augeasproviders_core to 4.0.1 [22:13:45] PROBLEM - cp32 NTP time on cp32 is CRITICAL: NTP CRITICAL: Offset 0.6245486736 secs [22:15:48] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset -0.01095995307 secs [22:17:31] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.2339855134 secs [22:57:31] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query en.religiononfire.mar.in.ua. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [23:02:14] PROBLEM - cp23 Current Load on cp23 is WARNING: LOAD WARNING - total load average: 3.41, 2.33, 1.31 [23:04:14] RECOVERY - cp23 Current Load on cp23 is OK: LOAD OK - total load average: 0.93, 1.70, 1.19 [23:04:30] PROBLEM - db112 Current Load on db112 is CRITICAL: CRITICAL - load average: 12.14, 7.79, 3.66 [23:06:30] PROBLEM - db112 Current Load on db112 is WARNING: WARNING - load average: 4.53, 6.88, 3.86 [23:08:30] RECOVERY - db112 Current Load on db112 is OK: OK - load average: 0.79, 4.68, 3.42 [23:26:02] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [23:43:36] PROBLEM - wiki.altilunium.my.id - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.altilunium.my.id All nameservers failed to answer the query. [23:57:31] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - en.religiononfire.mar.in.ua All nameservers failed to answer the query.