[00:02:09] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - en.religiononfire.mar.in.ua All nameservers failed to answer the query. [00:02:41] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.74, 3.20, 2.51 [00:04:39] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.08, 2.73, 2.42 [00:08:31] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.01, 3.54, 2.84 [00:10:29] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.73, 2.87, 2.68 [00:11:13] PROBLEM - os141 Current Load on os141 is CRITICAL: CRITICAL - load average: 4.55, 3.50, 2.30 [00:13:11] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 2.64, 3.22, 2.34 [00:26:06] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.63, 3.16, 2.63 [00:28:03] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.46, 2.49, 2.44 [00:31:48] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [00:33:46] PROBLEM - os141 Current Load on os141 is CRITICAL: CRITICAL - load average: 4.50, 3.76, 2.98 [00:40:52] [02mw-config] 07thelevking opened pull request 03#5258: Turn on DiscussionTools enhancements in isvwiki - 13https://github.com/miraheze/mw-config/pull/5258 [00:41:47] miraheze/mw-config - thelevking the build passed. [00:45:36] PROBLEM - os141 Current Load on os141 is WARNING: WARNING - load average: 3.66, 3.97, 3.60 [00:53:27] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 2.84, 3.23, 3.38 [01:09:05] PROBLEM - cp32 Varnish Backends on cp32 is CRITICAL: 3 backends are down. mw132 mw142 mw143 [01:11:06] RECOVERY - cp32 Varnish Backends on cp32 is OK: All 15 backends are healthy [01:11:42] RECOVERY - cp33 Varnish Backends on cp33 is OK: All 15 backends are healthy [01:16:07] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.34, 3.72, 3.47 [01:17:05] PROBLEM - cp32 Puppet on cp32 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/usr/local/bin/puppet-enabled] [01:18:03] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.56, 3.28, 3.33 [01:24:05] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.02, 2.91, 2.24 [01:26:02] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.93, 2.45, 2.14 [01:35:56] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.04, 3.43, 3.03 [01:42:38] RECOVERY - cp32 Puppet on cp32 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [01:43:56] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.40, 3.24, 3.38 [01:44:34] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.44, 3.26, 2.64 [01:46:28] PROBLEM - wikien.wildterra2.com - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wikien.wildterra2.com. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [01:48:29] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.08, 2.92, 2.68 [02:28:34] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.56, 3.49, 3.09 [02:30:29] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.77, 2.89, 2.91 [02:36:19] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.45, 3.56, 3.14 [02:38:16] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.85, 3.09, 3.03 [02:45:23] RECOVERY - wikien.wildterra2.com - reverse DNS on sslhost is OK: SSL OK - wikien.wildterra2.com reverse DNS resolves to cp22.miraheze.org - CNAME OK [03:03:42] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.75, 3.95, 2.94 [03:05:40] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.02, 3.44, 2.87 [03:07:37] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.80, 2.89, 2.74 [03:08:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.18, 3.85, 3.01 [03:10:05] PROBLEM - ns1 NTP time on ns1 is WARNING: NTP WARNING: Offset 0.185382843 secs [03:10:23] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.63, 3.01, 2.80 [03:24:04] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.2897063792 secs [03:28:04] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.02678099275 secs [03:36:05] RECOVERY - ns1 NTP time on ns1 is OK: NTP OK: Offset 0.09664386511 secs [03:43:28] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.19, 3.82, 3.09 [03:45:28] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.46, 3.34, 3.00 [04:37:55] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.62, 3.29, 2.56 [04:39:55] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.79, 3.17, 2.61 [05:38:47] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.64, 4.25, 3.11 [05:40:42] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.00, 3.44, 2.94 [05:42:37] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.26, 2.78, 2.77 [06:56:57] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.nj.cn.eu.org. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [07:01:07] PROBLEM - db121 Current Load on db121 is CRITICAL: CRITICAL - load average: 10.07, 11.42, 5.44 [07:03:07] PROBLEM - db121 Current Load on db121 is WARNING: WARNING - load average: 1.78, 7.83, 4.85 [07:05:07] RECOVERY - db121 Current Load on db121 is OK: OK - load average: 0.60, 5.39, 4.32 [07:26:24] RECOVERY - wiki.nj.cn.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.nj.cn.eu.org reverse DNS resolves to cp22.miraheze.org - CNAME OK [07:30:04] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1532003284 secs [07:30:46] PROBLEM - wiki.pixandbit.gay - LetsEncrypt on sslhost is CRITICAL: connect to address wiki.pixandbit.gay and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [07:32:15] PROBLEM - wiki.pixandbit.gay - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.pixandbit.gay could not be found [07:50:04] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.06932210922 secs [08:04:38] PROBLEM - wiki.altilunium.my.id - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.altilunium.my.id All nameservers failed to answer the query. [08:34:18] RECOVERY - wiki.altilunium.my.id - reverse DNS on sslhost is OK: SSL OK - wiki.altilunium.my.id reverse DNS resolves to cp23.miraheze.org - CNAME OK [10:20:50] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.eu.org. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [10:26:05] PROBLEM - gtfo-wiki.cn - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - gtfo-wiki.cn All nameservers failed to answer the query. [10:38:54] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - en.religiononfire.mar.in.ua All nameservers failed to answer the query. [10:50:08] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp22.miraheze.org - CNAME OK [10:57:05] PROBLEM - wiki.hsdfz.org.cn - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.hsdfz.org.cn' expires in 15 day(s) (Sun 09 Jul 2023 10:49:23 GMT +0000). [11:04:32] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/699ae74ce099...546fb2435232 [11:04:34] [02miraheze/ssl] 07MirahezeSSLBot 03546fb24 - Bot: Update SSL cert for wiki.hsdfz.org.cn [11:08:33] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [11:22:02] PROBLEM - beamngmodding.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'beamngmodding.com' expires in 15 day(s) (Sun 09 Jul 2023 10:54:44 GMT +0000). [11:24:47] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/546fb2435232...8ffc099d8f3f [11:24:48] [02miraheze/ssl] 07MirahezeSSLBot 038ffc099 - Bot: Update SSL cert for beamngmodding.com [11:26:42] RECOVERY - wiki.hsdfz.org.cn - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.hsdfz.org.cn' will expire on Thu 21 Sep 2023 10:04:25 GMT +0000. [11:27:00] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.nj.cn.eu.org. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [11:39:55] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.50, 2.64, 1.66 [11:41:55] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.93, 2.38, 1.69 [11:54:05] RECOVERY - gtfo-wiki.cn - reverse DNS on sslhost is OK: SSL OK - gtfo-wiki.cn reverse DNS resolves to cp22.miraheze.org - CNAME OK [11:56:26] RECOVERY - wiki.nj.cn.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.nj.cn.eu.org reverse DNS resolves to cp23.miraheze.org - CNAME OK [12:10:03] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 21.02, 10.61, 5.00 [12:12:04] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1132601798 secs [12:17:55] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 0.67, 3.20, 3.62 [12:19:55] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 0.68, 2.30, 3.23 [12:20:06] RECOVERY - beamngmodding.com - LetsEncrypt on sslhost is OK: OK - Certificate 'beamngmodding.com' will expire on Thu 21 Sep 2023 10:24:39 GMT +0000. [12:29:12] !log delete mw121/mw122 [12:29:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:30:57] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/1908c2aa1a25...9731e21232ea [12:31:00] [02miraheze/puppet] 07paladox 039731e21 - mwupgradetool: remove mw121/122 [12:31:13] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-1/±0] 13https://github.com/miraheze/puppet/commit/38715c1bcb47 [12:31:16] [02miraheze/puppet] 07paladox 0338715c1 - Remove mw121/mw122 [12:31:17] [02puppet] 07paladox created branch 03paladox-patch-4 - 13https://github.com/miraheze/puppet [12:31:20] [02puppet] 07paladox opened pull request 03#3245: Remove mw121/mw122 - 13https://github.com/miraheze/puppet/pull/3245 [12:31:37] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-1/±0] 13https://github.com/miraheze/puppet/compare/38715c1bcb47...e798d0bb36f1 [12:31:39] [02miraheze/puppet] 07paladox 03e798d0b - Delete mw122.yaml [12:31:42] [02puppet] 07paladox synchronize pull request 03#3245: Remove mw121/mw122 - 13https://github.com/miraheze/puppet/pull/3245 [12:31:48] [02puppet] 07paladox closed pull request 03#3245: Remove mw121/mw122 - 13https://github.com/miraheze/puppet/pull/3245 [12:31:50] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-2/±0] 13https://github.com/miraheze/puppet/compare/9731e21232ea...0d2e94ca9c49 [12:31:51] [02miraheze/puppet] 07paladox 030d2e94c - Remove mw121/mw122 (#3245) [12:31:52] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-4 [12:31:53] [02puppet] 07paladox deleted branch 03paladox-patch-4 - 13https://github.com/miraheze/puppet [12:32:20] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/dns/compare/cfdbce98f163...f948cda2d1d3 [12:32:23] [02miraheze/dns] 07paladox 03f948cda - Remove mw121/mw122 from dns [12:32:41] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 983 bytes in 3.214 second response time [12:33:53] PROBLEM - mw132 Current Load on mw132 is CRITICAL: CRITICAL - load average: 12.75, 13.45, 8.68 [12:35:51] PROBLEM - mw132 Current Load on mw132 is WARNING: WARNING - load average: 7.31, 10.98, 8.34 [12:36:10] PROBLEM - mw122 Puppet on mw122 is CRITICAL: connect to address 2a10:6740::6:310 port 5666: No route to hostconnect to host 2a10:6740::6:310 port 5666: No route to host [12:37:48] RECOVERY - mw132 Current Load on mw132 is OK: OK - load average: 5.08, 9.08, 7.97 [12:37:49] PROBLEM - mw121 HTTPS on mw121 is CRITICAL: connect to address 2a10:6740::6:309 and port 443: No route to hostHTTP CRITICAL - Unable to open TCP socket [12:37:50] PROBLEM - ping6 on mw121 is CRITICAL: CRITICAL - Destination Unreachable (2a10:6740::6:309) [12:37:51] PROBLEM - ping6 on mw122 is CRITICAL: CRITICAL - Destination Unreachable (2a10:6740::6:310) PROBLEM - mw122 HTTPS on mw122 is CRITICAL: connect to address 2a10:6740::6:310 and port 443: No route to hostHTTP CRITICAL - Unable to open TCP socket [12:38:04] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.01984903216 secs [12:38:09] PROBLEM - mw121 Disk Space on mw121 is CRITICAL: connect to address 2a10:6740::6:309 port 5666: No route to hostconnect to host 2a10:6740::6:309 port 5666: No route to host [12:38:09] PROBLEM - mw121 php-fpm on mw121 is CRITICAL: connect to address 2a10:6740::6:309 port 5666: No route to hostconnect to host 2a10:6740::6:309 port 5666: No route to host [12:38:09] PROBLEM - mw121 NTP time on mw121 is CRITICAL: connect to address 2a10:6740::6:309 port 5666: No route to hostconnect to host 2a10:6740::6:309 port 5666: No route to host [12:38:09] PROBLEM - mw122 php-fpm on mw122 is CRITICAL: connect to address 2a10:6740::6:310 port 5666: No route to hostconnect to host 2a10:6740::6:310 port 5666: No route to host [12:38:12] PROBLEM - Host mw121 is DOWN: CRITICAL - Destination Unreachable (2a10:6740::6:309) [12:38:12] PROBLEM - mw121 conntrack_table_size on mw121 is CRITICAL: connect to address 2a10:6740::6:309 port 5666: No route to hostconnect to host 2a10:6740::6:309 port 5666: No route to host [12:38:12] PROBLEM - mw121 PowerDNS Recursor on mw121 is CRITICAL: connect to address 2a10:6740::6:309 port 5666: No route to hostconnect to host 2a10:6740::6:309 port 5666: No route to host [12:38:15] PROBLEM - mw122 Current Load on mw122 is CRITICAL: connect to address 2a10:6740::6:310 port 5666: No route to hostconnect to host 2a10:6740::6:310 port 5666: No route to host [12:38:22] PROBLEM - mw122 MediaWiki Rendering on mw122 is CRITICAL: connect to address 2a10:6740::6:310 and port 443: No route to hostHTTP CRITICAL - Unable to open TCP socket [12:39:01] PROBLEM - Host mw122 is DOWN: CRITICAL - Destination Unreachable (2a10:6740::6:310) [12:43:58] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/dns/compare/f948cda2d1d3...5f6a71835685 [12:44:01] [02miraheze/dns] 07paladox 035f6a718 - Remove mw121/mw122 from ptr [12:44:26] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/dns/compare/5f6a71835685...779a8c072502 [12:44:29] [02miraheze/dns] 07paladox 03779a8c0 - Remove graylog121 from ptr [12:45:10] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/dns/compare/779a8c072502...4caa13516c49 [12:45:13] [02miraheze/dns] 07paladox 034caa135 - Update 0.0.0.0.0.4.7.6.0.1.a.2.ip6.arpa [12:47:29] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/dns/compare/4caa13516c49...a774f0915dd3 [12:47:32] [02miraheze/dns] 07paladox 03a774f09 - Add mw131/32/33/34 to ptr [12:50:41] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/dns/compare/a774f0915dd3...c3732d138f9e [12:50:43] [02miraheze/dns] 07paladox 03c3732d1 - Add rest of ptrs for vms on cloud13 [12:56:48] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/dns/compare/c3732d138f9e...3b4a719b77af [12:56:50] [02miraheze/dns] 07paladox 033b4a719 - Add rest of ptrs for vms on cloud14 [13:10:10] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [13:10:35] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 8.25, 4.44, 2.66 [13:12:35] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.92, 3.33, 2.46 [13:27:15] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.nj.cn.eu.org. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [13:57:09] RECOVERY - wiki.nj.cn.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.nj.cn.eu.org reverse DNS resolves to cp22.miraheze.org - CNAME OK [14:09:58] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [14:10:35] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.34, 3.28, 2.18 [14:12:35] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.51, 3.20, 2.30 [14:21:05] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.eu.org. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [14:34:50] PROBLEM - mw142 Current Load on mw142 is CRITICAL: CRITICAL - load average: 26.31, 15.18, 8.46 [14:36:46] PROBLEM - mw142 Current Load on mw142 is WARNING: WARNING - load average: 8.33, 11.99, 8.08 [14:38:42] RECOVERY - mw142 Current Load on mw142 is OK: OK - load average: 3.63, 9.02, 7.47 [14:44:30] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [14:50:31] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.andreijiroh.uk.eu.org All nameservers failed to answer the query. [15:13:25] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['ns.ankh.fr.eu.org.', 'ns1.eu.org.', 'ns1.eriomem.net.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [15:19:56] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp23.miraheze.org - CNAME OK [15:41:46] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.353495568 secs [15:59:46] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset 7.799267769e-05 secs [16:22:30] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1012380123 secs [16:24:30] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.08732944727 secs [17:23:22] PROBLEM - os141 Current Load on os141 is CRITICAL: CRITICAL - load average: 4.89, 2.89, 1.49 [17:23:25] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 10.03, 6.79, 3.69 [17:25:22] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 2.82, 2.71, 1.59 [17:25:25] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 5.04, 5.85, 3.71 [17:42:16] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [17:45:44] F in the chat for mw121/mw122 [18:08:35] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 11.39, 4.83, 2.95 [18:10:35] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.55, 4.00, 2.87 [18:12:10] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [18:12:35] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.38, 3.30, 2.74 [18:39:43] PROBLEM - wiki.zulaclub.net - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.zulaclub.net All nameservers failed to answer the query. [18:39:43] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.23, 3.61, 2.33 [18:41:43] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.26, 3.13, 2.31 [19:00:32] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1001354754 secs [19:02:30] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.09278392792 secs [19:08:35] RECOVERY - wiki.zulaclub.net - reverse DNS on sslhost is OK: SSL OK - wiki.zulaclub.net reverse DNS resolves to cp23.miraheze.org - CNAME OK [19:15:58] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - en.religiononfire.mar.in.ua All nameservers failed to answer the query. [19:45:53] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [19:46:30] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset -0.1122972369 secs [19:50:30] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset -0.09465184808 secs [20:39:56] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.religiononfire.mar.in.ua. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [21:07:34] PROBLEM - os141 Current Load on os141 is CRITICAL: CRITICAL - load average: 4.94, 2.84, 1.52 [21:08:53] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [21:19:37] PROBLEM - os141 Current Load on os141 is WARNING: WARNING - load average: 3.05, 3.84, 3.15 [21:25:35] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 2.08, 3.20, 3.13 [21:28:19] PROBLEM - cp32 NTP time on cp32 is WARNING: NTP WARNING: Offset 0.128290236 secs [21:36:08] PROBLEM - wiki.widwwa.co.uk - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.widwwa.co.uk' expires in 15 day(s) (Sun 09 Jul 2023 21:24:17 GMT +0000). [21:38:22] PROBLEM - os141 Current Load on os141 is CRITICAL: CRITICAL - load average: 4.12, 3.85, 3.34 [21:40:20] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 2.45, 3.25, 3.17 [21:42:19] RECOVERY - cp32 NTP time on cp32 is OK: NTP OK: Offset 0.08919522166 secs [21:43:36] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/8ffc099d8f3f...d61554d6ba2e [21:43:37] [02miraheze/ssl] 07MirahezeSSLBot 03d61554d - Bot: Update SSL cert for wiki.widwwa.co.uk [21:49:34] PROBLEM - mw132 Current Load on mw132 is CRITICAL: CRITICAL - load average: 11.12, 12.67, 8.12 [21:51:31] PROBLEM - mw132 Current Load on mw132 is WARNING: WARNING - load average: 6.99, 10.43, 7.84 [21:53:31] RECOVERY - mw132 Current Load on mw132 is OK: OK - load average: 5.43, 8.71, 7.53 [22:02:19] PROBLEM - cp32 NTP time on cp32 is WARNING: NTP WARNING: Offset 0.127884686 secs [22:08:40] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.06, 4.16, 2.82 [22:10:39] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.78, 3.59, 2.77 [22:12:38] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.97, 2.94, 2.63 [22:14:19] RECOVERY - cp32 NTP time on cp32 is OK: NTP OK: Offset 0.0936626792 secs [22:30:19] PROBLEM - cp32 NTP time on cp32 is WARNING: NTP WARNING: Offset 0.3564530015 secs [22:35:24] RECOVERY - wiki.widwwa.co.uk - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.widwwa.co.uk' will expire on Thu 21 Sep 2023 20:43:25 GMT +0000. [22:48:19] RECOVERY - cp32 NTP time on cp32 is OK: NTP OK: Offset 0.0518476665 secs [22:48:35] PROBLEM - nonbinary.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address nonbinary.wiki and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [22:59:08] PROBLEM - nonbinary.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for nonbinary.wiki could not be found [23:07:17] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.87, 2.59, 2.24 [23:09:17] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.83, 2.52, 2.25 [23:10:30] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1627874374 secs [23:16:30] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.05092680454 secs [23:55:17] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.62, 3.18, 2.27 [23:57:18] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.65, 2.58, 2.16