[00:07:17] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 17.14, 7.54, 4.40 [00:12:57] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.religiononfire.mar.in.ua. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [00:12:59] RECOVERY - wiki.zulaclub.net - reverse DNS on sslhost is OK: SSL OK - wiki.zulaclub.net reverse DNS resolves to cp23.miraheze.org - CNAME OK [00:22:44] PROBLEM - cp22 Current Load on cp22 is CRITICAL: CRITICAL - load average: 4.31, 3.12, 1.90 [00:23:16] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.89, 3.12, 3.95 [00:24:41] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 0.76, 2.16, 1.69 [00:25:15] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.06, 3.80, 4.09 [00:29:15] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.40, 3.21, 3.86 [00:30:40] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [00:36:21] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: CRITICAL - load average: 4.49, 3.72, 2.90 [00:38:15] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.32, 3.45, 2.89 [00:39:13] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.33, 2.51, 3.32 [00:40:09] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 3.20, 3.35, 2.92 [00:48:04] PROBLEM - swiftobject112 PowerDNS Recursor on swiftobject112 is CRITICAL: CRITICAL - Plugin timed out while executing system call [00:50:07] RECOVERY - swiftobject112 PowerDNS Recursor on swiftobject112 is OK: DNS OK: 6.348 seconds response time. miraheze.org returns 109.228.51.216,217.174.247.33,2a00:da00:1800:326::1,2a00:da00:1800:328::1 [01:00:43] PROBLEM - drowiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'drowiki.com' expires in 14 day(s) (Tue 25 Jul 2023 15:53:31 GMT +0000). [01:07:00] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: CRITICAL - load average: 4.29, 3.63, 3.00 [01:07:10] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.70, 4.08, 3.12 [01:11:00] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 2.81, 3.53, 3.15 [01:13:00] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 2.77, 3.25, 3.08 [01:15:11] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.28, 3.90, 3.55 [01:17:11] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.14, 3.28, 3.36 [01:27:00] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.84, 3.34, 3.10 [01:29:00] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 2.58, 3.15, 3.07 [01:30:03] PROBLEM - drowiki.com - LetsEncrypt on sslhost is CRITICAL: connect to address drowiki.com and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [01:42:13] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 6.15, 6.32, 7.75 [01:46:13] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 8.66, 7.06, 7.68 [01:48:13] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 7.21, 6.93, 7.55 [01:55:04] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.93, 3.22, 2.94 [01:57:04] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.16, 2.88, 2.86 [02:04:13] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 6.35, 6.36, 6.79 [02:07:03] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.95, 4.17, 3.18 [02:08:13] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 7.43, 7.00, 6.95 [02:10:14] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 9.13, 7.75, 7.23 [02:11:03] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.39, 3.91, 3.34 [02:13:02] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.88, 3.19, 3.15 [02:28:43] PROBLEM - drowiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'drowiki.com' expires in 14 day(s) (Tue 25 Jul 2023 15:53:31 GMT +0000). [02:31:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 5.88, 7.21, 7.98 [02:33:28] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.03, 7.48, 7.97 [02:39:15] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [02:39:48] PROBLEM - mw132 Current Load on mw132 is WARNING: WARNING - load average: 6.92, 10.24, 6.83 [02:41:48] RECOVERY - mw132 Current Load on mw132 is OK: OK - load average: 3.83, 8.02, 6.41 [02:58:04] PROBLEM - drowiki.com - LetsEncrypt on sslhost is CRITICAL: connect to address drowiki.com and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [03:00:14] RECOVERY - mon141 Backups Grafana on mon141 is OK: FILE_AGE OK: /var/log/grafana-backup.log is 11 seconds old and 26 bytes [03:03:46] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.68, 4.32, 2.81 [03:05:42] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.57, 3.19, 2.58 [03:05:55] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.91, 3.82, 2.95 [03:07:55] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.79, 3.61, 3.00 [03:11:54] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.15, 3.33, 3.08 [03:27:24] PROBLEM - drowiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'drowiki.com' expires in 14 day(s) (Tue 25 Jul 2023 15:53:31 GMT +0000). [03:36:51] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.99, 4.38, 3.16 [03:42:50] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.29, 3.50, 3.27 [03:43:32] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [03:44:50] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.30, 3.12, 3.16 [03:53:31] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 5.08, 6.78, 7.93 [03:56:41] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.88, 3.60, 2.88 [03:56:44] PROBLEM - drowiki.com - LetsEncrypt on sslhost is CRITICAL: connect to address drowiki.com and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [03:58:38] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.22, 3.21, 2.83 [04:07:31] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 5.51, 6.01, 6.77 [04:13:32] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['ns.ankh.fr.eu.org.', 'ns1.eu.org.', 'ns1.eriomem.net.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [04:14:53] PROBLEM - avid.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for avid.wiki could not be found [04:15:31] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 8.17, 6.89, 6.72 [04:15:34] PROBLEM - www.avid.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.avid.wiki could not be found [04:15:58] PROBLEM - avid.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address avid.wiki and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [04:17:31] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 6.50, 6.67, 6.66 [04:20:09] PROBLEM - mw134 Current Load on mw134 is CRITICAL: CRITICAL - load average: 14.56, 13.46, 7.73 [04:22:08] RECOVERY - mw134 Current Load on mw134 is OK: OK - load average: 4.27, 9.95, 7.14 [04:25:03] PROBLEM - www.avid.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address www.avid.wiki and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [04:33:59] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 8.38, 7.26, 6.75 [04:41:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.04, 7.43, 7.87 [04:47:58] PROBLEM - wiki.autocountsoft.com - reverse DNS on sslhost is WARNING: Timeout: The DNS operation timed out after 5.4065632820129395 seconds [04:51:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.53, 7.46, 7.62 [04:57:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 6.93, 7.51, 7.63 [05:00:37] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.11, 3.34, 2.31 [05:02:37] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 3.17, 3.14, 2.35 [05:05:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 9.15, 7.99, 7.66 [05:07:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.75, 7.95, 7.70 [05:08:15] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 5.95, 6.68, 7.93 [05:11:41] PROBLEM - swiftobject112 Disk Space on swiftobject112 is WARNING: DISK WARNING - free space: / 32161 MB (6% inode=80%); [05:13:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.14, 7.75, 7.64 [05:19:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 6.57, 7.58, 7.67 [05:20:14] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.73, 7.29, 7.43 [05:21:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.18, 7.80, 7.74 [05:34:23] PROBLEM - wikipwaadia.works - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wikipwaadia.works' expires in 15 day(s) (Thu 27 Jul 2023 05:29:36 GMT +0000). [05:40:42] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/82c0c854af3f...5c48e6e87219 [05:40:45] [02miraheze/ssl] 07MirahezeSSLBot 035c48e6e - Bot: Update SSL cert for wikipwaadia.works [05:44:31] PROBLEM - mw131 Current Load on mw131 is WARNING: WARNING - load average: 11.33, 11.69, 7.11 [05:46:24] RECOVERY - wiki.autocountsoft.com - reverse DNS on sslhost is OK: SSL OK - wiki.autocountsoft.com reverse DNS resolves to cp23.miraheze.org - CNAME OK [05:46:26] RECOVERY - mw131 Current Load on mw131 is OK: OK - load average: 4.66, 9.06, 6.69 [05:49:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.31, 7.81, 7.97 [05:51:28] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.06, 7.87, 7.96 [05:57:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.03, 7.89, 8.00 [06:01:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.04, 7.57, 7.80 [06:08:29] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - en.religiononfire.mar.in.ua All nameservers failed to answer the query. [06:08:52] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.religiononfire.mar.in.ua. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [06:09:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 6.91, 7.50, 7.78 [06:13:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.69, 7.94, 7.88 [06:17:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.34, 7.70, 7.81 [06:23:04] PROBLEM - cloud13 Puppet on cloud13 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [06:23:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 9.27, 8.08, 7.88 [06:28:26] PROBLEM - swiftobject121 Disk Space on swiftobject121 is WARNING: DISK WARNING - free space: / 28641 MB (6% inode=80%); [06:29:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 6.62, 7.74, 7.86 [06:31:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.44, 7.88, 7.88 [06:33:10] RECOVERY - wikipwaadia.works - LetsEncrypt on sslhost is OK: OK - Certificate 'wikipwaadia.works' will expire on Mon 09 Oct 2023 04:40:35 GMT +0000. [06:38:07] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - uk.religiononfire.mar.in.ua All nameservers failed to answer the query. [06:38:22] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [06:39:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 6.63, 7.75, 7.93 [06:43:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.17, 7.85, 7.92 [06:45:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.83, 7.81, 7.89 [06:51:04] RECOVERY - cloud13 Puppet on cloud13 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [06:53:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.94, 7.78, 7.73 [06:56:20] PROBLEM - drowiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'drowiki.com' expires in 14 day(s) (Tue 25 Jul 2023 15:53:31 GMT +0000). [06:57:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 6.92, 7.51, 7.65 [07:01:31] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.22, 7.47, 7.92 [07:03:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.50, 7.80, 7.70 [07:04:15] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 6.41, 7.26, 7.98 [07:05:31] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 8.40, 7.99, 8.03 [07:06:15] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 9.00, 7.76, 8.07 [07:07:23] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [07:11:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.81, 7.96, 7.83 [07:13:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.24, 7.95, 7.84 [07:15:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.56, 7.75, 7.78 [07:21:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.12, 7.72, 7.73 [07:27:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 6.83, 7.45, 7.64 [07:37:13] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.55, 4.44, 2.56 [07:39:13] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.48, 3.88, 2.60 [07:41:13] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.82, 3.23, 2.52 [07:53:27] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 6.62, 6.24, 6.72 [07:56:20] PROBLEM - drowiki.com - LetsEncrypt on sslhost is CRITICAL: connect to address drowiki.com and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [08:09:28] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.97, 6.96, 6.52 [08:11:25] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.1220511496 secs [08:11:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.46, 7.39, 6.73 [08:13:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.95, 7.56, 6.87 [08:17:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.12, 7.55, 7.00 [08:21:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.75, 7.78, 7.23 [08:23:25] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.08060014248 secs [08:23:28] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.94, 8.24, 7.47 [08:25:28] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.47, 7.98, 7.47 [08:33:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.70, 8.08, 7.67 [08:41:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.88, 8.00, 7.86 [08:43:17] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.nj.cn.eu.org. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [08:49:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.25, 8.05, 7.92 [08:55:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.40, 7.90, 7.91 [08:57:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.65, 8.17, 8.01 [08:59:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.62, 7.97, 7.96 [09:00:32] PROBLEM - perpustaxaan.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'perpustaxaan.wiki' expires in 14 day(s) (Tue 25 Jul 2023 16:40:00 GMT +0000). [09:03:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.18, 8.01, 7.97 [09:11:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.38, 7.87, 7.95 [09:12:29] RECOVERY - wiki.nj.cn.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.nj.cn.eu.org reverse DNS resolves to cp23.miraheze.org - CNAME OK [09:21:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.19, 7.55, 7.67 [09:23:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.40, 7.43, 7.60 [09:27:11] PROBLEM - wiki.dunmoore.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.dunmoore.net' expires in 13 day(s) (Mon 24 Jul 2023 15:43:21 GMT +0000). [09:27:28] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.27, 7.68, 7.65 [09:29:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.62, 7.64, 7.64 [09:30:28] PROBLEM - perpustaxaan.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address perpustaxaan.wiki and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [09:31:28] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.13, 7.76, 7.68 [09:36:33] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - uk.religiononfire.mar.in.ua All nameservers failed to answer the query. [09:39:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.54, 7.93, 7.83 [09:44:15] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 6.54, 7.52, 7.94 [09:47:28] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.09, 7.56, 7.61 [09:48:15] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.88, 8.13, 8.07 [09:49:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 6.84, 7.39, 7.55 [09:56:21] PROBLEM - wiki.dunmoore.net - LetsEncrypt on sslhost is CRITICAL: connect to address wiki.dunmoore.net and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [10:05:48] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [10:07:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.19, 7.84, 7.66 [10:09:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 6.99, 7.53, 7.57 [10:11:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.79, 7.89, 7.68 [10:13:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.25, 7.71, 7.65 [10:21:05] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.eu.org. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [10:28:13] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 6.16, 6.62, 7.88 [10:35:03] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [10:36:14] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 5.37, 5.40, 6.77 [10:37:00] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.41, 2.70, 2.09 [10:39:00] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 2.45, 2.57, 2.11 [10:41:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.43, 7.47, 7.23 [10:42:13] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 7.03, 6.36, 6.74 [10:43:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.69, 7.46, 7.26 [10:43:30] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - en.religiononfire.mar.in.ua All nameservers failed to answer the query. [10:46:13] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 8.20, 7.10, 6.94 [10:46:26] RECOVERY - swiftobject121 Disk Space on swiftobject121 is OK: DISK OK - free space: / 52498 MB (11% inode=82%); [10:50:22] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp23.miraheze.org - CNAME OK [10:51:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.38, 7.65, 7.40 [10:53:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 6.92, 7.42, 7.35 [10:55:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.84, 7.83, 7.50 [10:55:54] RECOVERY - swiftobject111 Disk Space on swiftobject111 is OK: DISK OK - free space: / 58771 MB (11% inode=83%); [10:57:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.91, 7.84, 7.55 [11:01:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.15, 7.62, 7.49 [11:03:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.60, 7.56, 7.49 [11:04:13] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 7.49, 7.92, 7.85 [11:07:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.02, 7.83, 7.61 [11:08:13] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 9.00, 8.27, 7.98 [11:09:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.01, 7.49, 7.51 [11:13:24] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [11:25:39] PROBLEM - swiftobject113 Disk Space on swiftobject113 is WARNING: DISK WARNING - free space: / 32105 MB (6% inode=80%); [11:43:19] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query cn.eu.org. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [11:45:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.44, 7.51, 7.31 [11:47:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.51, 7.51, 7.34 [12:12:30] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.nj.cn.eu.org All nameservers failed to answer the query. [12:21:08] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.andreijiroh.uk.eu.org. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [12:27:59] PROBLEM - wiki.bloxotel.tk - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.bloxotel.tk' expires in 15 day(s) (Thu 27 Jul 2023 12:00:38 GMT +0000). [12:29:06] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/5c48e6e87219...35fd90172c58 [12:29:08] [02miraheze/ssl] 07MirahezeSSLBot 0335fd901 - Bot: Update SSL cert for wiki.bloxotel.tk [12:35:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.68, 7.58, 7.33 [12:37:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.79, 7.67, 7.39 [12:41:40] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query cn.eu.org. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [12:55:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.22, 7.35, 7.25 [12:56:57] RECOVERY - wiki.bloxotel.tk - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.bloxotel.tk' will expire on Mon 09 Oct 2023 11:28:57 GMT +0000. [12:57:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.65, 7.44, 7.29 [13:11:27] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 5.94, 6.38, 6.80 [13:19:38] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp23.miraheze.org - CNAME OK [13:37:23] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 8.27, 5.98, 3.25 [13:43:22] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.09, 3.07, 2.88 [14:01:31] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 6.08, 6.48, 7.78 [14:06:08] PROBLEM - mw131 Current Load on mw131 is CRITICAL: CRITICAL - load average: 11.09, 12.34, 7.67 [14:07:31] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 8.41, 7.34, 7.70 [14:07:42] PROBLEM - os141 Current Load on os141 is CRITICAL: CRITICAL - load average: 5.78, 3.54, 1.78 [14:08:08] RECOVERY - mw131 Current Load on mw131 is OK: OK - load average: 3.65, 9.15, 7.07 [14:09:31] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.47, 7.55, 7.74 [14:10:29] PROBLEM - mw142 Current Load on mw142 is CRITICAL: CRITICAL - load average: 17.47, 18.45, 10.11 [14:11:31] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 8.72, 7.99, 7.88 [14:13:02] PROBLEM - cp32 Varnish Backends on cp32 is CRITICAL: 1 backends are down. mw142 [14:14:29] PROBLEM - mw142 Current Load on mw142 is WARNING: WARNING - load average: 4.06, 10.73, 8.84 [14:15:02] RECOVERY - cp32 Varnish Backends on cp32 is OK: All 15 backends are healthy [14:16:29] RECOVERY - mw142 Current Load on mw142 is OK: OK - load average: 3.04, 8.21, 8.15 [14:17:37] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 1.11, 3.00, 2.63 [14:31:41] RECOVERY - swiftobject112 Disk Space on swiftobject112 is OK: DISK OK - free space: / 58626 MB (11% inode=82%); [14:38:03] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [14:41:13] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.nj.cn.eu.org All nameservers failed to answer the query. [14:56:20] PROBLEM - drowiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'drowiki.com' expires in 14 day(s) (Tue 25 Jul 2023 15:53:31 GMT +0000). [15:07:19] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [15:08:08] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.12, 3.66, 2.50 [15:10:08] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.17, 3.41, 2.55 [15:12:07] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.79, 3.19, 2.57 [15:21:43] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.andreijiroh.uk.eu.org. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [15:26:20] PROBLEM - drowiki.com - LetsEncrypt on sslhost is CRITICAL: connect to address drowiki.com and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [15:37:30] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 6.81, 6.69, 6.37 [15:39:27] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 6.26, 6.45, 6.32 [16:05:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.70, 6.84, 6.46 [16:07:27] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 5.97, 6.55, 6.40 [16:07:33] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/aa3f668420d6 [16:07:36] [02miraheze/puppet] 07paladox 03aa3f668 - jobrunner: change nice value to -19 for non-high-pri [16:07:37] [02puppet] 07paladox created branch 03paladox-patch-4 - 13https://github.com/miraheze/puppet [16:07:38] [02puppet] 07paladox opened pull request 03#3284: jobrunner: change nice value to -19 for non-high-pri - 13https://github.com/miraheze/puppet/pull/3284 [16:07:44] [02puppet] 07paladox closed pull request 03#3284: jobrunner: change nice value to -19 for non-high-pri - 13https://github.com/miraheze/puppet/pull/3284 [16:07:45] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - uk.religiononfire.mar.in.ua All nameservers failed to answer the query. [16:07:46] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/d0a47a1a6e2b...a91a7e034123 [16:07:46] [02miraheze/puppet] 07paladox 03a91a7e0 - jobrunner: change nice value to -19 for non-high-pri (#3284) [16:07:49] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-4 [16:07:50] [02puppet] 07paladox deleted branch 03paladox-patch-4 - 13https://github.com/miraheze/puppet [16:13:00] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.47, 2.83, 2.07 [16:15:00] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 2.54, 2.68, 2.11 [16:17:25] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.1812106371 secs [16:20:16] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp23.miraheze.org - CNAME OK [16:21:25] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.07570621371 secs [16:37:01] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [16:42:09] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: CRITICAL - load average: 4.18, 3.35, 2.73 [16:44:03] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 2.98, 3.19, 2.75 [16:47:50] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/6efd0d03d36d [16:47:53] [02miraheze/puppet] 07paladox 036efd0d0 - mediawiki::jobrunner: tune jobs [16:47:55] [02puppet] 07paladox created branch 03paladox-patch-4 - 13https://github.com/miraheze/puppet [16:47:57] [02puppet] 07paladox opened pull request 03#3285: mediawiki::jobrunner: tune jobs - 13https://github.com/miraheze/puppet/pull/3285 [16:50:41] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/6efd0d03d36d...9e3a8369e2d5 [16:50:43] [02miraheze/puppet] 07paladox 039e3a836 - Update jobrunner-hi.json.erb [16:50:44] [02puppet] 07paladox synchronize pull request 03#3285: mediawiki::jobrunner: tune jobs - 13https://github.com/miraheze/puppet/pull/3285 [16:51:33] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.andreijiroh.uk.eu.org. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [16:53:15] [02puppet] 07paladox closed pull request 03#3285: mediawiki::jobrunner: tune jobs - 13https://github.com/miraheze/puppet/pull/3285 [16:53:18] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-4 [16:53:20] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/puppet/compare/a91a7e034123...6f41dc7ca589 [16:53:22] [02miraheze/puppet] 07paladox 036f41dc7 - mediawiki::jobrunner: tune jobs (#3285) [16:53:24] [02puppet] 07paladox deleted branch 03paladox-patch-4 - 13https://github.com/miraheze/puppet [17:01:12] PROBLEM - mwtask141 JobRunner Service on mwtask141 is CRITICAL: PROCS CRITICAL: 0 processes with args 'redisJobRunnerService' [17:01:15] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/ee2bb5b48c83 [17:01:18] [02miraheze/puppet] 07paladox 03ee2bb5b - Mediawiki::jobrunenr: Add upload runner [17:01:21] [02puppet] 07paladox created branch 03paladox-patch-4 - 13https://github.com/miraheze/puppet [17:01:22] [02puppet] 07paladox opened pull request 03#3286: Mediawiki::jobrunenr: Add upload runner - 13https://github.com/miraheze/puppet/pull/3286 [17:02:00] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/ee2bb5b48c83...d41fa96f8fbe [17:02:03] [02miraheze/puppet] 07paladox 03d41fa96 - Update jobrunner-hi.json.erb [17:02:04] [02puppet] 07paladox synchronize pull request 03#3286: Mediawiki::jobrunenr: Add upload runner - 13https://github.com/miraheze/puppet/pull/3286 [17:05:45] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/6f41dc7ca589...1807a86084fb [17:05:48] [02miraheze/puppet] 07paladox 031807a86 - Fix syntax [17:06:54] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 19.24, 7.20, 3.66 [17:07:12] RECOVERY - mwtask141 JobRunner Service on mwtask141 is OK: PROCS OK: 1 process with args 'redisJobRunnerService' [17:12:53] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.45, 3.49, 3.13 [17:14:53] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 0.93, 2.68, 2.88 [17:26:20] PROBLEM - drowiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'drowiki.com' expires in 13 day(s) (Tue 25 Jul 2023 15:53:31 GMT +0000). [17:28:49] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.21, 6.64, 7.98 [17:32:49] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 9.12, 8.07, 8.25 [17:33:37] PROBLEM - swiftobject122 Disk Space on swiftobject122 is WARNING: DISK WARNING - free space: / 53545 MB (6% inode=79%); [17:36:49] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.62, 4.47, 3.01 [17:36:49] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 4.91, 6.86, 7.76 [17:41:15] RECOVERY - wiki.nj.cn.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.nj.cn.eu.org reverse DNS resolves to cp23.miraheze.org - CNAME OK [17:42:48] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.04, 3.66, 3.24 [17:44:21] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1384485364 secs [17:44:47] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.70, 3.17, 3.10 [17:50:03] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp22.miraheze.org - CNAME OK [17:50:49] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 10.75, 8.44, 7.78 [17:56:20] PROBLEM - drowiki.com - LetsEncrypt on sslhost is CRITICAL: connect to address drowiki.com and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [18:04:21] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.07814481854 secs [18:07:44] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 7.79, 5.32, 3.42 [18:11:39] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.96, 3.28, 2.55 [18:13:11] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.nj.cn.eu.org All nameservers failed to answer the query. [18:13:38] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.20, 3.14, 2.58 [18:13:43] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.36, 3.65, 3.29 [18:15:43] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.81, 2.95, 3.08 [18:18:49] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 5.11, 6.00, 7.93 [18:22:14] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/55118ece9605 [18:22:15] [02miraheze/mw-config] 07paladox 0355118ec - Only show errors when running runJobs [18:22:17] [02mw-config] 07paladox created branch 03paladox-patch-1 - 13https://github.com/miraheze/mw-config [18:22:18] [02mw-config] 07paladox opened pull request 03#5279: Only show errors when running runJobs - 13https://github.com/miraheze/mw-config/pull/5279 [18:23:30] miraheze/mw-config - paladox the build passed. [18:26:11] PROBLEM - wiki.dunmoore.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.dunmoore.net' expires in 12 day(s) (Mon 24 Jul 2023 15:43:21 GMT +0000). [18:28:49] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 4.44, 5.13, 6.69 [18:35:27] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.47, 3.52, 3.15 [18:37:40] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.62, 2.98, 2.41 [18:39:25] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.40, 3.37, 3.17 [18:39:39] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.97, 3.94, 2.84 [18:40:49] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 6.98, 6.98, 6.79 [18:42:22] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query cn.eu.org. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [18:42:50] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 10.90, 7.98, 7.15 [18:43:22] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.84, 3.68, 3.36 [18:46:49] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 6.04, 7.74, 7.33 [18:47:20] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.50, 3.36, 3.32 [18:47:24] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/bddb8a62b083...1fc8d5216635 [18:47:26] [02miraheze/mw-config] 07paladox 031fc8d52 - Only show errors when running runJobs (#5279) [18:47:29] [02miraheze/mw-config] 07paladox deleted branch 03paladox-patch-1 [18:47:32] [02mw-config] 07paladox closed pull request 03#5279: Only show errors when running runJobs - 13https://github.com/miraheze/mw-config/pull/5279 [18:47:34] [02mw-config] 07paladox deleted branch 03paladox-patch-1 - 13https://github.com/miraheze/mw-config [18:47:38] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.74, 3.37, 3.12 [18:47:46] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/1fc8d5216635...e5a8cfd7ffac [18:47:49] [02miraheze/mw-config] 07paladox 03e5a8cfd - Don't set error_reporting generally [18:48:00] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [18:48:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:48:12] !log [paladox@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 12s [18:48:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:48:21] !log [paladox@mwtask141] starting deploy of {'config': True} to all [18:48:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:48:28] !log [paladox@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 7s [18:48:29] miraheze/mw-config - paladox the build passed. [18:48:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:48:49] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query en.religiononfire.mar.in.ua. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [18:48:49] miraheze/mw-config - paladox the build passed. [18:50:21] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-5 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/314aa1540ef5 [18:50:23] [02miraheze/puppet] 07paladox 03314aa15 - Only run refreshLink jobs on high priority jobrunner [18:50:24] [02puppet] 07paladox created branch 03paladox-patch-5 - 13https://github.com/miraheze/puppet [18:50:27] [02puppet] 07paladox opened pull request 03#3287: Only run refreshLink jobs on high priority jobrunner - 13https://github.com/miraheze/puppet/pull/3287 [18:50:49] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 10.67, 8.58, 7.71 [18:51:59] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-5 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/314aa1540ef5...cf71f5496ec6 [18:52:02] [02miraheze/puppet] 07paladox 03cf71f54 - Update jobrunner-hi.json.erb [18:52:04] [02puppet] 07paladox synchronize pull request 03#3287: Only run refreshLink jobs on high priority jobrunner - 13https://github.com/miraheze/puppet/pull/3287 [18:52:17] [02puppet] 07paladox closed pull request 03#3287: Only run refreshLink jobs on high priority jobrunner - 13https://github.com/miraheze/puppet/pull/3287 [18:52:18] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/puppet/compare/1807a86084fb...f8cb7e6d6b8a [18:52:19] [02miraheze/puppet] 07paladox 03f8cb7e6 - Only run refreshLink jobs on high priority jobrunner (#3287) [18:52:21] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-5 [18:52:22] [02puppet] 07paladox deleted branch 03paladox-patch-5 - 13https://github.com/miraheze/puppet [18:56:11] PROBLEM - wiki.dunmoore.net - LetsEncrypt on sslhost is CRITICAL: connect to address wiki.dunmoore.net and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [18:57:38] !log [@test131] starting deploy of {'config': True} to all [18:57:39] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [18:57:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:57:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:07:24] PROBLEM - beamngmodding.com - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query beamngmodding.com. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [19:07:34] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.41, 3.38, 2.77 [19:08:49] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 5.85, 7.48, 7.85 [19:09:34] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.79, 2.91, 2.68 [19:12:13] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 7.17, 7.36, 7.96 [19:16:50] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.17, 5.26, 6.59 [19:18:43] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [19:26:32] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 8.17, 7.47, 7.10 [19:32:17] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 5.15, 7.34, 7.28 [19:36:48] PROBLEM - beamngmodding.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - beamngmodding.com All nameservers failed to answer the query. [19:38:01] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.43, 6.16, 6.77 [19:38:13] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 8.51, 7.68, 7.31 [19:41:10] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.nj.cn.eu.org All nameservers failed to answer the query. [19:44:13] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 7.91, 7.87, 7.51 [19:46:14] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 9.89, 8.68, 7.86 [19:47:36] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 6.99, 7.09, 6.79 [19:51:26] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 8.42, 7.92, 7.21 [19:54:09] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.andreijiroh.uk.eu.org All nameservers failed to answer the query. [19:56:11] PROBLEM - wiki.dunmoore.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.dunmoore.net' expires in 12 day(s) (Mon 24 Jul 2023 15:43:21 GMT +0000). [19:58:14] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 6.71, 7.43, 7.95 [20:04:25] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 7.18, 3.94, 2.52 [20:04:51] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.69, 7.82, 7.82 [20:06:12] RECOVERY - beamngmodding.com - reverse DNS on sslhost is OK: SSL OK - beamngmodding.com reverse DNS resolves to cp22.miraheze.org - NS RECORDS OK [20:08:49] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 8.59, 7.42, 7.59 [20:11:11] RECOVERY - wiki.nj.cn.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.nj.cn.eu.org reverse DNS resolves to cp22.miraheze.org - CNAME OK [20:18:49] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 6.66, 7.54, 7.75 [20:20:49] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 9.78, 7.73, 7.75 [20:22:49] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 5.90, 6.97, 7.47 [20:23:24] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.eu.org. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [20:26:11] PROBLEM - wiki.dunmoore.net - LetsEncrypt on sslhost is CRITICAL: connect to address wiki.dunmoore.net and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [20:34:49] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.83, 6.21, 6.80 [20:40:13] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 6.96, 7.43, 7.91 [20:40:14] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 6.09, 6.54, 6.80 [20:40:29] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.36, 2.58, 3.90 [20:44:28] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 0.89, 1.65, 3.21 [20:46:13] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 9.00, 8.01, 7.97 [20:48:13] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 7.79, 7.98, 7.97 [20:50:14] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.11, 7.44, 7.03 [20:52:39] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.andreijiroh.uk.eu.org All nameservers failed to answer the query. [20:54:14] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.54, 7.71, 7.26 [20:56:13] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 8.37, 7.67, 7.77 [20:56:49]