[00:03:08] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.55, 3.67, 3.11 [00:03:52] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.01, 8.98, 5.74 [00:05:08] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.07, 3.12, 2.97 [00:07:42] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.0806787312 secs [00:19:51] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.38, 2.75, 3.90 [00:23:06] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.37, 3.45, 2.89 [00:25:06] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.10, 3.00, 2.80 [00:25:52] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.65, 2.17, 3.26 [00:35:52] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.13, 3.89, 3.29 [00:37:52] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.34, 3.60, 3.25 [00:39:30] PROBLEM - os141 Current Load on os141 is CRITICAL: CRITICAL - load average: 5.28, 4.24, 3.09 [00:39:51] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.17, 2.99, 3.06 [00:41:25] PROBLEM - os141 Current Load on os141 is WARNING: WARNING - load average: 3.46, 3.98, 3.14 [00:43:20] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 2.21, 3.36, 3.01 [01:06:44] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.45, 3.39, 2.59 [01:08:41] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.27, 2.87, 2.49 [01:18:31] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.82, 3.96, 3.09 [01:22:25] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.72, 2.86, 2.84 [01:46:47] PROBLEM - ns1 NTP time on ns1 is WARNING: NTP WARNING: Offset 0.1345265806 secs [02:06:56] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 12.94, 7.63, 4.39 [02:16:43] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.44, 3.48, 3.78 [02:16:45] RECOVERY - ns1 NTP time on ns1 is OK: NTP OK: Offset 0.07738816738 secs [02:20:37] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.51, 2.51, 3.31 [02:25:00] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.09, 3.56, 2.62 [02:27:00] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.99, 3.12, 2.59 [03:00:34] RECOVERY - mon141 Backups Grafana on mon141 is OK: FILE_AGE OK: /var/log/grafana-backup.log is 28 seconds old and 26 bytes [03:03:00] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 9.63, 5.30, 3.37 [03:05:15] PROBLEM - mw142 Current Load on mw142 is CRITICAL: CRITICAL - load average: 14.38, 15.71, 9.52 [03:07:14] PROBLEM - mw142 Current Load on mw142 is WARNING: WARNING - load average: 5.49, 11.80, 8.83 [03:09:01] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.10, 3.85, 3.39 [03:09:12] RECOVERY - mw142 Current Load on mw142 is OK: OK - load average: 4.30, 9.35, 8.29 [03:11:00] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.55, 3.16, 3.20 [03:37:57] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.55, 4.03, 2.69 [03:43:51] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.32, 3.57, 3.04 [03:45:52] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.47, 3.27, 2.99 [03:55:42] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.125220269 secs [04:05:42] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.09825190902 secs [04:05:53] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 7.13, 4.49, 3.09 [04:09:00] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.27, 3.72, 2.84 [04:09:51] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.98, 3.83, 3.15 [04:11:00] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.86, 3.00, 2.68 [04:13:51] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.41, 3.02, 2.97 [05:03:21] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query en.religiononfire.mar.in.ua. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [05:33:10] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - en.religiononfire.mar.in.ua All nameservers failed to answer the query. [06:02:58] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [06:08:31] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.96, 3.18, 2.36 [06:10:28] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.18, 3.62, 2.63 [06:12:25] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.87, 2.92, 2.49 [06:38:53] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.62, 3.37, 2.60 [06:40:50] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.47, 2.98, 2.54 [07:38:34] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.49, 2.50, 1.96 [07:40:31] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.90, 2.33, 1.96 [07:50:49] PROBLEM - fid.koyomi-ya.net - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query koyomi-ya.net. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [08:06:04] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [08:11:52] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.65, 3.66, 2.54 [08:13:51] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.90, 3.04, 2.45 [08:20:48] RECOVERY - fid.koyomi-ya.net - reverse DNS on sslhost is OK: SSL OK - fid.koyomi-ya.net reverse DNS resolves to cp23.miraheze.org - CNAME OK [08:35:52] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [09:10:50] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.92, 4.26, 2.60 [09:12:47] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.07, 3.79, 2.63 [09:14:45] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.77, 3.03, 2.49 [10:13:13] PROBLEM - mw141 Current Load on mw141 is CRITICAL: CRITICAL - load average: 25.95, 15.65, 8.52 [10:15:12] PROBLEM - mw142 Current Load on mw142 is CRITICAL: CRITICAL - load average: 11.53, 13.95, 8.60 [10:17:11] PROBLEM - mw142 Current Load on mw142 is WARNING: WARNING - load average: 6.58, 11.16, 8.22 [10:17:13] RECOVERY - mw141 Current Load on mw141 is OK: OK - load average: 4.51, 9.60, 7.66 [10:19:11] RECOVERY - mw142 Current Load on mw142 is OK: OK - load average: 4.21, 8.73, 7.68 [10:37:52] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.04, 3.50, 2.09 [10:39:51] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.31, 3.20, 2.17 [11:07:52] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.65, 3.23, 2.25 [11:15:52] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.31, 3.44, 3.07 [11:17:51] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 0.91, 2.54, 2.78 [11:40:37] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - en.religiononfire.mar.in.ua All nameservers failed to answer the query. [11:48:32] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/dns/compare/3b4a719b77af...6d44bc30f693 [11:48:35] [02miraheze/dns] 07paladox 036d44bc3 - Don't use multi for swift [12:10:26] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [12:42:10] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [12:51:38] PROBLEM - wiki.yuanpi.eu.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.yuanpi.eu.org' expires in 15 day(s) (Tue 11 Jul 2023 12:49:24 GMT +0000). [12:53:22] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/d61554d6ba2e...8c129bacfeab [12:53:23] [02miraheze/ssl] 07MirahezeSSLBot 038c129ba - Bot: Update SSL cert for wiki.yuanpi.eu.org [13:05:16] PROBLEM - zhacg.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'zhacg.wiki' expires in 15 day(s) (Tue 11 Jul 2023 12:56:06 GMT +0000). [13:09:52] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/8c129bacfeab...52ae6c885c67 [13:09:53] [02miraheze/ssl] 07MirahezeSSLBot 0352ae6c8 - Bot: Update SSL cert for zhacg.wiki [13:11:59] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - en.religiononfire.mar.in.ua All nameservers failed to answer the query. [13:21:37] RECOVERY - wiki.yuanpi.eu.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.yuanpi.eu.org' will expire on Sat 23 Sep 2023 11:53:14 GMT +0000. [13:31:04] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - uk.religiononfire.mar.in.ua All nameservers failed to answer the query. [13:41:34] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 8.97, 6.02, 2.99 [13:43:34] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 6.83, 6.34, 3.48 [13:45:33] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 2.69, 4.80, 3.25 [13:59:58] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [14:00:04] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 2.68, 3.45, 2.20 [14:02:04] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 1.33, 2.77, 2.10 [14:03:16] RECOVERY - zhacg.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'zhacg.wiki' will expire on Sat 23 Sep 2023 12:09:45 GMT +0000. [14:58:33] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 4.63, 3.40, 2.07 [15:02:29] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 1.76, 2.87, 2.20 [15:11:14] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [16:07:52] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 7.13, 4.08, 2.51 [16:09:51] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.05, 3.89, 2.65 [16:11:51] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.15, 3.32, 2.60 [16:17:50] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.21, 2.94, 2.26 [16:19:50] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.66, 2.42, 2.15 [17:08:02] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.01, 3.75, 2.80 [17:10:00] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.44, 3.28, 2.73 [17:35:49] RECOVERY - cloud11 IPMI Sensors on cloud11 is OK: IPMI Status: OK [17:39:49] PROBLEM - cloud11 IPMI Sensors on cloud11 is CRITICAL: IPMI Status: Critical [Cntlr 2 Bay 8 = Critical] [18:17:38] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.37, 3.42, 2.69 [18:19:38] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.65, 4.33, 3.09 [18:23:38] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.99, 3.46, 3.00 [18:25:38] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.94, 3.18, 2.94 [18:36:11] The irc rc bot is dead [19:59:01] PROBLEM - swiftproxy111 Disk Space on swiftproxy111 is CRITICAL: DISK CRITICAL - free space: / 1463 MB (5% inode=95%); [20:02:09] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 15.71, 9.89, 6.72 [20:02:11] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 10.11, 6.27, 4.46 [20:03:49] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 11.12, 7.41, 4.68 [20:09:48] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.19, 6.84, 5.62 [20:11:48] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 3.31, 5.62, 5.31 [20:15:01] RECOVERY - swiftproxy111 Disk Space on swiftproxy111 is OK: DISK OK - free space: / 11690 MB (41% inode=95%); [20:18:11] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 5.83, 7.30, 7.09 [20:22:11] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 3.58, 5.41, 6.37 [20:26:08] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 4.33, 5.62, 7.77 [20:26:24] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - uk.religiononfire.mar.in.ua All nameservers failed to answer the query. [20:32:08] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.12, 4.89, 6.77 [20:32:14] PROBLEM - os141 Current Load on os141 is WARNING: WARNING - load average: 3.70, 2.71, 1.43 [20:34:13] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 1.80, 2.37, 1.46 [20:55:18] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [21:02:13] PROBLEM - os141 Current Load on os141 is WARNING: WARNING - load average: 3.69, 3.30, 2.28 [21:03:52] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.62, 3.00, 2.44 [21:05:52] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.14, 3.97, 2.85 [21:06:13] PROBLEM - os141 Current Load on os141 is CRITICAL: CRITICAL - load average: 4.45, 3.64, 2.62 [21:07:52] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.78, 3.64, 2.89 [21:08:15] PROBLEM - os141 Current Load on os141 is WARNING: WARNING - load average: 3.63, 3.72, 2.79 [21:10:16] PROBLEM - os141 Current Load on os141 is CRITICAL: CRITICAL - load average: 4.40, 3.90, 2.96 [21:12:15] PROBLEM - os141 Current Load on os141 is WARNING: WARNING - load average: 3.39, 3.70, 3.00 [21:13:51] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.12, 2.98, 2.85 [21:14:13] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 2.29, 3.28, 2.94 [21:20:17] PROBLEM - os141 Current Load on os141 is WARNING: WARNING - load average: 3.85, 3.52, 3.11 [21:22:14] PROBLEM - os141 Current Load on os141 is CRITICAL: CRITICAL - load average: 4.68, 3.92, 3.30 [21:23:22] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.98, 3.08, 2.32 [21:24:13] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 2.06, 3.31, 3.16 [21:25:22] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.91, 2.52, 2.20 [22:08:16] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.79, 4.17, 2.86 [22:12:10] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.89, 3.66, 2.97 [22:14:08] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.70, 2.99, 2.80 [22:27:51] RECOVERY - cloud11 IPMI Sensors on cloud11 is OK: IPMI Status: OK [22:30:45] PROBLEM - mw132 Current Load on mw132 is CRITICAL: CRITICAL - load average: 13.24, 14.16, 8.83 [22:31:49] PROBLEM - cloud11 IPMI Sensors on cloud11 is CRITICAL: IPMI Status: Critical [Cntlr 2 Bay 8 = Critical] [22:32:45] PROBLEM - mw132 Current Load on mw132 is WARNING: WARNING - load average: 6.24, 11.16, 8.37 [22:34:45] RECOVERY - mw132 Current Load on mw132 is OK: OK - load average: 4.72, 8.91, 7.88 [22:57:14] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.10, 3.94, 2.78 [22:59:13] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.36, 3.25, 2.66 [23:06:14] PROBLEM - os141 Current Load on os141 is WARNING: WARNING - load average: 3.81, 2.23, 1.21 [23:07:34] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 7.71, 5.81, 3.31 [23:08:15] PROBLEM - os141 Current Load on os141 is CRITICAL: CRITICAL - load average: 5.66, 3.43, 1.78 [23:08:28] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.74, 3.79, 2.89 [23:09:34] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 8.18, 6.36, 3.80 [23:10:25] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.25, 3.44, 2.86 [23:11:34] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 5.39, 6.03, 4.00 [23:12:22] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.72, 3.40, 2.93 [23:14:13] PROBLEM - os141 Current Load on os141 is WARNING: WARNING - load average: 1.79, 3.53, 2.49 [23:16:13] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 0.38, 2.41, 2.20 [23:56:24] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL