[00:04:10] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 6.28, 3.77, 2.22 [01:12:57] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.10, 5.88, 4.92 [01:14:57] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.00, 5.42, 4.87 [03:08:21] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [03:37:42] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['ns.ankh.fr.eu.org.', 'ns1.eu.org.', 'ns1.eriomem.net.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [04:16:10] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 1.37, 2.88, 3.70 [04:18:10] RECOVERY - matomo121 Current Load on matomo121 is OK: LOAD OK - total load average: 1.42, 2.36, 3.40 [04:32:10] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 4.60, 3.79, 3.49 [04:34:10] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.88, 3.87, 3.56 [04:36:10] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 4.06, 4.01, 3.65 [04:40:10] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.21, 3.84, 3.68 [04:46:10] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 5.18, 4.34, 3.94 [04:52:10] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 2.98, 3.80, 3.86 [04:56:10] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 4.60, 3.91, 3.86 [04:58:10] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.64, 3.84, 3.85 [05:04:10] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 4.10, 3.99, 3.89 [05:06:10] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.39, 3.91, 3.89 [05:09:55] PROBLEM - wiki.zulaclub.net - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query zulaclub.net. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [05:14:10] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 4.04, 3.77, 3.80 [05:16:10] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.28, 3.70, 3.78 [05:20:10] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 4.80, 3.93, 3.83 [05:30:10] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.94, 3.95, 3.97 [05:32:10] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 4.18, 4.04, 4.00 [05:38:48] RECOVERY - wiki.zulaclub.net - reverse DNS on sslhost is OK: SSL OK - wiki.zulaclub.net reverse DNS resolves to cp24.miraheze.org - CNAME OK [05:40:10] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 2.82, 3.67, 3.90 [05:50:10] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 4.24, 3.71, 3.77 [05:54:10] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.84, 3.77, 3.77 [05:56:10] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 4.58, 4.04, 3.87 [06:53:06] PROBLEM - db101 Backups SQL on db101 is CRITICAL: FILE_AGE CRITICAL: /var/log/sql-backup.log is 1209634 seconds old and 57467 bytes [07:08:10] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 2.89, 3.06, 3.84 [07:12:10] RECOVERY - matomo121 Current Load on matomo121 is OK: LOAD OK - total load average: 0.66, 1.85, 3.18 [08:01:52] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [08:06:10] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 5.80, 4.02, 2.29 [08:31:13] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['NS.ANKH.FR.eu.org.', 'NS1.eu.org.', 'NS1.ERIOMEM.NET.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [10:28:38] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [10:57:59] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['NS.ANKH.FR.eu.org.', 'NS1.eu.org.', 'NS1.ERIOMEM.NET.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [12:10:10] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 1.05, 2.84, 3.90 [12:11:52] PROBLEM - ping6 on swiftproxy131 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 0.54 ms [12:13:50] RECOVERY - ping6 on swiftproxy131 is OK: PING OK - Packet loss = 0%, RTA = 0.50 ms [12:16:10] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 4.22, 3.37, 3.75 [12:18:10] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.98, 3.55, 3.76 [12:20:10] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 4.79, 3.84, 3.83 [13:21:39] PROBLEM - phab121 Current Load on phab121 is CRITICAL: CRITICAL - load average: 2.43, 1.72, 1.05 [13:23:39] PROBLEM - phab121 Current Load on phab121 is WARNING: WARNING - load average: 1.90, 1.70, 1.12 [13:25:39] RECOVERY - phab121 Current Load on phab121 is OK: OK - load average: 1.35, 1.61, 1.16 [15:10:10] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 1.07, 2.07, 3.72 [15:14:10] RECOVERY - matomo121 Current Load on matomo121 is OK: LOAD OK - total load average: 1.14, 1.64, 3.16 [16:04:10] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 6.47, 3.49, 2.16 [16:22:49] RECOVERY - cloud11 IPMI Sensors on cloud11 is OK: IPMI Status: OK [16:26:49] PROBLEM - cloud11 IPMI Sensors on cloud11 is CRITICAL: IPMI Status: Critical [Cntlr 2 Bay 8 = Critical] [16:57:51] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [17:27:53] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['ns.ankh.fr.eu.org.', 'ns1.eu.org.', 'ns1.eriomem.net.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [18:18:50] [02CreateWiki] 07OAuthority opened pull request 03#447: add migration questions (T10844) - 13https://github.com/miraheze/CreateWiki/pull/447 [18:23:14] miraheze/CreateWiki - OAuthority the build has errored. [18:23:27] [02CreateWiki] 07OAuthority synchronize pull request 03#447: add migration questions (T10844) - 13https://github.com/miraheze/CreateWiki/pull/447 [18:24:07] [02CreateWiki] 07OAuthority synchronize pull request 03#447: add migration questions (T10844) - 13https://github.com/miraheze/CreateWiki/pull/447 [18:25:03] [02CreateWiki] 07OAuthority synchronize pull request 03#447: add migration questions (T10844) - 13https://github.com/miraheze/CreateWiki/pull/447 [18:28:00] [02CreateWiki] 07OAuthority commented on pull request 03#447: add migration questions (T10844) - 13https://github.com/miraheze/CreateWiki/pull/447#issuecomment-1755998701 [19:41:26] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 6.98, 5.99, 5.31 [19:43:23] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.35, 5.66, 5.27 [20:27:51] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [20:32:10] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 1.41, 2.65, 3.71 [20:36:10] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 4.05, 3.08, 3.60 [20:36:50] PROBLEM - telepedia-wiki.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'telepedia-wiki.org' expires in 7 day(s) (Wed 18 Oct 2023 20:28:05 GMT +0000). [20:38:10] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 2.93, 3.03, 3.53 [20:46:10] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 4.07, 3.84, 3.73 [20:46:33] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.00, 6.29, 5.69 [20:48:30] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.55, 6.03, 5.67 [20:57:51] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['NS.ANKH.FR.eu.org.', 'NS1.eu.org.', 'NS1.ERIOMEM.NET.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [21:27:17] PROBLEM - os131 Current Load on os131 is WARNING: WARNING - load average: 3.76, 3.20, 2.34 [21:28:10] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 2.46, 3.35, 3.96 [21:31:20] PROBLEM - os131 Current Load on os131 is CRITICAL: CRITICAL - load average: 4.70, 3.71, 2.74 [21:33:20] PROBLEM - os131 Current Load on os131 is WARNING: WARNING - load average: 3.05, 3.44, 2.76 [21:34:10] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 4.04, 3.53, 3.83 [21:35:19] RECOVERY - os131 Current Load on os131 is OK: OK - load average: 2.62, 3.16, 2.74 [21:48:10] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 2.72, 3.62, 3.91 [21:54:10] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 5.03, 3.97, 3.93 [21:56:10] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 2.90, 3.54, 3.78 [22:00:10] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 4.01, 3.74, 3.81 [22:02:57] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.17, 6.06, 5.13 [22:04:10] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.67, 3.78, 3.82 [22:04:57] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 4.55, 5.49, 5.03 [22:06:10] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 5.03, 4.13, 3.94 [22:08:57] PROBLEM - fc.songcontests.eu - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query fc.songcontests.eu. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [22:38:23] RECOVERY - fc.songcontests.eu - reverse DNS on sslhost is OK: SSL OK - fc.songcontests.eu reverse DNS resolves to cp25.miraheze.org - CNAME OK [23:02:33] PROBLEM - db112 Current Load on db112 is CRITICAL: CRITICAL - load average: 17.15, 8.81, 5.12 [23:12:33] PROBLEM - db112 Current Load on db112 is WARNING: WARNING - load average: 2.04, 7.45, 7.17 [23:14:33] RECOVERY - db112 Current Load on db112 is OK: OK - load average: 3.84, 6.29, 6.76 [23:32:10] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 0.87, 2.34, 3.67 [23:34:10] RECOVERY - matomo121 Current Load on matomo121 is OK: LOAD OK - total load average: 0.66, 1.73, 3.28