[00:05:47] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 6.48, 4.19, 2.45 [00:50:38] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.eu.org. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [02:18:35] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.andreijiroh.uk.eu.org All nameservers failed to answer the query. [02:47:54] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.eu.org. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [03:01:31] PROBLEM - mon141 Backups Grafana on mon141 is CRITICAL: FILE_AGE CRITICAL: /var/log/grafana-backup.log is 1209670 seconds old and 92 bytes [03:17:18] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp25.miraheze.org - CNAME OK [04:21:47] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 0.84, 2.53, 3.58 [04:23:47] RECOVERY - matomo121 Current Load on matomo121 is OK: LOAD OK - total load average: 1.05, 2.07, 3.28 [04:36:42] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 5.16, 3.86, 3.40 [04:42:38] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.89, 3.85, 3.54 [04:48:35] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 4.02, 3.58, 3.50 [04:52:33] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.71, 3.97, 3.71 [05:02:26] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 4.43, 3.97, 3.78 [05:18:17] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 2.80, 3.74, 3.93 [05:20:15] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 4.07, 3.83, 3.93 [05:22:14] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 2.95, 3.54, 3.82 [05:26:12] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 4.46, 3.78, 3.84 [05:30:09] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 2.78, 3.83, 3.88 [05:36:06] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 5.60, 4.22, 4.01 [05:44:01] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 2.95, 3.81, 3.97 [06:03:49] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 4.16, 4.07, 3.91 [07:13:47] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 1.15, 2.86, 3.98 [07:17:47] RECOVERY - matomo121 Current Load on matomo121 is OK: LOAD OK - total load average: 0.68, 1.76, 3.27 [08:05:47] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 9.27, 4.58, 2.45 [08:52:01] PROBLEM - cloud13 Puppet on cloud13 is UNKNOWN: NRPE: Unable to read output [09:20:00] RECOVERY - cloud13 Puppet on cloud13 is OK: OK: Puppet is currently enabled, last run 26 seconds ago with 0 failures [09:41:01] RECOVERY - ns1 NTP time on ns1 is OK: NTP OK: Offset 0.06295138597 secs [09:57:01] PROBLEM - ns1 NTP time on ns1 is WARNING: NTP WARNING: Offset 0.1077485085 secs [11:16:56] [02miraheze/ErrorPages] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ErrorPages/compare/cb52a6de5346...32a63d81d859 [11:16:58] [02miraheze/ErrorPages] 07translatewiki 0332a63d8 - Localisation updates from https://translatewiki.net. [11:17:00] [02miraheze/landing] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/landing/compare/b51460244ec5...77c5af97468d [11:17:03] [02miraheze/landing] 07translatewiki 0377c5af9 - Localisation updates from https://translatewiki.net. [11:17:04] [02miraheze/ImportDump] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/3a7c44f944e7...8a5bfbdb4a27 [11:17:05] [02miraheze/ImportDump] 07translatewiki 038a5bfbd - Localisation updates from https://translatewiki.net. [11:17:08] [02miraheze/PDFEmbed] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/PDFEmbed/compare/65b9638cfbc8...1d7e32538216 [11:17:10] [02miraheze/PDFEmbed] 07translatewiki 031d7e325 - Localisation updates from https://translatewiki.net. [11:17:11] [02miraheze/ManageWiki] 07translatewiki pushed 031 commit to 03master [+0/-0/±3] 13https://github.com/miraheze/ManageWiki/compare/8abbf0e33918...2397a4145349 [11:17:13] [02miraheze/ManageWiki] 07translatewiki 032397a41 - Localisation updates from https://translatewiki.net. [11:17:15] [02miraheze/RottenLinks] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/RottenLinks/compare/e19a58e1b37e...dbc1a0e1a3e7 [11:17:17] [02miraheze/RottenLinks] 07translatewiki 03dbc1a0e - Localisation updates from https://translatewiki.net. [11:17:19] [02miraheze/DataDump] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/DataDump/compare/6534dc46b85f...e803e6ca40bf [11:17:22] [02miraheze/DataDump] 07translatewiki 03e803e6c - Localisation updates from https://translatewiki.net. [11:17:23] [02miraheze/WikiDiscover] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/WikiDiscover/compare/f85b9771c345...cadb84261545 [11:17:26] [02miraheze/WikiDiscover] 07translatewiki 03cadb842 - Localisation updates from https://translatewiki.net. [11:17:29] [02miraheze/SpriteSheet] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/SpriteSheet/compare/bf7751c2ae81...182f2b01b6c0 [11:17:31] [02miraheze/SpriteSheet] 07translatewiki 03182f2b0 - Localisation updates from https://translatewiki.net. [11:17:33] [02miraheze/MirahezeMagic] 07translatewiki pushed 031 commit to 03master [+0/-0/±3] 13https://github.com/miraheze/MirahezeMagic/compare/60c15dfdc3c1...33faf70467c7 [11:17:35] [02miraheze/MirahezeMagic] 07translatewiki 0333faf70 - Localisation updates from https://translatewiki.net. [11:22:29] miraheze/ErrorPages - translatewiki the build passed. [11:22:35] miraheze/landing - translatewiki the build passed. [11:23:02] miraheze/PDFEmbed - translatewiki the build passed. [11:23:52] miraheze/DataDump - translatewiki the build passed. [11:23:53] miraheze/RottenLinks - translatewiki the build passed. [11:24:15] miraheze/ManageWiki - translatewiki the build passed. [11:28:07] miraheze/SpriteSheet - translatewiki the build passed. [11:28:41] miraheze/WikiDiscover - translatewiki the build passed. [11:29:17] miraheze/ImportDump - translatewiki the build passed. [11:35:12] miraheze/MirahezeMagic - translatewiki the build passed. [11:51:47] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 2.61, 3.28, 3.95 [11:53:47] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 5.02, 3.80, 4.05 [11:57:47] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 1.64, 3.21, 3.79 [12:01:47] RECOVERY - matomo121 Current Load on matomo121 is OK: LOAD OK - total load average: 1.38, 2.25, 3.26 [12:05:47] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 4.88, 3.56, 3.56 [12:11:47] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.74, 3.81, 3.72 [12:13:47] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 4.33, 4.10, 3.84 [12:43:51] PROBLEM - wiki.tmyt105.leyhp.com - reverse DNS on sslhost is WARNING: Timeout: The DNS operation timed out after 5.406219720840454 seconds [13:13:29] RECOVERY - wiki.tmyt105.leyhp.com - reverse DNS on sslhost is OK: SSL OK - wiki.tmyt105.leyhp.com reverse DNS resolves to cp24.miraheze.org - CNAME OK [14:48:56] PROBLEM - wiki.walkscape.app - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.walkscape.app' expires in 15 day(s) (Wed 25 Oct 2023 14:32:05 GMT +0000). [14:51:31] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/79df4fd16b4c...4fc7eb7f2bce [14:51:34] [02miraheze/ssl] 07MirahezeSSLBot 034fc7eb7 - Bot: Update SSL cert for wiki.walkscape.app [14:53:48] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 1.33, 2.02, 3.73 [14:57:03] PROBLEM - os141 Current Load on os141 is WARNING: WARNING - load average: 3.87, 2.36, 1.29 [14:57:47] RECOVERY - matomo121 Current Load on matomo121 is OK: LOAD OK - total load average: 1.08, 1.52, 3.13 [14:59:02] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 1.89, 2.17, 1.36 [15:10:54] PROBLEM - mw141 Current Load on mw141 is WARNING: WARNING - load average: 10.14, 10.65, 6.76 [15:12:53] RECOVERY - mw141 Current Load on mw141 is OK: OK - load average: 3.75, 8.08, 6.29 [15:46:56] RECOVERY - wiki.walkscape.app - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.walkscape.app' will expire on Sun 07 Jan 2024 13:51:22 GMT +0000. [16:05:47] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 6.92, 4.43, 2.67 [16:49:06] PROBLEM - graylog131 Current Load on graylog131 is WARNING: WARNING - load average: 3.44, 3.02, 2.51 [16:51:06] RECOVERY - graylog131 Current Load on graylog131 is OK: OK - load average: 2.22, 2.66, 2.44 [17:22:11] PROBLEM - fc.songcontests.eu - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query songcontests.eu. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [17:51:23] RECOVERY - fc.songcontests.eu - reverse DNS on sslhost is OK: SSL OK - fc.songcontests.eu reverse DNS resolves to cp24.miraheze.org - CNAME OK [19:43:01] PROBLEM - ns1 NTP time on ns1 is CRITICAL: NTP CRITICAL: Offset 0.5812393725 secs [19:44:44] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/ssl/compare/4fc7eb7f2bce...27973a4ef904 [19:44:46] [02miraheze/ssl] 07MirahezeSSLBot 0327973a4 - Bot: Add SSL cert for wiki.zamnhacking.net [19:45:25] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/ssl/compare/27973a4ef904...b5e549f8bce0 [19:45:26] [02miraheze/ssl] 07MirahezeSSLBot 03b5e549f - Bot: Add SSL cert for mhdh.pj568.eu.org [19:48:42] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/ssl/compare/b5e549f8bce0...4e280531ae34 [19:48:44] [02miraheze/ssl] 07MirahezeSSLBot 034e28053 - Bot: Add SSL cert for wiki.moores.tech [19:54:30] [02ssl] 07MacFan4000 created branch 03MacFan4000-patch-1 - 13https://github.com/miraheze/ssl [19:54:32] [02miraheze/ssl] 07MacFan4000 pushed 031 commit to 03MacFan4000-patch-1 [+0/-1/±0] 13https://github.com/miraheze/ssl/commit/d6210a6387b3 [19:54:33] [02miraheze/ssl] 07MacFan4000 03d6210a6 - Delete certificates/gs.sidem.wiki.crt [19:54:47] [02ssl] 07MacFan4000 opened pull request 03#729: remove sidem.org per request - 13https://github.com/miraheze/ssl/pull/729 [19:55:28] [02ssl] 07MacFan4000 synchronize pull request 03#729: remove sidem.org per request - 13https://github.com/miraheze/ssl/pull/729 [19:55:30] [02miraheze/ssl] 07MacFan4000 pushed 031 commit to 03MacFan4000-patch-1 [+0/-1/±0] 13https://github.com/miraheze/ssl/compare/d6210a6387b3...661291f921cf [19:55:32] [02miraheze/ssl] 07MacFan4000 03661291f - Delete certificates/www.sidem.wiki.crt [19:56:48] [02miraheze/ssl] 07MacFan4000 pushed 031 commit to 03MacFan4000-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/661291f921cf...9524f93d988e [19:56:50] [02miraheze/ssl] 07MacFan4000 039524f93 - Update certs.yaml [19:56:53] [02ssl] 07MacFan4000 synchronize pull request 03#729: remove sidem.org per request - 13https://github.com/miraheze/ssl/pull/729 [19:58:06] [02ssl] 07MacFan4000 closed pull request 03#729: remove sidem.org per request - 13https://github.com/miraheze/ssl/pull/729 [19:58:08] [02miraheze/ssl] 07MacFan4000 deleted branch 03MacFan4000-patch-1 [19:58:11] [02miraheze/ssl] 07MacFan4000 pushed 031 commit to 03master [+0/-2/±1] 13https://github.com/miraheze/ssl/compare/4e280531ae34...711af7744c7c [19:58:14] [02miraheze/ssl] 07MacFan4000 03711af77 - remove sidem.org per request (#729) [19:58:15] [02ssl] 07MacFan4000 deleted branch 03MacFan4000-patch-1 - 13https://github.com/miraheze/ssl [20:09:46] [02dns] 07MacFan4000 opened pull request 03#442: add 3 zones - 13https://github.com/miraheze/dns/pull/442 [20:24:37] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 1.25, 2.87, 3.77 [20:36:25] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 6.23, 4.43, 3.94 [20:56:50] PROBLEM - www.lgbtqia.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.lgbtqia.wiki could not be found [21:08:06] RECOVERY - www.lgbtqia.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'lgbtqia.wiki' will expire on Tue 21 Nov 2023 06:22:54 GMT +0000. [21:48:54] [02puppet] 07OAuthority opened pull request 03#3444: add google fonts (T11269) - 13https://github.com/miraheze/puppet/pull/3444 [21:56:10] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.03, 3.71, 3.97 [22:04:10] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 4.75, 3.98, 3.94 [22:07:56] PROBLEM - mw143 Puppet on mw143 is UNKNOWN: NRPE: Unable to read output [22:33:56] RECOVERY - mw143 Puppet on mw143 is OK: OK: Puppet is currently enabled, last run 0 seconds ago with 0 failures [23:02:33] PROBLEM - db112 Current Load on db112 is CRITICAL: CRITICAL - load average: 12.64, 7.73, 4.61 [23:08:33] PROBLEM - db112 Current Load on db112 is WARNING: WARNING - load average: 2.64, 7.53, 5.98 [23:10:33] RECOVERY - db112 Current Load on db112 is OK: OK - load average: 2.11, 5.76, 5.51 [23:16:10] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 0.80, 2.62, 3.76 [23:18:10] RECOVERY - matomo121 Current Load on matomo121 is OK: LOAD OK - total load average: 1.03, 2.01, 3.39 [23:24:49] RECOVERY - cloud11 IPMI Sensors on cloud11 is OK: IPMI Status: OK [23:28:49] PROBLEM - cloud11 IPMI Sensors on cloud11 is CRITICAL: IPMI Status: Critical [Cntlr 2 Bay 8 = Critical]