[00:07:33] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.99, 3.70, 2.52
[00:11:30] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.89, 3.61, 2.79
[00:13:28] <icinga-miraheze>	 RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.74, 3.06, 2.69
[00:18:12] <icinga-miraheze>	 PROBLEM - drowiki.com - LetsEncrypt on sslhost is CRITICAL: connect to address drowiki.com and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket
[00:24:12] <icinga-miraheze>	 PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.23, 3.73, 2.62
[00:25:09] <icinga-miraheze>	 PROBLEM - mw132 Current Load on mw132 is WARNING: WARNING - load average: 9.23, 11.04, 7.25
[00:26:07] <icinga-miraheze>	 RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.72, 2.99, 2.49
[00:27:09] <icinga-miraheze>	 RECOVERY - mw132 Current Load on mw132 is OK: OK - load average: 3.62, 8.39, 6.74
[00:35:09] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.81, 4.00, 2.90
[00:37:08] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.16, 3.47, 2.86
[00:39:06] <icinga-miraheze>	 RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.79, 2.93, 2.73
[01:07:43] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 7.02, 5.31, 3.46
[01:11:39] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.90, 3.46, 3.12
[01:13:37] <icinga-miraheze>	 RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.15, 2.98, 2.98
[01:23:00] <icinga-miraheze>	 PROBLEM - cp32 NTP time on cp32 is WARNING: NTP WARNING: Offset 0.2240237892 secs
[01:37:18] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.66, 4.00, 2.92
[01:39:17] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 4.00, 3.61, 2.89
[01:41:15] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.50, 3.97, 3.11
[01:43:13] <icinga-miraheze>	 RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.90, 3.17, 2.92
[01:49:00] <icinga-miraheze>	 RECOVERY - cp32 NTP time on cp32 is OK: NTP OK: Offset 0.02173155546 secs
[02:06:58] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 13.23, 6.67, 3.81
[02:12:53] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.93, 3.83, 3.45
[02:14:51] <icinga-miraheze>	 RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.12, 3.26, 3.29
[02:36:41] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 11.15, 7.22, 4.26
[02:46:40] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.60, 3.15, 3.76
[02:48:40] <icinga-miraheze>	 RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 0.79, 2.37, 3.39
[03:08:40] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 12.99, 6.30, 3.89
[03:12:40] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.72, 3.61, 3.33
[03:14:40] <icinga-miraheze>	 RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.07, 2.77, 3.05
[03:42:12] <icinga-miraheze>	 PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query en.religiononfire.mar.in.ua. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL
[03:58:56] <icinga-miraheze>	 PROBLEM - wiki.fossbots.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.fossbots.org' expires in 7 day(s) (Mon 24 Jul 2023 03:47:41 GMT +0000).
[03:59:49] <icinga-miraheze>	 PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - uk.religiononfire.mar.in.ua All nameservers failed to answer the query.
[04:06:40] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 7.11, 4.90, 3.06
[04:10:40] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.10, 3.61, 2.92
[04:11:27] <icinga-miraheze>	 RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK
[04:12:40] <icinga-miraheze>	 RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.96, 2.94, 2.75
[04:29:39] <icinga-miraheze>	 RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK
[04:36:40] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.27, 3.91, 2.66
[04:38:40] <icinga-miraheze>	 RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.06, 3.11, 2.52
[04:50:46] <icinga-miraheze>	 PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query.
[05:20:47] <icinga-miraheze>	 PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['ns.ankh.fr.eu.org.', 'ns1.eu.org.', 'ns1.eriomem.net.'], 'CNAME': 'bouncingwiki.miraheze.org.'}
[05:38:06] <icinga-miraheze>	 PROBLEM - cp32 SSH on cp32 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[05:38:59] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 27.49, 15.08, 7.01
[05:39:16] <icinga-miraheze>	 PROBLEM - cp32 PowerDNS Recursor on cp32 is CRITICAL: CRITICAL - Plugin timed out while executing system call
[05:40:01] <icinga-miraheze>	 RECOVERY - cp32 SSH on cp32 is OK: SSH OK - OpenSSH_8.4p1 Debian-5+deb11u1 (protocol 2.0)
[05:41:13] <icinga-miraheze>	 RECOVERY - cp32 PowerDNS Recursor on cp32 is OK: DNS OK: 0.164 seconds response time. miraheze.org returns 108.175.15.182,2607:f1c0:1800:26f::1,2607:f1c0:1800:8100::1,74.208.203.152
[05:42:10] <icinga-miraheze>	 PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL
[05:50:46] <icinga-miraheze>	 PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query.
[05:56:43] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.62, 2.30, 3.80
[06:00:40] <icinga-miraheze>	 RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 0.48, 1.40, 3.10
[06:06:40] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.52, 3.78, 3.50
[06:10:41] <icinga-miraheze>	 RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.96, 2.98, 3.24
[06:11:25] <icinga-miraheze>	 RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK
[06:11:27] <icinga-miraheze>	 PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.56, 2.96, 2.32
[06:15:27] <icinga-miraheze>	 RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.31, 3.08, 2.52
[06:20:46] <icinga-miraheze>	 PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['NS.ANKH.FR.eu.org.', 'NS1.eu.org.', 'NS1.ERIOMEM.NET.'], 'CNAME': 'bouncingwiki.miraheze.org.'}
[06:30:04] <icinga-miraheze>	 PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset -0.1106107533 secs
[06:34:04] <icinga-miraheze>	 RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset -0.0864367485 secs
[06:42:17] <icinga-miraheze>	 PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.81, 3.51, 3.20
[06:46:11] <icinga-miraheze>	 PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.18, 3.73, 3.35
[06:48:08] <icinga-miraheze>	 PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.43, 3.61, 3.35
[06:52:02] <icinga-miraheze>	 RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.37, 3.34, 3.33
[07:02:40] <icinga-miraheze>	 PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.religiononfire.mar.in.ua. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL
[07:12:04] <icinga-miraheze>	 PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.1144073308 secs
[07:42:04] <icinga-miraheze>	 RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.07871061563 secs
[08:01:31] <icinga-miraheze>	 PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1732249856 secs
[08:17:31] <icinga-miraheze>	 RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.04667970538 secs
[08:31:31] <icinga-miraheze>	 PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset -0.1010807753 secs
[08:33:31] <icinga-miraheze>	 RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset -0.07107204199 secs
[09:01:57] <icinga-miraheze>	 RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK
[10:09:11] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.15, 3.92, 2.16
[10:11:09] <icinga-miraheze>	 RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.75, 3.37, 2.17
[10:38:45] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.41, 2.91, 1.80
[10:42:42] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.14, 3.57, 2.44
[10:44:40] <icinga-miraheze>	 RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.22, 3.02, 2.37
[11:08:40] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 12.01, 5.35, 2.84
[11:10:40] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.02, 3.75, 2.56
[11:12:40] <icinga-miraheze>	 RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.42, 2.94, 2.40
[11:38:41] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.89, 2.48, 1.70
[11:40:40] <icinga-miraheze>	 RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.27, 2.51, 1.82
[12:08:34] <icinga-miraheze>	 RECOVERY - cloud11 IPMI Sensors on cloud11 is OK: IPMI Status: OK
[12:09:27] <icinga-miraheze>	 PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.03, 3.11, 2.22
[12:12:11] <icinga-miraheze>	 PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - en.religiononfire.mar.in.ua All nameservers failed to answer the query.
[12:12:34] <icinga-miraheze>	 PROBLEM - cloud11 IPMI Sensors on cloud11 is CRITICAL: IPMI Status: Critical [Cntlr 2 Bay 8 = Critical]
[12:13:27] <icinga-miraheze>	 RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.34, 3.29, 2.51
[12:29:27] <icinga-miraheze>	 PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.83, 3.45, 3.05
[12:31:27] <icinga-miraheze>	 RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.90, 3.29, 3.04
[12:37:13] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.34, 2.43, 1.46
[12:39:11] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.54, 2.96, 1.78
[12:41:09] <icinga-miraheze>	 RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.90, 2.84, 1.87
[12:41:26] <icinga-miraheze>	 RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK
[12:42:18] <icinga-miraheze>	 PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.45, 3.56, 3.29
[12:52:02] <icinga-miraheze>	 RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.09, 3.32, 3.31
[12:55:52] <icinga-miraheze>	 PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.37, 3.76, 3.48
[12:57:49] <icinga-miraheze>	 PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.62, 3.68, 3.48
[12:59:46] <icinga-miraheze>	 RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 1.99, 3.12, 3.30
[13:08:46] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 9.25, 4.95, 2.62
[13:12:34] <icinga-miraheze>	 RECOVERY - cloud11 IPMI Sensors on cloud11 is OK: IPMI Status: OK
[13:16:34] <icinga-miraheze>	 PROBLEM - cloud11 IPMI Sensors on cloud11 is CRITICAL: IPMI Status: Critical [Cntlr 2 Bay 8 = Critical]
[13:22:40] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.52, 3.48, 3.49
[13:24:40] <icinga-miraheze>	 RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 0.77, 2.56, 3.15
[13:36:44] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.10, 3.69, 2.97
[13:39:00] <icinga-miraheze>	 PROBLEM - cp32 NTP time on cp32 is WARNING: NTP WARNING: Offset 0.1424467862 secs
[13:42:40] <icinga-miraheze>	 RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.24, 3.04, 3.10
[13:51:00] <icinga-miraheze>	 RECOVERY - cp32 NTP time on cp32 is OK: NTP OK: Offset 0.08112129569 secs
[14:01:00] <icinga-miraheze>	 PROBLEM - cp32 NTP time on cp32 is WARNING: NTP WARNING: Offset 0.2393352091 secs
[14:06:40] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.60, 2.86, 2.00
[14:06:53] <botifico-16655b7>	 [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/2fe927fd236a
[14:06:56] <botifico-16655b7>	 [02miraheze/puppet] 07paladox 032fe927f - php: decrease pm.max_requests to 10k
[14:06:58] <botifico-16655b7>	 [02puppet] 07paladox created branch 03paladox-patch-6 - 13https://github.com/miraheze/puppet
[14:07:00] <botifico-16655b7>	 [02puppet] 07paladox opened pull request 03#3293: php: decrease pm.max_requests to 10k - 13https://github.com/miraheze/puppet/pull/3293
[14:13:04] <botifico-16655b7>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/ba9b3f3ad7d5...e05a6aed148d
[14:13:05] <botifico-16655b7>	 [02miraheze/mw-config] 07paladox 03e05a6ae - Revert "Lower wgMaxShellMemory to 512 * 512 (#5282)"
[14:13:07] <botifico-16655b7>	 [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-6
[14:13:08] <botifico-16655b7>	 [02puppet] 07paladox closed pull request 03#3293: php: decrease pm.max_requests to 10k - 13https://github.com/miraheze/puppet/pull/3293
[14:13:10] <botifico-16655b7>	 [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/6f162fa33a94...8c5949262d07
[14:13:11] <botifico-16655b7>	 [02miraheze/puppet] 07paladox 038c59492 - php: decrease pm.max_requests to 10k (#3293)
[14:13:12] <botifico-16655b7>	 [02puppet] 07paladox deleted branch 03paladox-patch-6 - 13https://github.com/miraheze/puppet
[14:14:01] <miraheze-github>	 miraheze/mw-config - paladox the build passed.
[14:16:55] <MirahezeLSBot>	 !log [@mwtask141] starting deploy of {'config': True} to all
[14:17:00] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[14:17:04] <MirahezeLSBot>	 !log [@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 9s
[14:17:14] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[14:18:40] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.08, 3.50, 3.47
[14:19:01] <botifico-16655b7>	 [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/56223416dcd2
[14:19:03] <botifico-16655b7>	 [02miraheze/puppet] 07paladox 035622341 - mariadb: increase innodb_ft_cache_size to 200M
[14:19:04] <botifico-16655b7>	 [02puppet] 07paladox created branch 03paladox-patch-6 - 13https://github.com/miraheze/puppet
[14:19:07] <botifico-16655b7>	 [02puppet] 07paladox opened pull request 03#3294: mariadb: increase innodb_ft_cache_size to 200M - 13https://github.com/miraheze/puppet/pull/3294
[14:19:16] <botifico-16655b7>	 [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-6
[14:19:19] <botifico-16655b7>	 [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/8c5949262d07...57594614ab07
[14:19:22] <botifico-16655b7>	 [02miraheze/puppet] 07paladox 035759461 - mariadb: increase innodb_ft_cache_size to 200M (#3294)
[14:19:24] <botifico-16655b7>	 [02puppet] 07paladox closed pull request 03#3294: mariadb: increase innodb_ft_cache_size to 200M - 13https://github.com/miraheze/puppet/pull/3294
[14:19:27] <botifico-16655b7>	 [02puppet] 07paladox deleted branch 03paladox-patch-6 - 13https://github.com/miraheze/puppet
[14:20:40] <icinga-miraheze>	 RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.20, 2.70, 3.18
[14:23:04] <icinga-miraheze>	 RECOVERY - cp32 NTP time on cp32 is OK: NTP OK: Offset 0.09920081496 secs
[14:27:15] <MirahezeLSBot>	 !log [@test131] starting deploy of {'config': True} to all
[14:27:16] <MirahezeLSBot>	 !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s
[14:27:20] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[14:27:24] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[14:39:01] <icinga-miraheze>	 PROBLEM - cp32 NTP time on cp32 is WARNING: NTP WARNING: Offset -0.1117691398 secs
[14:40:03] <icinga-miraheze>	 PROBLEM - electrowiki.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'electrowiki.org' expires in 7 day(s) (Mon 24 Jul 2023 14:25:42 GMT +0000).
[14:41:01] <icinga-miraheze>	 RECOVERY - cp32 NTP time on cp32 is OK: NTP OK: Offset -0.07930192351 secs
[14:45:20] <icinga-miraheze>	 PROBLEM - www.electrowiki.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'electrowiki.org' expires in 7 day(s) (Mon 24 Jul 2023 14:25:42 GMT +0000).
[15:08:42] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 9.99, 4.64, 2.51
[15:14:40] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 0.69, 3.75, 3.14
[15:16:40] <icinga-miraheze>	 RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.83, 3.04, 2.95
[15:17:31] <icinga-miraheze>	 PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1105340123 secs
[15:21:31] <icinga-miraheze>	 RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.09857267141 secs
[15:49:33] <icinga-miraheze>	 PROBLEM - wiki.thedev.gq - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.thedev.gq' expires in 7 day(s) (Mon 24 Jul 2023 15:40:16 GMT +0000).
[15:50:05] <icinga-miraheze>	 PROBLEM - db121 Current Load on db121 is CRITICAL: CRITICAL - load average: 5.89, 8.28, 4.66
[15:52:05] <icinga-miraheze>	 RECOVERY - db121 Current Load on db121 is OK: OK - load average: 0.81, 5.55, 4.09
[16:06:42] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.38, 4.01, 2.29
[16:07:57] <icinga-miraheze>	 PROBLEM - db112 Disk Space on db112 is WARNING: DISK WARNING - free space: / 14543 MB (10% inode=99%);
[16:08:30] <icinga-miraheze>	 PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.98, 3.16, 2.22
[16:10:40] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.93, 3.68, 2.57
[16:12:21] <icinga-miraheze>	 RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 2.68, 3.06, 2.41
[16:12:40] <icinga-miraheze>	 RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.07, 3.07, 2.48
[16:38:52] <icinga-miraheze>	 PROBLEM - kirisame-kissaten.cf - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'kirisame-kissaten.cf' expires in 15 day(s) (Tue 01 Aug 2023 16:09:54 GMT +0000).
[16:39:31] <icinga-miraheze>	 PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1030865312 secs
[16:40:41] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.96, 2.96, 2.09
[16:41:05] <botifico-16655b7>	 [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/4c739c23d047...844ec465509b
[16:41:07] <botifico-16655b7>	 [02miraheze/ssl] 07MirahezeSSLBot 03844ec46 - Bot: Update SSL cert for kirisame-kissaten.cf
[16:42:40] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.16, 3.89, 2.54
[16:44:40] <icinga-miraheze>	 RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.49, 3.08, 2.42
[16:50:09] <icinga-miraheze>	 PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.54, 3.30, 2.99
[16:52:05] <icinga-miraheze>	 RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 2.98, 3.20, 2.99
[17:03:31] <icinga-miraheze>	 RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.09872707725 secs
[17:08:14] <icinga-miraheze>	 RECOVERY - kirisame-kissaten.cf - LetsEncrypt on sslhost is OK: OK - Certificate 'kirisame-kissaten.cf' will expire on Sat 14 Oct 2023 15:40:58 GMT +0000.
[17:10:44] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 9.76, 5.18, 3.09
[17:18:40] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 0.75, 3.48, 3.42
[17:20:40] <icinga-miraheze>	 RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 0.70, 2.53, 3.07
[17:35:41] <icinga-miraheze>	 PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - uk.religiononfire.mar.in.ua All nameservers failed to answer the query.
[17:35:59] <botifico-16655b7>	 [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/d68762c5607f
[17:36:02] <botifico-16655b7>	 [02miraheze/puppet] 07paladox 03d68762c - imagemagick: Add support for being able to set memory/disk/thread limits
[17:36:04] <botifico-16655b7>	 [02puppet] 07paladox created branch 03paladox-patch-6 - 13https://github.com/miraheze/puppet
[17:36:06] <botifico-16655b7>	 [02puppet] 07paladox opened pull request 03#3295: imagemagick: Add support for being able to set memory/disk/thread limits - 13https://github.com/miraheze/puppet/pull/3295
[17:37:15] <botifico-16655b7>	 [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+1/-1/±0] 13https://github.com/miraheze/puppet/compare/d68762c5607f...9fabc2dc2477
[17:37:18] <botifico-16655b7>	 [02miraheze/puppet] 07paladox 039fabc2d - Rename policy.xml to policy.xml.erb
[17:37:19] <botifico-16655b7>	 [02puppet] 07paladox synchronize pull request 03#3295: imagemagick: Add support for being able to set memory/disk/thread limits - 13https://github.com/miraheze/puppet/pull/3295
[17:37:25] <botifico-16655b7>	 [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/9fabc2dc2477...b339b77a4d52
[17:37:26] <botifico-16655b7>	 [02miraheze/puppet] 07paladox 03b339b77 - Update install.pp
[17:37:29] <botifico-16655b7>	 [02puppet] 07paladox synchronize pull request 03#3295: imagemagick: Add support for being able to set memory/disk/thread limits - 13https://github.com/miraheze/puppet/pull/3295
[17:51:00] <botifico-16655b7>	 [02puppet] 07paladox synchronize pull request 03#3295: imagemagick: Add support for being able to set memory/disk/thread limits - 13https://github.com/miraheze/puppet/pull/3295
[17:51:01] <botifico-16655b7>	 [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/b339b77a4d52...23b062bfc333
[17:51:04] <botifico-16655b7>	 [02miraheze/puppet] 07paladox 0323b062b - Update policy.xml.erb
[18:05:30] <icinga-miraheze>	 PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL
[18:09:27] <icinga-miraheze>	 PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.06, 3.29, 2.53
[18:09:37] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 13.08, 6.53, 3.51
[18:11:27] <icinga-miraheze>	 RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.34, 3.40, 2.68
[18:15:31] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.57, 4.00, 3.43
[18:17:30] <icinga-miraheze>	 RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.66, 3.21, 3.21
[18:35:20] <icinga-miraheze>	 RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK
[18:37:38] <icinga-miraheze>	 PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.42, 3.05, 2.62
[18:39:38] <icinga-miraheze>	 RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 2.62, 2.92, 2.63
[18:43:31] <icinga-miraheze>	 PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.92, 3.55, 3.29
[18:49:27] <icinga-miraheze>	 RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.10, 3.04, 3.20
[19:07:04] <icinga-miraheze>	 PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - uk.religiononfire.mar.in.ua All nameservers failed to answer the query.
[19:31:21] <botifico-16655b7>	 [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/ssl/compare/844ec465509b...bbbd465d5c56
[19:31:23] <botifico-16655b7>	 [02miraheze/ssl] 07MirahezeSSLBot 03bbbd465 - Bot: Add SSL cert for wiki.queenscourt.games
[19:37:15] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.29, 3.15, 2.12
[19:39:14] <icinga-miraheze>	 RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.90, 3.21, 2.28
[19:49:24] <botifico-16655b7>	 [02dns] 07MacFan4000 opened pull request 03#425: Add 5 zones - 13https://github.com/miraheze/dns/pull/425
[20:06:43] <icinga-miraheze>	 RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK
[20:07:00] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 7.26, 4.17, 2.64
[20:08:59] <icinga-miraheze>	 RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.15, 3.31, 2.51
[20:41:39] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 9.26, 8.07, 4.70
[20:46:45] <icinga-miraheze>	 PROBLEM - scum-wiki.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'scum-wiki.com' expires in 7 day(s) (Mon 24 Jul 2023 20:30:24 GMT +0000).
[20:51:32] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 0.94, 2.87, 3.77
[20:55:29] <icinga-miraheze>	 RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 0.88, 1.99, 3.22
[21:07:24] <icinga-miraheze>	 PROBLEM - mwtask141 HTTPS on mwtask141 is CRITICAL: connect to address 2a10:6740::6:504 and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket
[21:08:01] <icinga-miraheze>	 PROBLEM - mwtask141 JobRunner Service on mwtask141 is CRITICAL: PROCS CRITICAL: 0 processes with args 'redisJobRunnerService'
[21:09:23] <icinga-miraheze>	 RECOVERY - mwtask141 HTTPS on mwtask141 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 553 bytes in 0.015 second response time
[21:10:01] <icinga-miraheze>	 RECOVERY - mwtask141 JobRunner Service on mwtask141 is OK: PROCS OK: 1 process with args 'redisJobRunnerService'
[21:10:56] <icinga-miraheze>	 PROBLEM - cp32 NTP time on cp32 is WARNING: NTP WARNING: Offset 0.2250899076 secs
[21:11:24] <botifico-16655b7>	 [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-7 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/8961695a6673
[21:11:25] <botifico-16655b7>	 [02miraheze/puppet] 07paladox 038961695 - jobrunner: make sure network is running in systemd unit
[21:11:26] <botifico-16655b7>	 [02puppet] 07paladox created branch 03paladox-patch-7 - 13https://github.com/miraheze/puppet
[21:11:29] <botifico-16655b7>	 [02puppet] 07paladox opened pull request 03#3296: jobrunner: make sure network is running in systemd unit - 13https://github.com/miraheze/puppet/pull/3296
[21:12:20] <botifico-16655b7>	 [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/57594614ab07...6a7a11bdac57
[21:12:22] <botifico-16655b7>	 [02miraheze/puppet] 07paladox 036a7a11b - jobrunner: make sure network is running in systemd unit (#3296)
[21:12:24] <botifico-16655b7>	 [02puppet] 07paladox closed pull request 03#3296: jobrunner: make sure network is running in systemd unit - 13https://github.com/miraheze/puppet/pull/3296
[21:12:25] <botifico-16655b7>	 [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-7
[21:12:26] <botifico-16655b7>	 [02puppet] 07paladox deleted branch 03paladox-patch-7 - 13https://github.com/miraheze/puppet
[21:19:07] <botifico-16655b7>	 [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-7 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/b5dfe754ceb5
[21:19:08] <botifico-16655b7>	 [02miraheze/puppet] 07paladox 03b5dfe75 - cgroup: remove setting systemd.unified_cgroup_hierarchy
[21:19:10] <botifico-16655b7>	 [02puppet] 07paladox created branch 03paladox-patch-7 - 13https://github.com/miraheze/puppet
[21:19:13] <botifico-16655b7>	 [02puppet] 07paladox opened pull request 03#3297: cgroup: remove setting systemd.unified_cgroup_hierarchy - 13https://github.com/miraheze/puppet/pull/3297
[21:19:18] <botifico-16655b7>	 [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/6a7a11bdac57...f82fd165475c
[21:19:21] <botifico-16655b7>	 [02miraheze/puppet] 07paladox 03f82fd16 - cgroup: remove setting systemd.unified_cgroup_hierarchy (#3297)
[21:19:24] <botifico-16655b7>	 [02puppet] 07paladox closed pull request 03#3297: cgroup: remove setting systemd.unified_cgroup_hierarchy - 13https://github.com/miraheze/puppet/pull/3297
[21:19:25] <botifico-16655b7>	 [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-7
[21:19:27] <botifico-16655b7>	 [02puppet] 07paladox deleted branch 03paladox-patch-7 - 13https://github.com/miraheze/puppet
[21:28:56] <icinga-miraheze>	 RECOVERY - cp32 NTP time on cp32 is OK: NTP OK: Offset -0.01386934519 secs
[21:41:56] <icinga-miraheze>	 PROBLEM - bast121 APT on bast121 is CRITICAL: APT CRITICAL: 3 packages available for upgrade (3 critical updates).
[21:44:56] <icinga-miraheze>	 PROBLEM - cp32 NTP time on cp32 is WARNING: NTP WARNING: Offset 0.391463846 secs
[21:50:56] <icinga-miraheze>	 RECOVERY - cp32 NTP time on cp32 is OK: NTP OK: Offset -0.02049836516 secs
[21:56:19] <icinga-miraheze>	 PROBLEM - prometheus131 APT on prometheus131 is CRITICAL: APT CRITICAL: 3 packages available for upgrade (3 critical updates).
[21:58:00] <icinga-miraheze>	 PROBLEM - test131 APT on test131 is CRITICAL: APT CRITICAL: 3 packages available for upgrade (3 critical updates).
[21:59:55] <icinga-miraheze>	 PROBLEM - reports121 APT on reports121 is CRITICAL: APT CRITICAL: 3 packages available for upgrade (3 critical updates).
[22:02:15] <icinga-miraheze>	 PROBLEM - bast141 APT on bast141 is CRITICAL: APT CRITICAL: 3 packages available for upgrade (3 critical updates).
[22:03:50] <icinga-miraheze>	 PROBLEM - swiftobject101 APT on swiftobject101 is CRITICAL: APT CRITICAL: 3 packages available for upgrade (3 critical updates).
[22:03:51] <icinga-miraheze>	 PROBLEM - db112 APT on db112 is CRITICAL: APT CRITICAL: 3 packages available for upgrade (3 critical updates).
[22:05:57] <icinga-miraheze>	 PROBLEM - ldap141 APT on ldap141 is CRITICAL: APT CRITICAL: 3 packages available for upgrade (3 critical updates).
[23:28:56] <icinga-miraheze>	 PROBLEM - cp32 NTP time on cp32 is WARNING: NTP WARNING: Offset 0.1751306653 secs
[23:37:50] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.34, 4.06, 2.69
[23:39:48] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.53, 3.96, 2.83
[23:41:47] <icinga-miraheze>	 PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.13, 3.93, 2.94
[23:45:44] <icinga-miraheze>	 RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.79, 3.09, 2.84
[23:46:14] <icinga-miraheze>	 PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.1140767336 secs
[23:54:56] <icinga-miraheze>	 RECOVERY - cp32 NTP time on cp32 is OK: NTP OK: Offset 0.07828712463 secs