[00:03:37] PROBLEM - cp22 Current Load on cp22 is CRITICAL: LOAD CRITICAL - total load average: 16.82, 21.14, 10.03 [00:03:58] PROBLEM - cp23 Current Load on cp23 is CRITICAL: LOAD CRITICAL - total load average: 3.29, 7.21, 3.82 [00:07:58] PROBLEM - cp23 Current Load on cp23 is WARNING: LOAD WARNING - total load average: 1.50, 3.97, 3.24 [00:08:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 5.66, 4.11, 3.87 [00:09:58] PROBLEM - cp23 Current Load on cp23 is CRITICAL: LOAD CRITICAL - total load average: 15.77, 7.38, 4.51 [00:16:34] PROBLEM - cp22 PowerDNS Recursor on cp22 is CRITICAL: CRITICAL - Plugin timed out while executing system call [00:17:58] PROBLEM - cp23 Current Load on cp23 is WARNING: LOAD WARNING - total load average: 2.30, 3.65, 3.91 [00:18:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.60, 3.95, 4.00 [00:20:45] RECOVERY - cp22 PowerDNS Recursor on cp22 is OK: DNS OK: 0.518 seconds response time. miraheze.org returns 109.228.51.216,217.174.247.33,2a00:da00:1800:326::1,2a00:da00:1800:328::1 [00:21:58] RECOVERY - cp23 Current Load on cp23 is OK: LOAD OK - total load average: 0.78, 1.99, 3.15 [00:26:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.02, 3.96, 3.98 [00:28:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.79, 3.94, 3.98 [00:30:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.38, 4.06, 4.02 [00:32:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.52, 3.78, 3.91 [00:38:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.37, 3.80, 3.86 [00:44:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.64, 3.87, 3.92 [00:45:47] PROBLEM - cp33 Current Load on cp33 is WARNING: LOAD WARNING - total load average: 3.02, 3.49, 2.44 [00:47:46] RECOVERY - cp33 Current Load on cp33 is OK: LOAD OK - total load average: 2.05, 3.14, 2.44 [00:48:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.23, 4.01, 3.96 [00:50:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.79, 3.99, 3.97 [00:54:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.60, 4.17, 4.03 [01:01:40] PROBLEM - cp22 Current Load on cp22 is WARNING: LOAD WARNING - total load average: 0.57, 0.60, 3.69 [01:03:40] RECOVERY - cp22 Current Load on cp22 is OK: LOAD OK - total load average: 0.22, 0.47, 3.26 [01:08:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.70, 3.85, 4.00 [01:10:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 3.72, 3.95, 4.02 [01:12:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.40, 3.81, 3.97 [01:16:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.03, 3.92, 3.97 [01:50:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.91, 3.95, 4.00 [01:52:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.23, 4.04, 4.03 [02:00:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.60, 3.84, 3.96 [02:08:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.06, 3.84, 3.91 [02:59:32] PROBLEM - cp32 NTP time on cp32 is WARNING: NTP WARNING: Offset 0.1716911793 secs [03:01:08] RECOVERY - mon141 Backups Grafana on mon141 is OK: FILE_AGE OK: /var/log/grafana-backup.log is 62 seconds old and 784 bytes [04:17:01] PROBLEM - wiki.zulaclub.net - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.zulaclub.net. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [04:19:25] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - uk.religiononfire.mar.in.ua All nameservers failed to answer the query. [04:34:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.24, 3.44, 3.91 [04:46:59] RECOVERY - wiki.zulaclub.net - reverse DNS on sslhost is OK: SSL OK - wiki.zulaclub.net reverse DNS resolves to cp23.miraheze.org - CNAME OK [04:48:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.55, 3.79, 3.74 [04:48:50] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [04:52:16] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [04:52:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.59, 3.63, 3.68 [05:10:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.07, 3.70, 3.57 [05:12:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.78, 3.71, 3.58 [05:21:32] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [05:24:26] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.33, 3.27, 3.37 [05:36:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.57, 3.28, 3.29 [05:38:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.02, 3.53, 3.38 [05:40:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.43, 3.49, 3.39 [05:44:26] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.86, 3.29, 3.34 [05:47:39] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [05:48:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.71, 3.53, 3.42 [05:49:09] PROBLEM - wiki.qadrishattari.xyz - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.qadrishattari.xyz All nameservers failed to answer the query. [05:50:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.78, 3.94, 3.59 [05:56:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.43, 3.84, 3.70 [05:58:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.24, 4.00, 3.78 [06:00:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.57, 3.90, 3.77 [06:05:36] PROBLEM - wiki.templeofludus.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query templeofludus.org. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [06:06:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.55, 3.98, 3.82 [06:18:05] RECOVERY - wiki.qadrishattari.xyz - reverse DNS on sslhost is OK: SSL OK - wiki.qadrishattari.xyz reverse DNS resolves to cp23.miraheze.org - CNAME OK [06:24:30] !log [void@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/sql.php --wiki=rothwellwiki /srv/mediawiki/w/maintenance/archives/patch-watchlist_expiry.sql (END - exit=0) [06:24:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:28:01] PROBLEM - cp23 Current Load on cp23 is CRITICAL: LOAD CRITICAL - total load average: 62.38, 20.87, 7.85 [06:29:36] PROBLEM - cp22 Current Load on cp22 is CRITICAL: LOAD CRITICAL - total load average: 6.99, 5.44, 2.60 [06:33:36] RECOVERY - cp22 Current Load on cp22 is OK: LOAD OK - total load average: 0.90, 3.23, 2.38 [06:35:19] RECOVERY - wiki.templeofludus.org - reverse DNS on sslhost is OK: SSL OK - wiki.templeofludus.org reverse DNS resolves to cp23.miraheze.org - CNAME OK [06:49:58] PROBLEM - cp23 Current Load on cp23 is WARNING: LOAD WARNING - total load average: 1.54, 1.44, 3.98 [06:53:58] RECOVERY - cp23 Current Load on cp23 is OK: LOAD OK - total load average: 0.58, 1.27, 3.34 [07:33:19] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.eu.org. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [07:52:17] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query en.religiononfire.mar.in.ua. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [07:52:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.19, 3.38, 3.98 [08:03:13] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp23.miraheze.org - CNAME OK [08:16:26] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.80, 3.11, 3.38 [08:31:15] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 5.02, 4.36, 3.76 [08:40:55] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.84, 4.00, 3.91 [08:50:49] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [08:56:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.05, 3.70, 3.72 [08:58:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.81, 3.67, 3.70 [09:14:26] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.83, 3.07, 3.35 [09:40:48] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.25, 3.57, 3.41 [09:42:44] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.02, 3.34, 3.34 [10:07:20] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.92, 3.61, 3.30 [10:13:07] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.81, 3.28, 3.26 [10:28:38] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.58, 3.42, 3.32 [10:34:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.28, 3.86, 3.55 [10:36:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.31, 3.71, 3.53 [10:46:26] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.89, 3.21, 3.39 [11:06:06] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.69, 3.62, 3.37 [11:08:02] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.59, 3.57, 3.38 [11:11:54] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.74, 3.35, 3.35 [11:34:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.02, 3.42, 3.27 [11:40:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.90, 4.00, 3.61 [11:48:26] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.74, 3.09, 3.34 [12:08:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.69, 3.76, 3.38 [12:59:32] RECOVERY - cp32 NTP time on cp32 is OK: NTP OK: Offset 0.09986099601 secs [13:18:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 2.51, 3.05, 3.85 [13:22:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.90, 3.63, 3.86 [13:48:00] PROBLEM - wiki.cjgh.xyz - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.cjgh.xyz could not be found [13:50:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 2.78, 3.59, 3.90 [14:02:26] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.38, 3.03, 3.39 [14:41:27] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.85, 3.57, 3.31 [14:41:38] RECOVERY - cloud11 IPMI Sensors on cloud11 is OK: IPMI Status: OK [14:43:47] PROBLEM - tudien.como.vn - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - tudien.como.vn All nameservers failed to answer the query. [14:45:37] PROBLEM - cloud11 IPMI Sensors on cloud11 is CRITICAL: IPMI Status: Critical [Cntlr 2 Bay 8 = Critical] [14:50:43] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [14:51:06] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.42, 3.97, 3.59 [14:53:02] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.80, 3.92, 3.62 [14:54:58] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.47, 4.11, 3.72 [14:58:49] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.57, 3.85, 3.71 [15:00:45] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.07, 4.07, 3.82 [15:02:41] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.65, 3.92, 3.79 [15:13:33] PROBLEM - tudien.como.vn - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for tudien.como.vn could not be found [15:15:55] PROBLEM - wiki.cubestudios.xyz - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.cubestudios.xyz' expires in 15 day(s) (Sun 10 Sep 2023 15:09:34 GMT +0000). [15:21:07] PROBLEM - wiki.randomstuffimade.page - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.randomstuffimade.page' expires in 15 day(s) (Sun 10 Sep 2023 15:16:02 GMT +0000). [15:21:38] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/92bc0728f1fc...795b08c0483d [15:21:39] [02miraheze/ssl] 07MirahezeSSLBot 03795b08c - Bot: Update SSL cert for wiki.cubestudios.xyz [15:24:26] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.84, 3.15, 3.38 [15:30:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.19, 3.46, 3.46 [15:34:26] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.05, 3.29, 3.40 [15:49:32] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [15:52:16] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - en.religiononfire.mar.in.ua All nameservers failed to answer the query. [15:57:54] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.24, 3.72, 3.46 [16:21:32] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [16:24:56] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.44, 3.54, 3.97 [16:26:52] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.50, 3.79, 4.00 [16:49:38] PROBLEM - cp33 Current Load on cp33 is CRITICAL: LOAD CRITICAL - total load average: 11.77, 6.38, 3.74 [16:55:33] RECOVERY - cp33 Current Load on cp33 is OK: LOAD OK - total load average: 1.24, 3.10, 3.09 [17:20:50] why icinga messages are not broadcasted to discord? [17:22:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.20, 3.65, 3.95 [17:26:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.08, 3.88, 3.97 [17:28:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.76, 3.74, 3.91 [17:36:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.79, 4.02, 3.89 [17:42:26] PROBLEM - tudien.como.vn - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - tudien.como.vn All nameservers failed to answer the query. [18:12:12] PROBLEM - tudien.como.vn - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for tudien.como.vn could not be found [18:35:42] Because they are extremely spammy and would make discussions virtually impossible on discord, technically not even botifico is supposed to be relayed but somehow it keeps happening [18:47:52] Oh is it not? I kind of like seeing what commits are going through [18:48:38] ah i understand [18:49:16] and the mirahezelsbot commands? [18:49:50] also discord provides easy logging in a way [19:03:21] sorry for the ping but where i could help within the miraheze organization? technical-wise, testing, feedback, etc @orduin [19:30:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.21, 3.28, 3.93 [19:40:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.52, 3.66, 3.69 [19:44:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 4.00, 3.96, 3.82 [19:48:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.22, 3.89, 3.80 [19:58:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 2.73, 3.70, 3.84 [20:04:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.32, 3.71, 3.76 [20:06:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.95, 3.81, 3.79 [20:10:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.06, 3.89, 3.82 [20:10:30] [02miraheze/puppet] 07The-Voidwalker pushed 031 commit to 03The-Voidwalker-patch-1 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/f9bef8e81f0a [20:10:33] [02miraheze/puppet] 07The-Voidwalker 03f9bef8e - all_certs.pp assume nginx service exists [20:10:34] [02puppet] 07The-Voidwalker created branch 03The-Voidwalker-patch-1 - 13https://github.com/miraheze/puppet [20:10:37] [02puppet] 07The-Voidwalker opened pull request 03#3396: all_certs.pp assume nginx service exists - 13https://github.com/miraheze/puppet/pull/3396 [20:12:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.31, 3.74, 3.78 [20:12:55] !log [void@puppet141] nginx -t && nginx -s reload on cp* [20:13:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:13:55] RECOVERY - wiki.cubestudios.xyz - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.cubestudios.xyz' will expire on Thu 23 Nov 2023 14:21:30 GMT +0000. [20:15:06] I know in the past it was excluded from relay [20:17:22] RECOVERY - isv.orain.org - LetsEncrypt on sslhost is OK: OK - Certificate 'orain.org' will expire on Tue 21 Nov 2023 20:39:44 GMT +0000. [20:18:08] https://github.com/miraheze/puppet/blob/master/modules/irc/templates/relaybot/config.ini.erb#L33 [20:18:39] Not sure that would match “botifico” [20:19:57] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - uk.religiononfire.mar.in.ua All nameservers failed to answer the query. [20:24:53] RECOVERY - www.pyramidgames.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'pyramidgames.wiki' will expire on Wed 22 Nov 2023 14:28:59 GMT +0000. [20:25:05] RECOVERY - pyramidgames.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'pyramidgames.wiki' will expire on Wed 22 Nov 2023 14:28:59 GMT +0000. [20:27:04] [02puppet] 07paladox closed pull request 03#3396: all_certs.pp assume nginx service exists - 13https://github.com/miraheze/puppet/pull/3396 [20:27:05] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/5e297bf177e9...c4362b9109e7 [20:27:08] [02miraheze/puppet] 07The-Voidwalker 03c4362b9 - all_certs.pp assume nginx service exists (#3396) [20:27:11] [02miraheze/puppet] 07paladox deleted branch 03The-Voidwalker-patch-1 [20:27:12] [02puppet] 07paladox deleted branch 03The-Voidwalker-patch-1 - 13https://github.com/miraheze/puppet [20:28:26] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.94, 3.02, 3.32 [20:29:22] RECOVERY - allthetropes.orain.org - LetsEncrypt on sslhost is OK: OK - Certificate 'orain.org' will expire on Tue 21 Nov 2023 20:39:44 GMT +0000. [20:29:22] RECOVERY - poserdazfreebies.orain.org - LetsEncrypt on sslhost is OK: OK - Certificate 'orain.org' will expire on Tue 21 Nov 2023 20:39:44 GMT +0000. [20:29:55] RECOVERY - encyclopediarobotica.org - LetsEncrypt on sslhost is OK: OK - Certificate 'encyclopediarobotica.org' will expire on Wed 22 Nov 2023 04:12:47 GMT +0000. [20:30:29] RECOVERY - orain.org - LetsEncrypt on sslhost is OK: OK - Certificate 'orain.org' will expire on Tue 21 Nov 2023 20:39:44 GMT +0000. [20:39:20] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.10, 3.50, 3.36 [20:41:15] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.65, 3.51, 3.38 [20:43:11] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.77, 3.29, 3.31 [20:49:22] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.religiononfire.mar.in.ua. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [21:05:28] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.57, 3.33, 3.25 [21:11:16] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.25, 3.23, 3.23 [21:18:46] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [21:19:00] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_40 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/26c935976e38...0bfe72842740 [21:19:03] [02miraheze/mediawiki] 07paladox 030bfe728 - Update CentralAuth [21:19:33] !log [paladox@test131] starting deploy of {'world': True, 'l10n': True} to all [21:19:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:22:16] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - en.religiononfire.mar.in.ua All nameservers failed to answer the query. [21:29:18] !log [paladox@test131] finished deploy of {'world': True, 'l10n': True} to all - SUCCESS in 584s [21:29:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:29:32] RECOVERY - ns1 NTP time on ns1 is OK: NTP OK: Offset 0.01255449653 secs [21:31:35] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.45, 3.20, 3.07 [21:31:59] PROBLEM - cp23 Current Load on cp23 is CRITICAL: LOAD CRITICAL - total load average: 6.31, 3.84, 2.15 [21:33:31] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.25, 3.20, 3.09 [21:33:32] PROBLEM - ns1 NTP time on ns1 is WARNING: NTP WARNING: Offset 0.1067777574 secs [21:35:58] RECOVERY - cp23 Current Load on cp23 is OK: LOAD OK - total load average: 1.26, 3.40, 2.47 [21:51:32] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query en.religiononfire.mar.in.ua. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [21:52:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.35, 3.86, 3.47 [22:02:10] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [22:17:01] PROBLEM - os131 Current Load on os131 is WARNING: WARNING - load average: 3.70, 2.74, 1.54 [22:19:03] RECOVERY - os131 Current Load on os131 is OK: OK - load average: 1.90, 2.37, 1.55 [22:20:48] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [22:32:09] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['ns.ankh.fr.eu.org.', 'ns1.eu.org.', 'ns1.eriomem.net.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [22:33:50] !log install npm on test131 [22:33:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:54:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.01, 3.50, 3.90 [23:02:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.36, 3.76, 3.84 [23:03:28] PROBLEM - db112 Current Load on db112 is CRITICAL: CRITICAL - load average: 13.49, 7.71, 3.55 [23:04:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 4.00, 3.88, 3.88 [23:06:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.08, 3.90, 3.89 [23:07:28] RECOVERY - db112 Current Load on db112 is OK: OK - load average: 2.43, 6.64, 4.23 [23:08:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 2.94, 3.57, 3.77 [23:12:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.32, 3.83, 3.83 [23:18:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.66, 3.99, 3.95 [23:22:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.01, 3.82, 3.87 [23:24:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 2.72, 3.53, 3.77 [23:38:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.37, 3.74, 3.68 [23:40:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.05, 3.61, 3.65