[00:02:50] PROBLEM - cp22 Current Load on cp22 is CRITICAL: LOAD CRITICAL - total load average: 18.29, 11.92, 5.02 [00:02:56] PROBLEM - cp23 Current Load on cp23 is CRITICAL: LOAD CRITICAL - total load average: 4.41, 4.32, 2.08 [00:04:56] RECOVERY - cp23 Current Load on cp23 is OK: LOAD OK - total load average: 1.14, 3.22, 1.96 [00:14:45] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.65, 3.61, 3.22 [00:14:56] PROBLEM - cp23 Current Load on cp23 is WARNING: LOAD WARNING - total load average: 3.89, 3.34, 2.42 [00:16:56] RECOVERY - cp23 Current Load on cp23 is OK: LOAD OK - total load average: 1.55, 2.63, 2.27 [00:17:26] PROBLEM - wiki.altilunium.my.id - LetsEncrypt on sslhost is CRITICAL: connect to address wiki.altilunium.my.id and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [00:20:38] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.20, 3.34, 3.22 [00:28:50] PROBLEM - cp22 Current Load on cp22 is WARNING: LOAD WARNING - total load average: 0.57, 1.49, 3.81 [00:30:50] RECOVERY - cp22 Current Load on cp22 is OK: LOAD OK - total load average: 0.37, 1.08, 3.37 [00:34:18] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.65, 3.48, 3.33 [00:42:10] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.26, 2.99, 3.20 [01:11:32] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.53, 4.04, 3.45 [01:13:30] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.62, 3.97, 3.50 [01:15:28] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 3.88, 4.03, 3.59 [01:19:24] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.72, 3.94, 3.66 [01:23:19] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.10, 3.93, 3.71 [01:35:06] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.37, 3.94, 3.91 [01:39:02] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.58, 4.11, 3.97 [02:04:58] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 2.90, 3.40, 3.93 [02:06:18] RECOVERY - ping6 on cp33 is OK: PING OK - Packet loss = 0%, RTA = 119.34 ms [02:06:24] RECOVERY - ping6 on cp32 is OK: PING OK - Packet loss = 0%, RTA = 116.74 ms [02:06:28] RECOVERY - ping6 on ns1 is OK: PING OK - Packet loss = 0%, RTA = 105.44 ms [02:09:40] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - en.religiononfire.mar.in.ua All nameservers failed to answer the query. [02:17:41] RECOVERY - wiki.altilunium.my.id - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.altilunium.my.id' will expire on Wed 11 Oct 2023 05:49:31 GMT +0000. [02:18:58] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.91, 3.03, 3.40 [02:39:07] !log [paladox@mwtask141] starting deploy of {'world': True, 'l10n': True} to all [02:39:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:46:16] !log [paladox@mwtask141] finished deploy of {'world': True, 'l10n': True} to all - SUCCESS in 428s [02:46:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:49:52] PROBLEM - wiki.altilunium.my.id - LetsEncrypt on sslhost is CRITICAL: connect to address wiki.altilunium.my.id and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [03:00:56] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ssl/commit/d8f802323168 [03:00:58] [02miraheze/ssl] 07paladox 03d8f8023 - Don't redirect avid.miraheze.org [03:01:01] [02ssl] 07paladox created branch 03paladox-patch-1 - 13https://github.com/miraheze/ssl [03:01:02] [02ssl] 07paladox opened pull request 03#715: Don't redirect avid.miraheze.org - 13https://github.com/miraheze/ssl/pull/715 [03:01:08] [02ssl] 07paladox closed pull request 03#715: Don't redirect avid.miraheze.org - 13https://github.com/miraheze/ssl/pull/715 [03:01:09] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/cb534a348b7f...24a4b5ca827c [03:01:10] [02miraheze/ssl] 07paladox 0324a4b5c - Don't redirect avid.miraheze.org (#715) [03:01:13] [02miraheze/ssl] 07paladox deleted branch 03paladox-patch-1 [03:01:15] [02ssl] 07paladox deleted branch 03paladox-patch-1 - 13https://github.com/miraheze/ssl [03:05:00] !log [paladox@mwtask141] starting deploy of {'world': True, 'l10n': True} to all [03:05:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:07:59] RECOVERY - wiki.ff6worldscollide.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.ff6worldscollide.com' will expire on Thu 09 Nov 2023 16:13:51 GMT +0000. [03:08:39] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [03:09:21] !log [paladox@mwtask141] finished deploy of {'world': True, 'l10n': True} to all - SUCCESS in 260s [03:09:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:10:21] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 5.05, 3.92, 3.20 [03:12:19] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.54, 3.66, 3.18 [03:20:10] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.74, 3.23, 3.17 [03:23:47] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.77, 3.54, 3.31 [03:25:11] [02miraheze/mediawiki] 07paladox deleted branch 03REL1_39_1 [03:25:12] [02mediawiki] 07paladox deleted branch 03REL1_39_1 - 13https://github.com/miraheze/mediawiki [03:25:43] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.20, 3.07, 3.17 [03:29:24] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_40 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/2b100d145de5...e13cee9ab0c4 [03:29:27] [02miraheze/mediawiki] 07paladox 03e13cee9 - Update TabberNeue [03:30:35] !log [paladox@test131] starting deploy of {'world': True, 'l10n': True} to all [03:30:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:48:43] !log [paladox@test131] finished deploy of {'world': True, 'l10n': True} to all - SUCCESS in 1087s [03:48:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:56:23] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/e22f639ea3f6 [03:56:25] [02miraheze/mw-config] 07paladox 03e22f639 - Switch memcaches around [03:56:27] [02mw-config] 07paladox created branch 03paladox-patch-1 - 13https://github.com/miraheze/mw-config [03:56:30] [02mw-config] 07paladox opened pull request 03#5297: Switch memcaches around - 13https://github.com/miraheze/mw-config/pull/5297 [03:56:52] [02mw-config] 07paladox closed pull request 03#5297: Switch memcaches around - 13https://github.com/miraheze/mw-config/pull/5297 [03:56:53] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/9f8d93b2973f...c20812ed0479 [03:56:55] [02miraheze/mw-config] 07paladox 03c20812e - Switch memcaches around (#5297) [03:56:58] [02miraheze/mw-config] 07paladox deleted branch 03paladox-patch-1 [03:56:59] [02mw-config] 07paladox deleted branch 03paladox-patch-1 - 13https://github.com/miraheze/mw-config [03:57:03] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [03:57:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:57:14] !log [paladox@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 11s [03:57:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:57:22] !log [paladox@mwtask141] starting deploy of {'config': True} to all [03:57:27] miraheze/mw-config - paladox the build passed. [03:57:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:57:35] !log [paladox@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 13s [03:57:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:57:56] miraheze/mw-config - paladox the build passed. [04:03:25] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.eu.org. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [04:20:13] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-11 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/7b48c3d17244 [04:20:15] [02miraheze/puppet] 07paladox 037b48c3d - Allow caching api.php/rest.php [04:20:18] [02puppet] 07paladox created branch 03paladox-patch-11 - 13https://github.com/miraheze/puppet [04:20:22] [02puppet] 07paladox opened pull request 03#3382: varnish: Allow caching api.php/rest.php - 13https://github.com/miraheze/puppet/pull/3382 [04:23:33] [02puppet] 07paladox closed pull request 03#3382: varnish: Allow caching api.php/rest.php - 13https://github.com/miraheze/puppet/pull/3382 [04:23:36] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/281d5eeef446...6104a26f6a55 [04:23:38] [02miraheze/puppet] 07paladox 036104a26 - varnish: Allow caching api.php/rest.php (#3382) [04:23:39] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-11 [04:23:41] [02puppet] 07paladox deleted branch 03paladox-patch-11 - 13https://github.com/miraheze/puppet [04:26:39] !log [@test131] starting deploy of {'config': True} to all [04:26:40] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [04:26:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:26:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:31:28] PROBLEM - wiki.zulaclub.net - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.zulaclub.net All nameservers failed to answer the query. [04:32:33] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp22.miraheze.org - CNAME OK [04:45:35] [02puppet] 07paladox closed pull request 03#3378: reload-vcl: discard reload_ vcl as well - 13https://github.com/miraheze/puppet/pull/3378 [04:45:36] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-10 [04:45:38] [02puppet] 07paladox deleted branch 03paladox-patch-10 - 13https://github.com/miraheze/puppet [04:46:24] [02puppet] 07paladox synchronize pull request 03#3357: miraheze-backup: backup mediawiki-xml dumps to archive - 13https://github.com/miraheze/puppet/pull/3357 [04:46:25] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-9 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/fc52e66ebd9c...695aa55a9be5 [04:46:28] [02miraheze/puppet] 07paladox 03695aa55 - Update miraheze-backup.py.erb [04:53:08] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.13, 3.48, 3.15 [04:53:44] PROBLEM - wiki.altilunium.my.id - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.altilunium.my.id All nameservers failed to answer the query. [05:00:49] RECOVERY - wiki.zulaclub.net - reverse DNS on sslhost is OK: SSL OK - wiki.zulaclub.net reverse DNS resolves to cp23.miraheze.org - CNAME OK [05:22:29] PROBLEM - wiki.altilunium.my.id - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.altilunium.my.id could not be found [05:29:44] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/c20812ed0479...4dc3d9f43512 [05:29:46] [02miraheze/mw-config] 07paladox 034dc3d9f - Switch $wgMainCacheType to memcached-mem-1 [05:29:56] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [05:30:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:30:08] !log [paladox@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 12s [05:30:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:30:12] !log [paladox@mwtask141] starting deploy of {'config': True} to all [05:30:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:30:19] !log [paladox@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 6s [05:30:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:30:38] miraheze/mw-config - paladox the build passed. [05:38:52] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/4dc3d9f43512...03a4027da76e [05:38:53] [02miraheze/mw-config] 07paladox 0303a4027 - Revert "Switch $wgMainCacheType to memcached-mem-1" [05:39:07] !log [@mwtask141] starting deploy of {'config': True} to all [05:39:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:39:14] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/03a4027da76e...cd81b3854d19 [05:39:15] !log [@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 8s [05:39:15] [02miraheze/mw-config] 07paladox 03cd81b38 - Switch mysql-multiwrite to memcached-mem-1 [05:39:18] !log [paladox@mwtask141] starting deploy of {'config': True} to all [05:39:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:39:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:39:25] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [05:39:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:39:33] !log [paladox@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 8s [05:39:35] !log [paladox@mwtask141] starting deploy of {'config': True} to all [05:39:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:39:43] !log [paladox@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 7s [05:39:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:39:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:39:55] miraheze/mw-config - paladox the build passed. [05:40:17] miraheze/mw-config - paladox the build passed. [05:52:32] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 2.13, 2.91, 3.88 [05:56:09] !log [@test131] starting deploy of {'config': True} to all [05:56:10] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [05:56:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:56:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:58:32] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.25, 2.38, 3.33 [06:05:19] PROBLEM - puppetdb121 SSH on puppetdb121 is CRITICAL: connect to address 2a10:6740::6:313 and port 22: No route to host [06:05:27] PROBLEM - Host puppetdb121 is DOWN: CRITICAL - Destination Unreachable (2a10:6740::6:313) [06:05:27] PROBLEM - puppetdb121 ferm_active on puppetdb121 is CRITICAL: connect to address 2a10:6740::6:313 port 5666: No route to hostconnect to host 2a10:6740::6:313 port 5666: No route to host [06:07:09] PROBLEM - mw134 Puppet on mw134 is UNKNOWN: NRPE: Unable to read output [06:12:32] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.64, 3.45, 3.11 [06:14:32] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.91, 3.48, 3.15 [06:18:32] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.10, 3.62, 3.27 [06:28:27] PROBLEM - cp23 Current Load on cp23 is CRITICAL: LOAD CRITICAL - total load average: 10.21, 13.57, 6.35 [06:28:32] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.27, 3.81, 3.61 [06:36:27] PROBLEM - cp23 Current Load on cp23 is WARNING: LOAD WARNING - total load average: 0.67, 3.15, 3.96 [06:37:08] RECOVERY - mw134 Puppet on mw134 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [06:40:27] RECOVERY - cp23 Current Load on cp23 is OK: LOAD OK - total load average: 0.23, 1.55, 3.11 [06:44:22] PROBLEM - reports121 NTP time on reports121 is CRITICAL: NTP CRITICAL: Offset 0.6041014493 secs [06:44:32] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.31, 3.90, 3.70 [06:48:32] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.41, 3.78, 3.70 [06:50:32] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.27, 3.96, 3.78 [06:58:32] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.76, 3.98, 3.90 [07:14:52] RECOVERY - reports121 NTP time on reports121 is OK: NTP OK: Offset 3.159046173e-05 secs [07:16:32] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.17, 2.99, 3.37 [07:21:05] PROBLEM - reports121 NTP time on reports121 is WARNING: NTP WARNING: Offset 0.471688509 secs [07:24:00] PROBLEM - cloud13 Puppet on cloud13 is UNKNOWN: NRPE: Unable to read output [07:27:11] PROBLEM - reports121 NTP time on reports121 is CRITICAL: NTP CRITICAL: Offset 0.9087584913 secs [07:31:15] RECOVERY - reports121 NTP time on reports121 is OK: NTP OK: Offset 0.05222541094 secs [07:49:43] PROBLEM - reports121 NTP time on reports121 is WARNING: NTP WARNING: Offset 0.4786115885 secs [07:50:00] RECOVERY - cloud13 Puppet on cloud13 is OK: OK: Puppet is currently enabled, last run 23 seconds ago with 0 failures [08:20:13] RECOVERY - reports121 NTP time on reports121 is OK: NTP OK: Offset -0.01191923022 secs [08:42:02] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [09:12:02] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['NS.ANKH.FR.eu.org.', 'NS1.eu.org.', 'NS1.ERIOMEM.NET.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [09:33:49] RECOVERY - wiki.altilunium.my.id - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.altilunium.my.id' will expire on Wed 11 Oct 2023 05:49:31 GMT +0000. [10:04:33] PROBLEM - wiki.altilunium.my.id - LetsEncrypt on sslhost is CRITICAL: connect to address wiki.altilunium.my.id and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [10:49:58] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset -0.4995683432 secs [12:12:32] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.67, 3.31, 2.55 [12:14:32] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.23, 3.24, 2.61 [12:27:19] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.81, 3.52, 3.08 [12:31:12] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.88, 3.24, 3.07 [12:40:55] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.41, 3.93, 3.42 [12:42:51] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.10, 3.56, 3.34 [12:44:47] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.52, 3.21, 3.24 [12:54:32] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.20, 3.60, 3.37 [12:56:32] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.84, 3.63, 3.41 [13:00:32] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.20, 3.15, 3.28 [13:11:36] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.religiononfire.mar.in.ua. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [13:30:09] RECOVERY - wiki.altilunium.my.id - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.altilunium.my.id' will expire on Wed 11 Oct 2023 05:49:31 GMT +0000. [13:40:40] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - uk.religiononfire.mar.in.ua All nameservers failed to answer the query. [14:04:07] PROBLEM - reports121 NTP time on reports121 is CRITICAL: NTP CRITICAL: Offset 0.541847676 secs [14:09:43] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [14:10:13] RECOVERY - reports121 NTP time on reports121 is OK: NTP OK: Offset 0.0007719993591 secs [14:12:02] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [14:33:04] PROBLEM - wiki.altilunium.my.id - LetsEncrypt on sslhost is CRITICAL: connect to address wiki.altilunium.my.id and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [14:42:02] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['ns.ankh.fr.eu.org.', 'ns1.eu.org.', 'ns1.eriomem.net.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [14:44:54] PROBLEM - reports121 NTP time on reports121 is CRITICAL: NTP CRITICAL: Offset 0.6214890182 secs [15:10:32] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.41, 2.97, 2.43 [15:12:32] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.92, 3.02, 2.53 [15:17:30] RECOVERY - reports121 NTP time on reports121 is OK: NTP OK: Offset 0.001211673021 secs [16:05:28] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/6104a26f6a55...cf91919a6bd8 [16:05:29] [02miraheze/puppet] 07paladox 03cf91919 - mediawiki: don't run mediawiki-xml backups temporarily [16:42:59] PROBLEM - mwtask141 Backups MediaWiki XML on mwtask141 is UNKNOWN: NRPE: Command 'check_Backups MediaWiki XML' not defined [18:01:09] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/6f56aeb917b9 [18:01:11] [02miraheze/mw-config] 07paladox 036f56aeb - Use HookContainer [18:01:13] [02mw-config] 07paladox created branch 03paladox-patch-1 - 13https://github.com/miraheze/mw-config [18:01:15] [02mw-config] 07paladox opened pull request 03#5298: Use HookContainer - 13https://github.com/miraheze/mw-config/pull/5298 [18:02:15] miraheze/mw-config - paladox the build passed. [18:08:45] PROBLEM - reports121 NTP time on reports121 is WARNING: NTP WARNING: Offset 0.4066672921 secs [18:09:38] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.83, 3.47, 2.59 [18:15:38] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.84, 3.94, 3.11 [18:21:38] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.28, 3.35, 3.11 [18:27:38] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.60, 3.92, 3.40 [18:29:38] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.99, 3.93, 3.47 [18:31:38] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.38, 4.06, 3.58 [18:33:38] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.33, 3.85, 3.56 [18:39:38] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.27, 3.73, 3.57 [18:41:37] RECOVERY - reports121 NTP time on reports121 is OK: NTP OK: Offset -0.0001196861267 secs [18:41:38] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.00, 3.59, 3.55 [18:51:38] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.32, 3.84, 3.64 [18:55:38] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.53, 3.81, 3.69 [18:59:50] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-3 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/a9d12b46330a [18:59:52] [02miraheze/mw-config] 07paladox 03a9d12b4 - Fix logic [18:59:54] [02mw-config] 07paladox created branch 03paladox-patch-3 - 13https://github.com/miraheze/mw-config [18:59:55] [02mw-config] 07paladox opened pull request 03#5299: Fix logic - 13https://github.com/miraheze/mw-config/pull/5299 [19:00:55] miraheze/mw-config - paladox the build passed. [19:01:38] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 1.79, 2.91, 3.38 [19:02:28] [02mw-config] 07paladox closed pull request 03#5299: Fix logic - 13https://github.com/miraheze/mw-config/pull/5299 [19:02:30] [02miraheze/mw-config] 07paladox deleted branch 03paladox-patch-3 [19:02:33] [02mw-config] 07paladox deleted branch 03paladox-patch-3 - 13https://github.com/miraheze/mw-config [19:08:22] PROBLEM - cloud10 APT on cloud10 is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. [19:10:23] PROBLEM - cloud10 APT on cloud10 is CRITICAL: APT CRITICAL: 109 packages available for upgrade (7 critical updates). [19:23:38] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.73, 3.11, 2.85 [19:25:38] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.91, 3.05, 2.86 [20:11:25] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-3 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/5c833adac17c [20:11:28] [02miraheze/mw-config] 07paladox 035c833ad - Store subdomain in url field in cache [20:11:31] [02mw-config] 07paladox created branch 03paladox-patch-3 - 13https://github.com/miraheze/mw-config [20:11:33] [02mw-config] 07paladox opened pull request 03#5300: Store subdomain in url field in cache - 13https://github.com/miraheze/mw-config/pull/5300 [20:12:37] miraheze/mw-config - paladox the build has errored. [20:16:51] [02mw-config] 07paladox synchronize pull request 03#5300: Store subdomain in url field in cache - 13https://github.com/miraheze/mw-config/pull/5300 [20:16:54] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-3 [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/5c833adac17c...78aa9a82d333 [20:16:55] [02miraheze/mw-config] 07paladox 0378aa9a8 - Update MirahezeFunctions.php [20:17:50] miraheze/mw-config - paladox the build passed. [20:28:03] [02mw-config] 07paladox synchronize pull request 03#5300: Store subdomain in url field in cache - 13https://github.com/miraheze/mw-config/pull/5300 [20:28:06] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-3 [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/78aa9a82d333...8429af2a2958 [20:28:08] [02miraheze/mw-config] 07paladox 038429af2 - Update MirahezeFunctions.php [20:29:05] miraheze/mw-config - paladox the build passed. [20:29:33] [02mw-config] 07paladox synchronize pull request 03#5300: Store subdomain in url field in cache - 13https://github.com/miraheze/mw-config/pull/5300 [20:29:36] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-3 [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/8429af2a2958...26a9d5b13cd3 [20:29:39] [02miraheze/mw-config] 07paladox 0326a9d5b - Update MirahezeFunctions.php [20:30:33] miraheze/mw-config - paladox the build has errored. [20:35:05] [02mw-config] 07paladox synchronize pull request 03#5300: Store subdomain in url field in cache - 13https://github.com/miraheze/mw-config/pull/5300 [20:35:07] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-3 [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/26a9d5b13cd3...8f1453542f37 [20:35:09] [02miraheze/mw-config] 07paladox 038f14535 - Update MirahezeFunctions.php [20:36:10] miraheze/mw-config - paladox the build passed. [21:03:11] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [21:15:45] RECOVERY - cp32 NTP time on cp32 is OK: NTP OK: Offset 0.004304111004 secs [21:28:15] [02mw-config] 07paladox synchronize pull request 03#5300: Store subdomain in url field in cache - 13https://github.com/miraheze/mw-config/pull/5300 [21:28:15] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-3 [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/8f1453542f37...9f70c34dc38d [21:28:16] [02miraheze/mw-config] 07paladox 039f70c34 - Update MirahezeFunctions.php [21:29:12] miraheze/mw-config - paladox the build passed. [21:33:12] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.gab.pt.eu.org. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [21:41:20] !log [paladox@test131] starting deploy of {'config': True} to all [21:41:22] !log [paladox@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [21:41:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:41:30] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:03:14] PROBLEM - cp22 Current Load on cp22 is CRITICAL: LOAD CRITICAL - total load average: 3.10, 4.69, 2.47 [23:04:30] PROBLEM - db112 Current Load on db112 is CRITICAL: CRITICAL - load average: 13.62, 8.49, 4.00 [23:05:14] RECOVERY - cp22 Current Load on cp22 is OK: LOAD OK - total load average: 1.11, 3.40, 2.26 [23:08:30] PROBLEM - db112 Current Load on db112 is WARNING: WARNING - load average: 2.95, 7.52, 4.81 [23:10:30] RECOVERY - db112 Current Load on db112 is OK: OK - load average: 0.80, 5.19, 4.28 [23:42:36] RECOVERY - arcanewaterswiki.org - reverse DNS on sslhost is OK: SSL OK - arcanewaterswiki.org reverse DNS resolves to cp23.miraheze.org - NS RECORDS OK