[00:02:11] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 6.89, 4.73, 3.08 [00:04:11] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 2.62, 3.89, 2.98 [00:06:11] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 2.14, 3.38, 2.91 [00:06:32] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.67, 4.29, 3.36 [00:06:45] PROBLEM - cp22 Current Load on cp22 is CRITICAL: CRITICAL - load average: 9.97, 8.76, 4.33 [00:08:29] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.44, 3.86, 3.31 [00:10:26] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.78, 4.78, 3.72 [00:12:39] PROBLEM - notes.mridulpm.in - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:12:46] PROBLEM - portalsofphereon.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:13:21] PROBLEM - mw132 Current Load on mw132 is CRITICAL: CRITICAL - load average: 12.96, 9.78, 6.54 [00:13:54] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.71, 3.30, 2.96 [00:14:51] PROBLEM - cp22 SSH on cp22 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:15:06] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.60, 3.36, 2.68 [00:15:21] RECOVERY - mw132 Current Load on mw132 is OK: OK - load average: 7.32, 8.58, 6.49 [00:15:38] PROBLEM - cp22 PowerDNS Recursor on cp22 is CRITICAL: CRITICAL - Plugin timed out while executing system call [00:15:52] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 2.97, 3.06, 2.90 [00:16:16] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 217.174.247.33/cpweb, 2a00:da00:1800:326::1/cpweb [00:17:05] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.56, 3.07, 2.66 [00:17:34] PROBLEM - cp22 HTTP 4xx/5xx ERROR Rate on cp22 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [00:17:34] PROBLEM - cp22 Puppet on cp22 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [00:19:18] [02miraheze/ssl] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/435949bab33b...bd9383e76de4 [00:19:19] [url] Comparing 435949bab33b...bd9383e76de4 · miraheze/ssl · GitHub | github.com [00:19:19] [02miraheze/ssl] 07MacFan4000 03bd9383e - Update wiki.cryptoroyale.one.crt [00:19:37] PROBLEM - cp22 conntrack_table_size on cp22 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [00:19:46] PROBLEM - cp22 Varnish Backends on cp22 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [00:19:47] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 41.99, 19.20, 8.03 [00:20:04] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 4 datacenters are down: 217.174.247.33/cpweb, 109.228.51.216/cpweb, 2a00:da00:1800:326::1/cpweb, 2a00:da00:1800:328::1/cpweb [00:20:11] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.65, 3.58, 3.71 [00:20:22] PROBLEM - cp23 SSH on cp23 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:20:25] PROBLEM - cp23 HTTPS on cp23 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:20:51] PROBLEM - cp22 HTTPS on cp22 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:21:23] PROBLEM - cp23 PowerDNS Recursor on cp23 is CRITICAL: CRITICAL - Plugin timed out while executing system call [00:21:58] PROBLEM - cp23 Puppet on cp23 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [00:22:15] [02miraheze/ssl] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/bd9383e76de4...e36f697273b3 [00:22:16] [url] Comparing bd9383e76de4...e36f697273b3 · miraheze/ssl · GitHub | github.com [00:22:16] [02miraheze/ssl] 07MacFan4000 03e36f697 - Update ethene.wiki.crt [00:22:24] RECOVERY - cp23 HTTPS on cp23 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 3426 bytes in 0.159 second response time [00:22:26] RECOVERY - cp23 SSH on cp23 is OK: SSH OK - OpenSSH_8.4p1 Debian-5+deb11u1 (protocol 2.0) [00:24:53] RECOVERY - cp22 PowerDNS Recursor on cp22 is OK: DNS OK: 0.616 seconds response time. miraheze.org returns 109.228.51.216,217.174.247.33,2a00:da00:1800:326::1,2a00:da00:1800:328::1 [00:25:02] RECOVERY - cp23 PowerDNS Recursor on cp23 is OK: DNS OK: 0.187 seconds response time. miraheze.org returns 109.228.51.216,217.174.247.33,2a00:da00:1800:326::1,2a00:da00:1800:328::1 [00:25:05] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.75, 2.77, 3.29 [00:25:22] RECOVERY - cp22 Puppet on cp22 is OK: OK: Puppet is currently enabled, last run 35 minutes ago with 0 failures [00:25:28] RECOVERY - cp22 conntrack_table_size on cp22 is OK: OK: nf_conntrack is 1 % full [00:25:28] RECOVERY - cp22 HTTP 4xx/5xx ERROR Rate on cp22 is OK: OK - NGINX Error Rate is 6% [00:25:30] RECOVERY - cp22 Varnish Backends on cp22 is OK: All 14 backends are healthy [00:25:31] RECOVERY - cp22 SSH on cp22 is OK: SSH OK - OpenSSH_8.4p1 Debian-5+deb11u1 (protocol 2.0) [00:25:35] RECOVERY - cp22 HTTPS on cp22 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 3428 bytes in 0.091 second response time [00:25:38] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [00:25:38] RECOVERY - cp23 Puppet on cp23 is OK: OK: Puppet is currently enabled, last run 26 minutes ago with 0 failures [00:25:46] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [00:26:22] PROBLEM - db141 MariaDB on db141 is CRITICAL: Can't connect to MySQL server on 'db141.miraheze.org' (115) [00:29:20] PROBLEM - cp33 Puppet on cp33 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [00:31:34] PROBLEM - cp23 Puppet on cp23 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [00:35:54] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.49, 4.21, 3.71 [00:37:52] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.86, 3.99, 3.71 [00:39:13] MacFan4000: ssl broke nginx on cp* [00:39:41] ? [00:40:01] Expected I believe as private keys have to be replaced [00:40:15] Afaik it will be fixed next puppet run [00:40:30] this is because we of course lost all certbot data [00:41:08] MacFan4000: will stop alerting next puppet run as it doesn't try to reload nginx again, but doesn't mean it is fixed I don't think? [00:41:48] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.69, 2.77, 3.27 [00:42:15] PROBLEM - notes.mridulpm.in - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'notes.mridulpm.in' expires in 11 day(s) (Sat 03 Dec 2022 11:10:16 GMT +0000). [00:42:19] PROBLEM - portalsofphereon.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.portalsofphereon.com' expires in 10 day(s) (Fri 02 Dec 2022 13:30:32 GMT +0000). [00:43:01] I don’t think I did anything incorrectly [00:46:28] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 0.82, 1.62, 3.64 [00:46:29] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 5.81, 4.46, 3.51 [00:48:26] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 2.62, 3.79, 3.38 [00:48:26] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 1.56, 1.49, 3.34 [00:49:22] RECOVERY - www.ethene.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'ethene.wiki' will expire on Sun 19 Feb 2023 23:21:39 GMT +0000. [00:50:22] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 6.82, 4.44, 3.63 [00:53:07] PROBLEM - wiki.elysiamc.tk - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.elysiamc.tk could not be found [00:54:14] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 2.40, 3.41, 3.40 [00:55:26] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.18, 3.28, 2.67 [00:56:10] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 1.56, 2.79, 3.17 [00:59:25] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.57, 3.39, 2.90 [01:00:14] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 5.85, 4.18, 3.44 [01:01:30] RECOVERY - cp33 Puppet on cp33 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [01:03:28] RECOVERY - wiki.cryptoroyale.one - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.cryptoroyale.one' will expire on Sun 19 Feb 2023 23:18:50 GMT +0000. [01:04:10] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 1.17, 3.16, 3.26 [01:04:25] RECOVERY - cp23 Puppet on cp23 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [01:05:34] PROBLEM - cp22 Current Load on cp22 is WARNING: WARNING - load average: 0.60, 0.87, 3.92 [01:06:22] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.12, 4.66, 3.38 [01:09:26] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 0.07, 0.49, 3.09 [01:16:44] RECOVERY - ethene.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'ethene.wiki' will expire on Sun 19 Feb 2023 23:21:39 GMT +0000. [01:24:33] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/45aa868d3530...3cc1ba5aca19 [01:24:33] [url] Comparing 45aa868d3530...3cc1ba5aca19 · miraheze/mw-config · GitHub | github.com [01:24:36] [02miraheze/mw-config] 07Universal-Omega 033cc1ba5 - Re-enable ImportDump requests [01:25:40] miraheze/mw-config - Universal-Omega the build passed. [01:27:00] !log [@test131] starting deploy of {'config': True} to all [01:27:01] !log [@test131] DEPLOY ABORTED: Canary check failed for beta.betaheze.org@localhost [01:27:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:27:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:30:03] [02miraheze/MirahezeMagic] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/52ed56c33c89...d442c124000e [01:30:03] [url] Comparing 52ed56c33c89...d442c124000e · miraheze/MirahezeMagic · GitHub | github.com [01:30:06] [02miraheze/MirahezeMagic] 07Universal-Omega 03d442c12 - Add miraheze-global-renames-disabled message [01:31:04] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/6a775a294d25...03ca6e3c9d80 [01:31:04] [url] Comparing 6a775a294d25...03ca6e3c9d80 · miraheze/puppet · GitHub | github.com [01:31:06] [02miraheze/puppet] 07Universal-Omega 0303ca6e3 - Absent manageInactiveWikis.php cron [01:38:59] miraheze/MirahezeMagic - Universal-Omega the build passed. [01:39:56] !log [@mwtask141] starting deploy of {'config': True} to all [01:40:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:40:06] !log [@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 9s [01:40:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:50:51] RECOVERY - wiki.elysiamc.tk - reverse DNS on sslhost is OK: SSL OK - wiki.elysiamc.tk reverse DNS resolves to cp23.miraheze.org - CNAME OK [01:56:17] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.43, 3.34, 2.79 [01:58:17] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.56, 3.28, 2.85 [02:00:16] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 8.05, 4.92, 2.41 [02:06:10] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 0.49, 3.64, 2.93 [02:08:08] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 0.49, 2.60, 2.63 [02:20:22] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.11, 2.29, 3.78 [02:21:39] PROBLEM - wiki.elysiamc.tk - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.elysiamc.tk could not be found [02:24:22] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.36, 3.37, 3.79 [02:26:22] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.75, 2.63, 3.47 [02:27:16] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.11, 3.60, 2.89 [02:28:22] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.01, 2.10, 3.17 [02:29:14] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 2.48, 3.17, 2.81 [02:32:31] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.4443313777 secs [02:44:31] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset -0.005779862404 secs [02:45:06] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.88, 3.14, 2.73 [02:47:05] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.52, 2.65, 2.61 [02:57:05] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 8.41, 4.65, 3.37 [03:00:17] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 5.27, 3.77, 2.61 [03:02:15] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 3.64, 3.91, 2.81 [03:03:30] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 74.208.203.152/cpweb, 2607:f1c0:1800:26f::1/cpweb [03:04:13] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 0.92, 2.79, 2.53 [03:05:29] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [03:06:11] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.16, 4.56, 3.29 [03:08:10] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.69, 3.93, 3.22 [03:09:58] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.1871373355 secs [03:10:08] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 7.60, 5.83, 4.04 [03:15:17] PROBLEM - cp22 Current Load on cp22 is CRITICAL: CRITICAL - load average: 17.64, 8.66, 3.84 [03:16:02] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.82, 3.52, 3.60 [03:16:15] PROBLEM - cp22 PowerDNS Recursor on cp22 is CRITICAL: CRITICAL - Plugin timed out while executing system call [03:18:00] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.94, 3.01, 3.40 [03:18:13] RECOVERY - cp22 PowerDNS Recursor on cp22 is OK: DNS OK: 1.131 second response time. miraheze.org returns 109.228.51.216,217.174.247.33,2a00:da00:1800:326::1,2a00:da00:1800:328::1 [03:19:24] RECOVERY - wiki.elysiamc.tk - reverse DNS on sslhost is OK: SSL OK - wiki.elysiamc.tk reverse DNS resolves to cp23.miraheze.org - CNAME OK [03:27:55] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.72, 5.86, 4.45 [03:29:46] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 6.60, 3.66, 2.19 [03:29:57] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.08270338178 secs [03:31:44] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 3.54, 3.86, 2.47 [03:31:51] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.55, 3.74, 3.89 [03:32:06] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.79, 2.59, 4.00 [03:33:42] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 1.06, 2.82, 2.25 [03:35:08] PROBLEM - cp22 Current Load on cp22 is WARNING: WARNING - load average: 0.56, 1.96, 3.97 [03:36:05] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.81, 1.89, 3.37 [03:37:45] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 3.77, 4.18, 3.98 [03:39:08] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 0.22, 1.07, 3.14 [03:39:43] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.16, 3.05, 3.59 [03:41:41] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 0.85, 2.30, 3.25 [03:47:08] PROBLEM - cp22 Current Load on cp22 is WARNING: WARNING - load average: 3.67, 3.15, 3.18 [03:49:08] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 0.79, 2.28, 2.86 [03:50:17] PROBLEM - wiki.elysiamc.tk - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.elysiamc.tk could not be found [03:55:56] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.19, 4.65, 3.84 [03:59:55] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.82, 3.36, 3.51 [04:01:09] PROBLEM - cp22 Current Load on cp22 is CRITICAL: CRITICAL - load average: 5.17, 3.73, 2.97 [04:01:54] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.82, 2.95, 3.35 [04:03:08] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 1.63, 3.07, 2.84 [04:07:17] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.36, 4.01, 3.33 [04:17:08] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.40, 3.51, 3.73 [04:19:06] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 0.70, 2.55, 3.35 [04:28:54] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 7.00, 5.43, 4.04 [04:30:44] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.46, 3.38, 2.70 [04:32:43] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.11, 2.45, 2.43 [04:32:51] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.44, 3.32, 3.50 [04:36:47] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.91, 4.02, 3.71 [04:40:43] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.53, 3.74, 3.71 [04:44:39] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.03, 2.34, 3.14 [04:48:02] RECOVERY - wiki.elysiamc.tk - reverse DNS on sslhost is OK: SSL OK - wiki.elysiamc.tk reverse DNS resolves to cp22.miraheze.org - CNAME OK [05:04:49] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=flexentertainmentwiki --no-updates --username-prefix=wikia:flex-entertainment /home/reception/ImportDump/flexentertainmentwiki-20221122042800.xml (START) [05:04:50] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=flexentertainmentwiki --no-updates --username-prefix=wikia:flex-entertainment /home/reception/ImportDump/flexentertainmentwiki-20221122042800.xml (END - exit=256) [05:04:51] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=flexentertainmentwiki (START) [05:04:52] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=flexentertainmentwiki (END - exit=0) [05:04:53] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=flexentertainmentwiki --active --update (END - exit=0) [05:04:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:04:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:05:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:05:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:05:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:05:17] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.99, 3.61, 2.86 [05:07:16] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.10, 3.06, 2.75 [05:08:32] !log reception@mwtask141:~$ sudo -u www-data php /srv/mediawiki/w/maintenance/dumpBackup.php --full --include-files --wiki zhangzhehanwiki > /home/reception/zhangzhehanwiki22112022.xml [05:08:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:12:50] !log reception@mwtask141:~$ sudo -u www-data php /srv/mediawiki/w/maintenance/dumpBackup.php --full --include-files --wiki linenwiki > /home/reception/linen22112022.xml [05:12:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:13:06] !log reception@mwtask141:~$ sudo -u www-data php /srv/mediawiki/w/maintenance/dumpBackup.php --full --include-files --wiki changelingwiki > /home/reception/changeling22112022.xml [05:13:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:24:26] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.61, 3.07, 2.18 [05:26:26] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.09, 2.65, 2.13 [05:29:55] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 6.39, 3.18, 1.66 [05:32:48] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.97, 3.61, 2.81 [05:36:45] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.16, 3.56, 2.92 [05:38:43] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.91, 3.86, 3.15 [05:39:21] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.67, 3.19, 2.49 [05:39:50] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 0.58, 3.07, 2.99 [05:40:41] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.78, 4.86, 3.64 [05:42:39] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.00, 3.42, 3.25 [05:43:21] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.74, 2.59, 2.41 [05:44:37] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 0.46, 2.46, 2.92 [05:47:19] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 3.47, 4.43, 3.31 [05:49:18] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.34, 3.29, 3.02 [05:55:17] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.36, 4.03, 3.38 [05:57:17] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.83, 3.08, 3.11 [05:59:29] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 5.44, 4.14, 2.98 [06:00:00] PROBLEM - cp32 SSH on cp32 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:00:35] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 108.175.15.182/cpweb, 2607:f1c0:1800:8100::1/cpweb [06:00:45] PROBLEM - cp32 PowerDNS Recursor on cp32 is CRITICAL: CRITICAL - Plugin timed out while executing system call [06:01:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 35.83, 20.48, 9.94 [06:01:28] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 2.71, 3.59, 2.93 [06:01:29] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 108.175.15.182/cpweb, 2607:f1c0:1800:8100::1/cpweb [06:01:33] PROBLEM - cp32 HTTPS on cp32 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:02:33] PROBLEM - cp32 Puppet on cp32 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [06:02:45] PROBLEM - cp32 Varnish Backends on cp32 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [06:03:23] PROBLEM - cp32 conntrack_table_size on cp32 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [06:03:28] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 0.64, 2.49, 2.60 [06:04:43] RECOVERY - cp32 Puppet on cp32 is OK: OK: Puppet is currently enabled, last run 24 minutes ago with 0 failures [06:04:44] RECOVERY - cp32 Varnish Backends on cp32 is OK: All 14 backends are healthy [06:04:52] RECOVERY - cp32 PowerDNS Recursor on cp32 is OK: DNS OK: 0.986 seconds response time. miraheze.org returns 2607:f1c0:1800:26f::1,74.208.203.152 [06:05:20] RECOVERY - cp32 conntrack_table_size on cp32 is OK: OK: nf_conntrack is 1 % full [06:05:29] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [06:05:38] RECOVERY - cp32 HTTPS on cp32 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 3424 bytes in 0.416 second response time [06:06:09] RECOVERY - cp32 SSH on cp32 is OK: SSH OK - OpenSSH_8.4p1 Debian-5+deb11u1 (protocol 2.0) [06:06:27] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [06:25:08] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.75, 3.59, 2.85 [06:25:50] PROBLEM - www.petrawiki.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:27:44] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 18.71, 11.83, 5.50 [06:29:20] PROBLEM - cp22 Current Load on cp22 is CRITICAL: CRITICAL - load average: 7.25, 6.93, 3.79 [06:31:13] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.35, 1.71, 3.93 [06:33:12] PROBLEM - cp22 Current Load on cp22 is WARNING: WARNING - load average: 0.40, 3.52, 3.10 [06:35:08] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 0.94, 2.77, 2.88 [06:37:08] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.63, 3.47, 3.91 [06:39:03] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 0.87, 2.71, 3.71 [06:39:33] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 0.58, 2.30, 3.59 [06:41:03] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 0.75, 2.02, 3.33 [06:41:31] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 0.45, 1.70, 3.22 [06:43:02] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.03, 3.01, 3.75 [06:46:58] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.86, 2.31, 3.27 [06:52:41] PROBLEM - gluster122 Current Load on gluster122 is WARNING: WARNING - load average: 3.60, 3.63, 2.64 [06:54:39] RECOVERY - gluster122 Current Load on gluster122 is OK: OK - load average: 2.83, 3.24, 2.61 [06:55:12] PROBLEM - www.petrawiki.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'en.petrawiki.org' expires in 10 day(s) (Fri 02 Dec 2022 14:48:33 GMT +0000). [06:58:36] PROBLEM - gluster122 Current Load on gluster122 is CRITICAL: CRITICAL - load average: 7.60, 6.26, 3.99 [07:01:28] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 6.44, 3.99, 2.97 [07:05:28] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 1.97, 3.61, 3.12 [07:06:38] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 8.40, 5.37, 3.73 [07:07:28] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 0.56, 2.52, 2.78 [07:12:20] PROBLEM - cp32 PowerDNS Recursor on cp32 is CRITICAL: CRITICAL - Plugin timed out while executing system call [07:14:16] RECOVERY - cp32 PowerDNS Recursor on cp32 is OK: DNS OK: 0.313 seconds response time. miraheze.org returns 108.175.15.182,2607:f1c0:1800:26f::1,2607:f1c0:1800:8100::1,74.208.203.152 [07:15:08] PROBLEM - cp22 Current Load on cp22 is WARNING: WARNING - load average: 3.51, 2.86, 1.87 [07:17:08] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 3.26, 3.24, 2.15 [07:29:28] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 7.68, 3.89, 2.54 [07:30:25] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.45, 1.96, 3.83 [07:31:28] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 2.82, 3.36, 2.51 [07:32:15] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 2.01, 3.47, 2.27 [07:34:15] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 0.86, 2.56, 2.08 [07:34:22] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 0.93, 1.37, 3.16 [07:36:35] PROBLEM - gluster122 Current Load on gluster122 is WARNING: WARNING - load average: 1.33, 1.25, 3.70 [07:40:22] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 8.67, 5.20, 4.16 [07:40:35] RECOVERY - gluster122 Current Load on gluster122 is OK: OK - load average: 1.63, 1.48, 3.22 [07:44:22] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.31, 3.15, 3.57 [07:45:08] PROBLEM - cp22 Current Load on cp22 is CRITICAL: CRITICAL - load average: 4.12, 3.00, 2.05 [07:45:14] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.78, 3.00, 2.41 [07:46:22] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 0.89, 2.36, 3.23 [07:47:08] PROBLEM - cp22 Current Load on cp22 is WARNING: WARNING - load average: 3.43, 3.06, 2.19 [07:47:15] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 2.26, 2.69, 2.37 [07:49:08] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 0.78, 2.25, 2.00 [08:00:25] PROBLEM - db121 Current Load on db121 is CRITICAL: CRITICAL - load average: 8.20, 10.82, 5.44 [08:01:34] PROBLEM - db121 MariaDB on db121 is CRITICAL: Can't connect to MySQL server on 'db121.miraheze.org' (115) [08:02:21] PROBLEM - db121 Current Load on db121 is WARNING: WARNING - load average: 1.29, 7.38, 4.81 [08:04:17] RECOVERY - db121 Current Load on db121 is OK: OK - load average: 0.19, 5.02, 4.25 [08:04:55] !log restart mysql on db121 [08:04:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [08:05:51] !log SET GLOBAL read_only=0; on db121 [08:05:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [08:07:34] RECOVERY - db121 MariaDB on db121 is OK: Uptime: 172 Threads: 13 Questions: 70990 Slow queries: 4472 Opens: 2657 Open tables: 2651 Queries per second avg: 412.732 [08:10:07] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.23, 3.59, 3.00 [08:16:01] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.17, 2.99, 3.10 [08:23:58] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.2667074203 secs [08:38:35] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.30, 4.46, 3.51 [08:40:31] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.4890996516 secs [08:46:28] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 0.81, 3.00, 3.37 [08:49:10] PROBLEM - mw132 Current Load on mw132 is CRITICAL: CRITICAL - load average: 12.75, 13.58, 8.21 [08:51:10] PROBLEM - mw132 Current Load on mw132 is WARNING: WARNING - load average: 5.25, 10.41, 7.69 [08:51:58] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.00957095623 secs [08:53:10] RECOVERY - mw132 Current Load on mw132 is OK: OK - load average: 4.89, 8.34, 7.25 [08:54:22] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.77, 2.84, 2.96 [08:56:22] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.92, 2.51, 2.83 [08:58:30] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 4.66, 2.89, 1.69 [08:58:31] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset -0.006181746721 secs [09:00:28] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 3.00, 3.06, 1.91 [09:02:19] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 10.09, 5.05, 3.10 [09:04:19] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.91, 3.56, 2.79 [09:06:18] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 0.55, 2.48, 2.48 [09:12:11] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.89, 5.78, 4.15 [09:16:07] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 0.58, 2.95, 3.36 [09:16:10] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 5.66, 3.45, 2.17 [09:18:08] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 1.35, 2.64, 2.03 [09:21:18] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.53, 3.79, 2.84 [09:31:53] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 4.02, 4.23, 2.93 [09:33:51] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 1.41, 3.31, 2.76 [09:38:43] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.35, 4.73, 3.51 [09:39:13] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 0.75, 2.99, 3.98 [09:40:41] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.33, 3.63, 3.25 [09:42:40] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.83, 4.26, 3.53 [09:43:12] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 0.57, 1.63, 3.20 [09:46:07] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.61, 3.25, 2.62 [09:48:06] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 2.46, 2.97, 2.59 [09:51:46] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.93, 3.48, 2.85 [09:52:31] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 0.53, 2.63, 3.62 [09:53:42] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 3.00, 3.19, 2.82 [09:55:08] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.81, 3.04, 2.86 [09:56:27] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 0.67, 1.86, 3.11 [09:57:07] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 0.97, 2.20, 2.57 [10:00:33] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.03, 3.45, 3.08 [10:04:26] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 2.02, 2.74, 2.88 [10:08:22] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.83, 4.13, 3.18 [10:20:22] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 0.31, 2.56, 3.58 [10:22:22] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 0.45, 1.97, 3.25 [10:36:23] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 9.09, 4.53, 3.19 [10:44:22] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.15, 3.52, 3.43 [10:46:22] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.98, 3.13, 3.31 [11:06:51] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1023629308 secs [11:14:51] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.09531882405 secs [11:24:36] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.96, 2.51, 1.54 [11:24:51] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1254346967 secs [11:26:35] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.43, 2.77, 1.76 [11:30:51] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.09409222007 secs [11:32:33] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.34, 3.06, 2.27 [11:37:10] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.55, 3.62, 2.55 [11:39:09] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.30, 3.08, 2.49 [11:58:48] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.73, 4.05, 2.88 [12:00:26] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.28, 3.48, 2.63 [12:02:25] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 0.91, 2.53, 2.38 [12:02:44] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.53, 3.79, 3.14 [12:04:42] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.44, 3.77, 3.18 [12:06:41] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.59, 3.81, 3.28 [12:10:37] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.52, 3.16, 3.13 [12:15:08] PROBLEM - cp22 Current Load on cp22 is CRITICAL: CRITICAL - load average: 5.11, 3.43, 2.11 [12:21:20] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.71, 4.59, 3.21 [12:22:52] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset -0.1068929136 secs [12:24:51] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset -0.0881151855 secs [12:29:19] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.95, 3.95, 3.48 [12:31:18] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.07, 4.11, 3.60 [12:35:08] PROBLEM - cp22 Current Load on cp22 is WARNING: WARNING - load average: 0.60, 3.02, 3.97 [12:39:08] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 1.04, 1.80, 3.23 [12:39:16] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 0.80, 2.99, 3.54 [12:41:15] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 0.96, 2.36, 3.25 [12:55:10] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.65, 2.98, 2.59 [12:59:09] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.92, 3.21, 2.81 [13:07:33] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.28, 3.50, 2.95 [13:07:58] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.126412034 secs [13:09:31] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.24, 2.81, 2.77 [13:14:01] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.08675634861 secs [13:23:16] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 2.93, 4.92, 3.65 [13:25:14] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.07, 3.60, 3.32 [13:27:12] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.11, 4.56, 3.73 [13:30:44] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 5.41, 3.91, 2.57 [13:31:09] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.92, 3.61, 3.56 [13:31:09] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-7 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/fbae977756cf [13:31:12] [02miraheze/puppet] 07paladox 03fbae977 - mariadb: fix logrotate config [13:31:14] [02puppet] 07paladox created branch 03paladox-patch-7 - 13https://github.com/miraheze/puppet [13:31:15] [url] Page not found · GitHub · GitHub | github.com [13:31:17] [02puppet] 07paladox opened pull request 03#3011: mariadb: fix logrotate config - 13https://github.com/miraheze/puppet/pull/3011 [13:31:18] [url] Page not found · GitHub · GitHub | github.com [13:31:48] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-7 [+1/-0/±0] 13https://github.com/miraheze/puppet/compare/fbae977756cf...be4594896a13 [13:31:49] [url] Comparing fbae977756cf...be4594896a13 · miraheze/puppet · GitHub | github.com [13:31:51] [02miraheze/puppet] 07paladox 03be45948 - Create mysql-server.logrotate.conf [13:31:52] [02puppet] 07paladox synchronize pull request 03#3011: mariadb: fix logrotate config - 13https://github.com/miraheze/puppet/pull/3011 [13:31:53] [url] Page not found · GitHub · GitHub | github.com [13:32:40] [02puppet] 07paladox closed pull request 03#3011: mariadb: fix logrotate config - 13https://github.com/miraheze/puppet/pull/3011 [13:32:40] [url] Page not found · GitHub · GitHub | github.com [13:32:41] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 1.33, 3.01, 2.41 [13:32:43] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/puppet/compare/03ca6e3c9d80...82b201e23e5b [13:32:43] [url] Comparing 03ca6e3c9d80...82b201e23e5b · miraheze/puppet · GitHub | github.com [13:32:44] [02miraheze/puppet] 07paladox 0382b201e - mariadb: fix logrotate config (#3011) [13:32:46] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-7 [13:32:49] [02puppet] 07paladox deleted branch 03paladox-patch-7 - 13https://github.com/miraheze/puppet [13:32:49] [url] Page not found · GitHub · GitHub | github.com [13:33:07] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.20, 2.79, 3.26 [13:38:35] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.53, 3.23, 2.79 [13:40:34] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 2.85, 3.00, 2.75 [13:49:18] PROBLEM - wiki.elysiamc.tk - LetsEncrypt on sslhost is CRITICAL: connect to address wiki.elysiamc.tk and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [13:50:49] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 13.68, 5.70, 3.77 [13:54:45] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.79, 3.78, 3.39 [13:54:50] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.75, 3.82, 2.68 [13:56:43] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 3.18, 3.36, 3.27 [13:56:49] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.18, 3.68, 2.78 [13:58:49] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.75, 4.54, 3.18 [14:02:47] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.03, 3.44, 3.06 [14:04:46] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.18, 2.61, 2.79 [14:19:03] RECOVERY - wiki.elysiamc.tk - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.elysiamc.tk' will expire on Sun 01 Jan 2023 16:18:43 GMT +0000. [14:38:59] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 7.32, 7.82, 4.77 [14:44:53] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 0.56, 2.97, 3.55 [14:46:51] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 0.40, 2.12, 3.16 [14:47:08] PROBLEM - cp22 Current Load on cp22 is CRITICAL: CRITICAL - load average: 4.06, 4.99, 3.03 [14:49:08] PROBLEM - cp22 Current Load on cp22 is WARNING: WARNING - load average: 1.54, 3.94, 2.90 [14:51:08] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 0.81, 2.84, 2.62 [14:55:28] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.46, 3.20, 2.37 [14:57:28] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.33, 3.10, 2.45 [15:00:37] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 2.77, 4.39, 3.60 [15:02:24] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 5.47, 4.74, 3.07 [15:02:35] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.09, 3.76, 3.47 [15:04:22] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 1.15, 3.33, 2.75 [15:04:33] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 8.90, 5.26, 4.01 [15:08:29] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.76, 3.46, 3.57 [15:12:26] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.09, 2.42, 3.14 [15:15:13] PROBLEM - cp22 Current Load on cp22 is CRITICAL: CRITICAL - load average: 9.50, 5.58, 3.04 [15:23:08] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 1.40, 3.03, 3.08 [15:37:59] PROBLEM - wiki.elysiamc.tk - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.elysiamc.tk could not be found [15:39:20] PROBLEM - mw141 Current Load on mw141 is CRITICAL: CRITICAL - load average: 13.65, 11.74, 7.21 [15:40:46] PROBLEM - mw142 Current Load on mw142 is CRITICAL: CRITICAL - load average: 9.43, 12.08, 8.00 [15:41:20] RECOVERY - mw141 Current Load on mw141 is OK: OK - load average: 5.98, 9.31, 6.86 [15:42:46] RECOVERY - mw142 Current Load on mw142 is OK: OK - load average: 4.99, 9.61, 7.59 [15:43:48] [02dns] 07MacFan4000 opened pull request 03#373: add acme challenges for m.miraheze.org - 13https://github.com/miraheze/dns/pull/373 [15:43:48] [url] Page not found · GitHub · GitHub | github.com [15:44:11] Reception123: could you please merge ^ [15:44:50] [02dns] 07Reception123 closed pull request 03#373: add acme challenges for m.miraheze.org - 13https://github.com/miraheze/dns/pull/373 [15:44:50] ... [15:44:51] [02miraheze/dns] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/dns/compare/e49fecae7fea...73ba1f64a854 [15:44:51] [url] Comparing e49fecae7fea...73ba1f64a854 · miraheze/dns · GitHub | github.com [15:44:53] [02miraheze/dns] 07MacFan4000 0373ba1f6 - add acme challenges for m.miraheze.org (#373) [15:44:56] ty [15:45:10] PROBLEM - cp22 Current Load on cp22 is CRITICAL: CRITICAL - load average: 10.60, 5.06, 3.06 [15:45:14] no problem [15:46:51] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1179256439 secs [15:48:51] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.09395855665 secs [15:51:08] PROBLEM - cp22 Current Load on cp22 is WARNING: WARNING - load average: 1.60, 3.59, 3.17 [15:53:08] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 2.45, 3.32, 3.13 [15:56:10] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.76, 3.63, 2.77 [15:58:09] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.40, 2.86, 2.60 [15:58:15] [02miraheze/ssl] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/e36f697273b3...a5a344402936 [15:58:15] [url] Comparing e36f697273b3...a5a344402936 · miraheze/ssl · GitHub | github.com [15:58:16] [02miraheze/ssl] 07MacFan4000 03a5a3444 - Update m.miraheze.org.crt [15:59:28] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 5.42, 3.40, 2.13 [16:03:28] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 1.75, 3.67, 2.65 [16:03:46] RECOVERY - m.miraheze.org - LetsEncrypt on sslhost is OK: OK - Certificate 'm.miraheze.org' will expire on Mon 20 Feb 2023 14:57:21 GMT +0000. [16:05:28] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 0.76, 2.69, 2.42 [16:06:36] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.95, 3.75, 2.56 [16:06:52] RECOVERY - wiki.elysiamc.tk - reverse DNS on sslhost is OK: SSL OK - wiki.elysiamc.tk reverse DNS resolves to cp23.miraheze.org - CNAME OK [16:11:46] PROBLEM - m.miraheze.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'm.miraheze.org' expired on Tue 22 Nov 2022 14:02:45 GMT +0000. [16:13:46] RECOVERY - m.miraheze.org - LetsEncrypt on sslhost is OK: OK - Certificate 'm.miraheze.org' will expire on Mon 20 Feb 2023 14:57:21 GMT +0000. [16:15:11] PROBLEM - cp22 Current Load on cp22 is CRITICAL: CRITICAL - load average: 6.95, 4.10, 2.87 [16:16:26] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.11, 3.90, 3.75 [16:17:08] PROBLEM - cp22 Current Load on cp22 is WARNING: WARNING - load average: 2.36, 3.41, 2.77 [16:19:08] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 0.75, 2.42, 2.48 [16:20:51] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset -0.1143043041 secs [16:22:22] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 0.80, 2.23, 3.06 [16:24:51] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset -0.08612385392 secs [16:26:00] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.85, 3.48, 2.49 [16:27:59] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.33, 2.73, 2.33 [16:28:22] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 7.87, 4.62, 3.68 [16:29:02] [02miraheze/ErrorPages] 07Universal-Omega pushed 031 commit to 03remove-unused-css [+0/-0/±1] 13https://github.com/miraheze/ErrorPages/commit/16556c34b3c6 [16:29:03] [02miraheze/ErrorPages] 07Universal-Omega 0316556c3 - Remove unused CSS from error pages [16:29:06] [02ErrorPages] 07Universal-Omega created branch 03remove-unused-css - 13https://github.com/miraheze/ErrorPages [16:29:06] [url] Page not found · GitHub · GitHub | github.com [16:29:09] [02ErrorPages] 07Universal-Omega opened pull request 03#18: Remove unused CSS from error pages - 13https://github.com/miraheze/ErrorPages/pull/18 [16:29:09] [url] Page not found · GitHub · GitHub | github.com [16:30:00] miraheze/ErrorPages - Universal-Omega the build passed. [16:30:22] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.60, 3.91, 3.55 [16:32:02] [02miraheze/ErrorPages] 07Universal-Omega pushed 031 commit to 03remove-unused-css [+0/-0/±1] 13https://github.com/miraheze/ErrorPages/compare/16556c34b3c6...5a3ec2f0d1fa [16:32:03] [url] Comparing 16556c34b3c6...5a3ec2f0d1fa · miraheze/ErrorPages · GitHub | github.com [16:32:05] [02miraheze/ErrorPages] 07Universal-Omega 035a3ec2f - Update 502.html [16:32:06] [02ErrorPages] 07Universal-Omega synchronize pull request 03#18: Remove unused CSS from error pages - 13https://github.com/miraheze/ErrorPages/pull/18 [16:32:06] [url] Page not found · GitHub · GitHub | github.com [16:32:22] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.11, 2.89, 3.21 [16:33:02] miraheze/ErrorPages - Universal-Omega the build passed. [16:33:38] [02miraheze/ErrorPages] 07Universal-Omega pushed 031 commit to 03remove-unused-css [+0/-0/±1] 13https://github.com/miraheze/ErrorPages/compare/5a3ec2f0d1fa...731607837e3c [16:33:39] [url] Comparing 5a3ec2f0d1fa...731607837e3c · miraheze/ErrorPages · GitHub | github.com [16:33:40] [02miraheze/ErrorPages] 07Universal-Omega 037316078 - Update 404.php [16:33:42] [02ErrorPages] 07Universal-Omega synchronize pull request 03#18: Remove unused CSS from error pages - 13https://github.com/miraheze/ErrorPages/pull/18 [16:33:43] [url] Page not found · GitHub · GitHub | github.com [16:34:34] miraheze/ErrorPages - Universal-Omega the build passed. [16:36:00] [02miraheze/ErrorPages] 07Universal-Omega pushed 031 commit to 03remove-unused-css [+0/-0/±1] 13https://github.com/miraheze/ErrorPages/compare/731607837e3c...331955b5a5ce [16:36:01] [url] Comparing 731607837e3c...331955b5a5ce · miraheze/ErrorPages · GitHub | github.com [16:36:03] [02miraheze/ErrorPages] 07Universal-Omega 03331955b - Update 504.php [16:36:05] [02ErrorPages] 07Universal-Omega synchronize pull request 03#18: Remove unused CSS from error pages - 13https://github.com/miraheze/ErrorPages/pull/18 [16:36:05] [url] Page not found · GitHub · GitHub | github.com [16:36:58] miraheze/ErrorPages - Universal-Omega the build passed. [16:38:23] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.33, 4.93, 3.94 [16:38:56] [02miraheze/ErrorPages] 07Universal-Omega pushed 031 commit to 03remove-unused-css [+0/-0/±1] 13https://github.com/miraheze/ErrorPages/compare/331955b5a5ce...6cf6813d4909 [16:38:57] [url] Comparing 331955b5a5ce...6cf6813d4909 · miraheze/ErrorPages · GitHub | github.com [16:38:58] [02miraheze/ErrorPages] 07Universal-Omega 036cf6813 - Update MissingWiki.php [16:39:00] [02ErrorPages] 07Universal-Omega synchronize pull request 03#18: Remove unused CSS from error pages - 13https://github.com/miraheze/ErrorPages/pull/18 [16:39:00] [url] Page not found · GitHub · GitHub | github.com [16:39:45] miraheze/ErrorPages - Universal-Omega the build passed. [16:40:34] [02miraheze/ErrorPages] 07Universal-Omega pushed 031 commit to 03remove-unused-css [+0/-0/±1] 13https://github.com/miraheze/ErrorPages/compare/6cf6813d4909...f31d2fecf57e [16:40:35] [url] Comparing 6cf6813d4909...f31d2fecf57e · miraheze/ErrorPages · GitHub | github.com [16:40:36] [02miraheze/ErrorPages] 07Universal-Omega 03f31d2fe - Update db141Wiki.php [16:40:37] [02ErrorPages] 07Universal-Omega synchronize pull request 03#18: Remove unused CSS from error pages - 13https://github.com/miraheze/ErrorPages/pull/18 [16:40:38] [url] Page not found · GitHub · GitHub | github.com [16:41:31] miraheze/ErrorPages - Universal-Omega the build passed. [16:41:49] [02miraheze/ssl] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/a5a344402936...7da716289361 [16:41:50] [url] Comparing a5a344402936...7da716289361 · miraheze/ssl · GitHub | github.com [16:41:52] [02miraheze/ssl] 07MacFan4000 037da7162 - remove redirect that used an invalid cert [16:45:13] [02miraheze/ssl] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/7da716289361...2b93e7faa8ee [16:45:14] [url] Comparing 7da716289361...2b93e7faa8ee · miraheze/ssl · GitHub | github.com [16:45:15] [02miraheze/ssl] 07MacFan4000 032b93e7f - Update en.petrawiki.org.crt [16:48:22] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.34, 3.32, 3.84 [16:50:00] PROBLEM - cp22 Puppet on cp22 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [16:52:22] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 13.41, 5.34, 4.32 [16:59:05] PROBLEM - db112 Current Load on db112 is CRITICAL: CRITICAL - load average: 7.47, 8.88, 6.05 [16:59:35] [02ErrorPages] 07Universal-Omega synchronize pull request 03#18: Remove unused CSS from error pages - 13https://github.com/miraheze/ErrorPages/pull/18 [16:59:35] [url] Page not found · GitHub · GitHub | github.com [16:59:38] [02miraheze/ErrorPages] 07Universal-Omega pushed 031 commit to 03remove-unused-css [+0/-0/±1] 13https://github.com/miraheze/ErrorPages/compare/f31d2fecf57e...9b31330eaecb [16:59:38] [url] Comparing f31d2fecf57e...9b31330eaecb · miraheze/ErrorPages · GitHub | github.com [16:59:39] [02miraheze/ErrorPages] 07Universal-Omega 039b31330 - Update 502.html [17:00:30] miraheze/ErrorPages - Universal-Omega the build passed. [17:06:53] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/e004a00f69d6 [17:06:55] [02miraheze/puppet] 07Universal-Omega 03e004a00 - varnish: remove unused CSS/JS from 503 error page [17:06:57] [02puppet] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/puppet [17:06:57] [url] Page not found · GitHub · GitHub | github.com [17:07:00] [02puppet] 07Universal-Omega opened pull request 03#3012: varnish: remove unused CSS/JS from 503 error page - 13https://github.com/miraheze/puppet/pull/3012 [17:07:00] [url] Page not found · GitHub · GitHub | github.com [17:07:52] RECOVERY - petrawiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'en.petrawiki.org' will expire on Mon 20 Feb 2023 15:44:41 GMT +0000. [17:08:57] RECOVERY - db112 Current Load on db112 is OK: OK - load average: 3.46, 6.78, 6.63 [17:09:02] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.46, 3.14, 3.98 [17:11:02] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.75, 3.71, 4.06 [17:13:01] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.28, 3.21, 3.84 [17:16:40] RECOVERY - en.petrawiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'en.petrawiki.org' will expire on Mon 20 Feb 2023 15:44:41 GMT +0000. [17:17:01] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.27, 2.26, 3.31 [17:17:29] RECOVERY - cp22 Puppet on cp22 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [17:23:50] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.78, 3.32, 2.68 [17:25:39] probably should have posted here but just left a message in #miraheze, odd because it's db101 not db141? [17:25:47] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 2.81, 2.95, 2.61 [17:27:02] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.69, 3.66, 3.35 [17:27:02] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1834937036 secs [17:27:42] PROBLEM - db112 Current Load on db112 is CRITICAL: CRITICAL - load average: 8.43, 7.22, 6.34 [17:29:02] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.89, 3.52, 3.36 [17:30:46] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 6.45, 4.69, 2.77 [17:32:45] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 2.06, 3.76, 2.67 [17:33:02] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.06489819288 secs [17:33:02] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.08, 3.94, 3.59 [17:34:44] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 1.55, 2.94, 2.49 [17:38:08] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 5.02, 3.78, 3.12 [17:39:30] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.01, 3.53, 3.07 [17:40:03] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.79, 3.75, 3.19 [17:40:07] BrandonWM: replied in #miraheze [17:41:28] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 2.04, 2.99, 2.93 [17:41:57] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 2.71, 3.38, 3.12 [17:43:01] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.45, 3.79, 3.99 [17:47:02] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 0.87, 2.15, 3.26 [17:47:19] PROBLEM - wiki.sanctuary-rts.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.sanctuary-rts.com' expires in 7 day(s) (Wed 30 Nov 2022 17:34:07 GMT +0000). [17:51:02] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.43, 4.00, 3.74 [17:51:58] gotcha [17:53:02] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.60, 3.03, 3.41 [17:53:04] PROBLEM - db112 Current Load on db112 is WARNING: WARNING - load average: 4.78, 6.70, 7.84 [17:54:37] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.06, 3.42, 2.88 [17:55:02] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 3.03, 2.88, 3.30 [17:56:32] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.86, 3.27, 2.89 [17:56:59] RECOVERY - db112 Current Load on db112 is OK: OK - load average: 0.98, 3.98, 6.52 [17:59:02] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.67, 4.70, 3.91 [18:03:02] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.91, 3.87, 3.81 [18:03:27] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 3.26, 3.54, 2.70 [18:05:02] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 10.02, 6.04, 4.59 [18:05:26] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 0.78, 2.48, 2.41 [18:11:01] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.09, 3.04, 3.76 [18:13:01] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 0.57, 2.21, 3.37 [18:33:57] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.31, 3.80, 3.17 [18:35:52] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.10, 3.59, 3.17 [18:37:46] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 2.07, 3.04, 3.01 [18:38:13] PROBLEM - cp32 HTTPS on cp32 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:38:42] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 25.90, 13.53, 6.60 [18:39:08] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 108.175.15.182/cpweb, 2607:f1c0:1800:8100::1/cpweb [18:39:15] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 108.175.15.182/cpweb, 2607:f1c0:1800:8100::1/cpweb [18:39:47] PROBLEM - cp32 SSH on cp32 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:39:50] PROBLEM - cp32 PowerDNS Recursor on cp32 is CRITICAL: CRITICAL - Plugin timed out while executing system call [18:40:38] PROBLEM - cp32 Puppet on cp32 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [18:41:08] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [18:41:15] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [18:41:44] RECOVERY - cp32 SSH on cp32 is OK: SSH OK - OpenSSH_8.4p1 Debian-5+deb11u1 (protocol 2.0) [18:41:51] RECOVERY - cp32 PowerDNS Recursor on cp32 is OK: DNS OK: 0.852 seconds response time. miraheze.org returns 108.175.15.182,2607:f1c0:1800:26f::1,2607:f1c0:1800:8100::1,74.208.203.152 [18:42:11] RECOVERY - cp32 HTTPS on cp32 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 3425 bytes in 0.417 second response time [18:42:34] RECOVERY - cp32 Puppet on cp32 is OK: OK: Puppet is currently enabled, last run 31 minutes ago with 0 failures [18:45:19] PROBLEM - cp22 Current Load on cp22 is CRITICAL: CRITICAL - load average: 7.43, 4.74, 2.70 [18:49:07] PROBLEM - cp22 Current Load on cp22 is WARNING: WARNING - load average: 2.97, 3.88, 2.82 [18:49:37] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.69, 4.09, 3.11 [18:51:01] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 1.04, 2.85, 2.56 [18:51:37] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.41, 3.52, 3.02 [18:53:37] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.36, 3.76, 3.16 [18:59:37] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.36, 3.24, 3.25 [19:03:38] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 8.75, 5.30, 4.04 [19:03:44] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.22, 2.13, 3.72 [19:07:37] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.32, 3.87, 3.79 [19:13:22] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.27, 2.01, 3.14 [19:13:37] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.06, 2.33, 3.18 [19:23:06] [02ErrorPages] 07Universal-Omega closed pull request 03#18: Remove unused CSS from error pages - 13https://github.com/miraheze/ErrorPages/pull/18 [19:23:07] [url] Page not found · GitHub · GitHub | github.com [19:23:07] [02miraheze/ErrorPages] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±5] 13https://github.com/miraheze/ErrorPages/compare/04beab46df99...8cd2270b6778 [19:23:08] [url] Comparing 04beab46df99...8cd2270b6778 · miraheze/ErrorPages · GitHub | github.com [19:23:08] [02miraheze/ErrorPages] 07Universal-Omega 038cd2270 - Remove unused CSS from error pages (#18) [19:23:11] [02miraheze/ErrorPages] 07Universal-Omega deleted branch 03remove-unused-css [19:23:12] [02ErrorPages] 07Universal-Omega deleted branch 03remove-unused-css - 13https://github.com/miraheze/ErrorPages [19:23:13] [url] Page not found · GitHub · GitHub | github.com [19:23:22] [02puppet] 07Universal-Omega closed pull request 03#3012: varnish: remove unused CSS/JS from 503 error page - 13https://github.com/miraheze/puppet/pull/3012 [19:23:23] ... [19:23:25] [02miraheze/puppet] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 [19:23:26] [02puppet] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/puppet [19:23:27] ... [19:23:29] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/82b201e23e5b...4d95697c89a0 [19:23:30] [url] Comparing 82b201e23e5b...4d95697c89a0 · miraheze/puppet · GitHub | github.com [19:23:32] [02miraheze/puppet] 07Universal-Omega 034d95697 - varnish: remove unused CSS/JS from 503 error page (#3012) [19:24:04] miraheze/ErrorPages - Universal-Omega the build passed. [19:25:37] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.44, 2.96, 2.92 [19:27:02] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.43, 3.94, 3.30 [19:27:37] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.12, 3.31, 3.04 [19:29:02] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.97, 3.17, 3.10 [19:29:37] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.45, 3.10, 3.01 [19:36:38] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 5.45, 3.88, 3.05 [19:38:33] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.63, 3.64, 3.05 [19:39:55] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/54294d555178 [19:39:58] [02miraheze/mw-config] 07Universal-Omega 0354294d5 - Pool db141 back in as c6 [19:40:00] [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-2 - 13https://github.com/miraheze/mw-config [19:40:00] [url] Page not found · GitHub · GitHub | github.com [19:40:01] [02mw-config] 07Universal-Omega opened pull request 03#4988: Pool db141 back in as c6 - 13https://github.com/miraheze/mw-config/pull/4988 [19:40:01] [url] Page not found · GitHub · GitHub | github.com [19:40:27] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 2.95, 3.27, 2.98 [19:40:48] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/54294d555178...e571f2385d15 [19:40:49] [url] Comparing 54294d555178...e571f2385d15 · miraheze/mw-config · GitHub | github.com [19:40:49] [02miraheze/mw-config] 07Universal-Omega 03e571f23 - Update LocalSettings.php [19:40:50] [02mw-config] 07Universal-Omega synchronize pull request 03#4988: Pool db141 back in as c6 - 13https://github.com/miraheze/mw-config/pull/4988 [19:40:50] [url] Page not found · GitHub · GitHub | github.com [19:41:05] miraheze/mw-config - Universal-Omega the build passed. [19:41:40] [02mw-config] 07Universal-Omega commented on pull request 03#4988: Pool db141 back in as c6 - 13https://github.com/miraheze/mw-config/pull/4988#issuecomment-1324156806 [19:41:40] [url] Pool db141 back in as c6 by Universal-Omega · Pull Request #4988 · miraheze/mw-config · GitHub | github.com [19:41:52] miraheze/mw-config - Universal-Omega the build passed. [19:46:48] PROBLEM - cp22 Current Load on cp22 is CRITICAL: CRITICAL - load average: 7.13, 4.47, 2.57 [19:49:00] [02mw-config] 07Universal-Omega synchronize pull request 03#4988: Pool db141 back in as c6 - 13https://github.com/miraheze/mw-config/pull/4988 [19:49:00] [url] Page not found · GitHub · GitHub | github.com [19:49:02] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/e571f2385d15...08772652202c [19:49:03] [url] Comparing e571f2385d15...08772652202c · miraheze/mw-config · GitHub | github.com [19:49:04] [02miraheze/mw-config] 07Universal-Omega 030877265 - Update Database.php [19:50:12] miraheze/mw-config - Universal-Omega the build passed. [19:50:48] PROBLEM - cp22 Current Load on cp22 is WARNING: WARNING - load average: 2.15, 3.84, 2.79 [19:51:50] [02mw-config] 07Universal-Omega edited pull request 03#4988: Pool db141 back in as c6/db142 - 13https://github.com/miraheze/mw-config/pull/4988 [19:51:50] [url] Page not found · GitHub · GitHub | github.com [19:52:48] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 0.78, 2.77, 2.52 [19:54:43] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.82, 3.60, 3.01 [19:56:27] PROBLEM - cp23 APT on cp23 is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. [19:56:38] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.76, 3.91, 3.18 [19:58:26] PROBLEM - cp23 APT on cp23 is CRITICAL: APT CRITICAL: 35 packages available for upgrade (26 critical updates). [19:58:33] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.41, 3.29, 3.04 [20:01:44] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.77, 3.77, 3.38 [20:05:28] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.86, 4.60, 3.43 [20:05:40] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 2.53, 3.23, 3.25 [20:09:19] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.50, 3.81, 3.36 [20:09:24] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.46, 4.36, 3.57 [20:11:15] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.21, 2.96, 3.10 [20:13:25] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 2.93, 3.68, 3.48 [20:17:24] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 2.35, 3.10, 3.29 [20:19:33] PROBLEM - atnarsia.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'atnarsia.com' expires in 15 day(s) (Thu 08 Dec 2022 20:09:51 GMT +0000). [20:25:37] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.21, 2.81, 2.34 [20:26:06] PROBLEM - gametheorywiki.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'gametheorywiki.org' expires in 15 day(s) (Thu 08 Dec 2022 20:03:38 GMT +0000). [20:27:37] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.05, 2.56, 2.31 [20:32:17] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 4.82, 3.40, 2.23 [20:34:16] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 0.99, 2.39, 1.99 [20:46:48] PROBLEM - cp22 Current Load on cp22 is CRITICAL: CRITICAL - load average: 7.10, 4.25, 2.62 [20:50:15] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.75, 3.42, 2.85 [20:50:48] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 1.04, 2.95, 2.52 [20:53:26] PROBLEM - wiki.virtualjet.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.virtualjet.net' expires in 15 day(s) (Thu 08 Dec 2022 20:33:54 GMT +0000). [20:54:06] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.85, 4.12, 3.24 [20:57:03] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 3.21, 4.08, 3.13 [20:57:56] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.71, 3.45, 3.22 [20:59:02] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.10, 3.31, 2.96 [20:59:51] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 0.55, 2.51, 2.90 [21:02:59] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 5.35, 3.91, 2.57 [21:04:58] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 2.45, 3.41, 2.55 [21:05:27] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-7 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/4e7b654d6086 [21:05:30] [02miraheze/puppet] 07paladox 034e7b654 - mariadb: stop using thread pool [21:05:31] [02puppet] 07paladox opened pull request 03#3013: mariadb: stop using thread pool - 13https://github.com/miraheze/puppet/pull/3013 [21:05:32] [url] Page not found · GitHub · GitHub | github.com [21:05:33] [02puppet] 07paladox created branch 03paladox-patch-7 - 13https://github.com/miraheze/puppet [21:05:33] ... [21:06:31] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 26.54, 27.73, 13.60 [21:06:57] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 1.69, 2.96, 2.50 [21:08:25] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.12, 3.25, 2.81 [21:14:22] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 4.00, 3.68, 3.17 [21:17:57] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.50, 3.99, 3.48 [21:18:22] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 1.81, 3.01, 3.04 [21:19:52] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 2.41, 3.38, 3.31 [21:23:41] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.12, 3.65, 3.47 [21:27:16] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 6.14, 4.32, 3.54 [21:27:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 5.85, 4.05, 3.27 [21:27:30] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 6.69, 4.68, 3.89 [21:29:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 2.88, 3.58, 3.20 [21:30:47] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 13.41, 6.53, 3.81 [21:31:11] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 2.69, 3.60, 3.44 [21:31:25] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 2.74, 3.66, 3.66 [21:31:27] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 2.87, 3.23, 3.10 [21:35:07] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.48, 4.08, 3.65 [21:35:25] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.35, 3.98, 3.78 [21:37:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.70, 3.74, 3.38 [21:39:03] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 2.96, 3.70, 3.61 [21:39:24] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.01, 3.60, 3.68 [21:39:27] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 2.13, 3.21, 3.23 [21:42:43] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 0.66, 2.98, 3.78 [21:43:25] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 5.25, 4.20, 3.87 [21:44:42] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 0.47, 2.11, 3.36 [21:45:19] PROBLEM - cp22 Current Load on cp22 is CRITICAL: CRITICAL - load average: 7.05, 3.33, 2.08 [21:46:49] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.1929442883 secs [21:46:54] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 5.08, 4.19, 3.77 [21:49:24] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.38, 3.99, 3.94 [21:50:50] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 2.86, 3.64, 3.65 [21:51:01] PROBLEM - cp22 Current Load on cp22 is WARNING: WARNING - load average: 1.48, 3.41, 2.71 [21:51:25] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 3.61, 4.07, 3.99 [21:52:55] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 0.52, 2.47, 2.45 [21:53:24] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 2.05, 3.38, 3.75 [21:56:43] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 2.67, 2.83, 3.27 [21:57:02] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.42, 1.95, 3.83 [21:59:24] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 3.07, 2.92, 3.39 [22:04:49] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.09774526954 secs [22:05:24] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.55, 3.70, 3.55 [22:05:27] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 5.52, 3.94, 3.16 [22:07:01] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.91, 2.45, 3.26 [22:07:24] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.65, 3.56, 3.51 [22:07:27] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 2.49, 3.38, 3.06 [22:13:25] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 2.46, 2.99, 3.29 [22:16:48] PROBLEM - cp22 Current Load on cp22 is CRITICAL: CRITICAL - load average: 5.33, 3.73, 2.35 [22:17:24] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 5.13, 4.65, 3.91 [22:18:02] PROBLEM - gluster122 Current Load on gluster122 is CRITICAL: CRITICAL - load average: 4.30, 3.14, 2.70 [22:18:48] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 1.81, 3.05, 2.27 [22:19:59] RECOVERY - gluster122 Current Load on gluster122 is OK: OK - load average: 1.57, 2.56, 2.54 [22:21:24] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 2.60, 3.80, 3.75 [22:25:24] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.02, 3.40, 3.56 [22:25:51] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.21, 3.90, 3.25 [22:27:25] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 2.56, 3.04, 3.41 [22:27:46] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.95, 3.58, 3.21 [22:29:25] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.51, 3.33, 3.45 [22:29:41] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.45, 3.00, 3.05 [22:31:25] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 3.01, 3.00, 3.31 [22:36:13] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.10, 3.57, 2.96 [22:38:10] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 2.11, 3.07, 2.85 [22:44:31] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=raamasamgebhyahwiki --no-updates --username-prefix=mh:kranditri /home/universalomega/ImportDump/raamasamgebhyahwiki-20221122213229.xml (START) [22:44:32] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=raamasamgebhyahwiki --no-updates --username-prefix=mh:kranditri /home/universalomega/ImportDump/raamasamgebhyahwiki-20221122213229.xml (END - exit=2) [22:44:33] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=raamasamgebhyahwiki (START) [22:44:34] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=raamasamgebhyahwiki (END - exit=2) [22:44:35] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=raamasamgebhyahwiki --active --update (END - exit=2) [22:44:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:44:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:44:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:45:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:45:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:52:04] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 3.08, 3.43, 2.49 [22:54:03] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 2.52, 2.94, 2.41 [23:18:49] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 3.63, 2.70, 1.98 [23:20:14] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.80, 3.55, 3.00 [23:20:48] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 0.98, 2.09, 1.86 [23:22:09] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 2.04, 3.00, 2.86 [23:25:54] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 15.20, 6.17, 3.68 [23:29:45] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.99, 3.88, 3.26 [23:31:40] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.38, 3.02, 3.02 [23:46:48] PROBLEM - cp22 Current Load on cp22 is WARNING: WARNING - load average: 3.98, 3.67, 2.39 [23:48:48] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 0.74, 2.56, 2.14 [23:49:31] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 4.90, 3.51, 2.33 [23:49:50] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.95, 3.86, 3.30 [23:51:30] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 2.79, 3.16, 2.35 [23:51:44] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 2.65, 3.41, 3.21 [23:53:39] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 2.77, 3.13, 3.12 [23:54:24] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.95, 3.28, 2.69