[00:00:00] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.44, 5.18, 4.34 [00:00:04] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.11, 7.23, 7.00 [00:01:20] !log [agent@test131] DEPLOY ABORTED: Non-Zero Exit Code in prep, see output. [00:01:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:01:36] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 14.87, 11.78, 11.78 [00:03:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.11, 11.88, 11.85 [00:05:36] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.05, 11.65, 11.75 [00:07:05] !log [agent@mwtask141] DEPLOY ABORTED: Non-Zero Exit Code in prep, see output. [00:07:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:07:42] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.54, 7.23, 7.37 [00:11:13] [02miraheze/MirahezeMagic] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/7a6a2a4a2b35...0cf56afdef4b [00:11:15] [02miraheze/MirahezeMagic] 07AgentIsai 030cf56af - Remove trailing dot [00:11:38] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1305420101 secs [00:12:10] [02miraheze/MirahezeMagic] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/0cf56afdef4b...89b91fe64ba2 [00:12:11] [02miraheze/MirahezeMagic] 07AgentIsai 0389b91fe - Fix JSON [00:15:25] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.22, 5.77, 6.66 [00:15:30] [02miraheze/mediawiki] 07AgentIsai pushed 031 commit to 03REL1_40 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/2ee9e3548763...0ae3be336821 [00:15:33] [02miraheze/mediawiki] 07AgentIsai 030ae3be3 - Update MirahezeMagic [00:17:15] !log [agent@test131] starting deploy of {'world': True, 'l10n': True} to all [00:17:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:20:32] [02miraheze/mediawiki] 07AgentIsai pushed 031 commit to 03REL1_39 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/d544fcf00a06...d65a4cc901c3 [00:20:34] [02miraheze/mediawiki] 07AgentIsai 03d65a4cc - Update MirahezeMagic [00:20:53] !log [agent@mwtask141] starting deploy of {'world': True, 'l10n': True} to all [00:20:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:21:57] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.48, 3.18, 3.99 [00:22:56] miraheze/MirahezeMagic - AgentIsai the build passed. [00:23:50] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.22, 3.74, 3.36 [00:25:47] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.84, 3.20, 3.22 [00:27:57] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.06, 3.49, 3.78 [00:29:38] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.07336428761 secs [00:30:04] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 12.58, 7.94, 6.91 [00:31:57] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.08, 2.94, 3.49 [00:31:58] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.34, 6.91, 6.64 [00:33:53] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.02, 6.35, 6.47 [00:36:01] !log [agent@test131] finished deploy of {'world': True, 'l10n': True} to all - SUCCESS in 1126s [00:36:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:37:57] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.10, 4.11, 3.79 [00:39:57] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.11, 3.75, 3.70 [00:41:32] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 10.97, 7.92, 7.06 [00:42:10] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.69, 3.73, 3.08 [00:43:57] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.37, 3.02, 3.40 [00:44:10] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.57, 3.39, 3.04 [00:45:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.77, 7.68, 7.16 [00:53:25] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.91, 6.10, 6.56 [01:03:34] [02miraheze/MirahezeMagic] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/89b91fe64ba2...7f02b6015c36 [01:03:35] [02miraheze/MirahezeMagic] 07AgentIsai 037f02b60 - Add log type [01:04:31] [02miraheze/mediawiki] 07AgentIsai pushed 031 commit to 03REL1_39 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/d65a4cc901c3...907f2f4a08e8 [01:04:33] [02miraheze/mediawiki] 07AgentIsai 03907f2f4 - Update MirahezeMagic [01:07:55] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 12.82, 8.71, 6.28 [01:08:28] [02miraheze/mediawiki] 07AgentIsai pushed 031 commit to 03REL1_40 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/0ae3be336821...ff7417da694a [01:08:31] [02miraheze/mediawiki] 07AgentIsai 03ff7417d - Update MirahezeMagic [01:08:55] !log [agent@test131] starting deploy of {'folders': 'w/extensions/MirahezeMagic'} to all [01:08:56] !log [agent@test131] finished deploy of {'folders': 'w/extensions/MirahezeMagic'} to all - SUCCESS in 0s [01:09:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:09:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:11:10] miraheze/MirahezeMagic - AgentIsai the build passed. [01:14:38] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.66, 4.65, 3.59 [01:15:35] !log [agent@test131] starting deploy of {'world': True} to all [01:15:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:16:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.35, 7.01, 6.55 [01:18:32] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.72, 3.96, 3.56 [01:18:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.24, 7.04, 6.60 [01:19:12] !log [agent@mwtask141] finished deploy of {'world': True, 'l10n': True} to all - SUCCESS in 3498s [01:19:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:19:45] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 3.83, 7.20, 7.36 [01:20:27] !log [agent@test131] finished deploy of {'world': True} to all - SUCCESS in 291s [01:20:28] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.26, 3.09, 3.29 [01:20:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:20:40] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.25, 6.36, 6.40 [01:23:42] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 5.18, 5.42, 6.55 [01:24:13] !log [agent@mwtask141] starting deploy of {'world': True} to all [01:24:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:25:13] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 7.07, 5.42, 3.97 [01:27:37] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 11.37, 8.31, 7.45 [01:31:33] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 4.61, 7.31, 7.32 [01:31:55] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [01:33:31] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 12.34, 9.90, 8.30 [01:33:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.85, 10.86, 11.88 [01:35:30] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 5.40, 7.77, 7.69 [01:35:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 15.51, 12.82, 12.50 [01:37:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.64, 10.92, 11.85 [01:39:26] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 8.42, 7.88, 7.73 [01:41:25] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 4.86, 6.91, 7.41 [01:42:17] !log [agent@mwtask141] finished deploy of {'world': True} to all - SUCCESS in 1084s [01:42:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:45:21] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 8.52, 7.21, 7.35 [01:46:37] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.47, 3.17, 3.86 [01:47:19] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 7.37, 6.95, 7.22 [01:47:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.33, 11.76, 11.75 [01:49:18] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 2.87, 5.27, 6.57 [01:51:03] !log [agent@test131] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/beta.json /srv/mediawiki/w/maintenance/sql.php /srv/mediawiki-staging/w/extensions/CheckUser/schema/mysql/patch-cu_changes-add-cuc_only_for_read_old.sql (START) [01:51:08] !log [agent@test131] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/beta.json /srv/mediawiki/w/maintenance/sql.php /srv/mediawiki-staging/w/extensions/CheckUser/schema/mysql/patch-cu_changes-add-cuc_only_for_read_old.sql (END - exit=0) [01:51:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:51:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:51:29] !log [agent@test131] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/beta.json /srv/mediawiki/w/maintenance/sql.php /srv/mediawiki-staging/1.40/extensions/CheckUser/schema/mysql/patch-cu_private_event-def.sql (START) [01:51:33] !log [agent@test131] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/beta.json /srv/mediawiki/w/maintenance/sql.php /srv/mediawiki-staging/1.40/extensions/CheckUser/schema/mysql/patch-cu_private_event-def.sql (END - exit=256) [01:51:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:51:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:52:07] !log [agent@test131] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/beta.json /srv/mediawiki/w/maintenance/sql.php /srv/mediawiki-staging/w/extensions/CheckUser/schema/mysql/patch-cu_private_event-def.sql (START) [01:52:11] !log [agent@test131] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/beta.json /srv/mediawiki/w/maintenance/sql.php /srv/mediawiki-staging/w/extensions/CheckUser/schema/mysql/patch-cu_private_event-def.sql (END - exit=0) [01:52:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:52:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:52:27] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.59, 2.48, 3.34 [01:52:28] !log [agent@test131] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/beta.json /srv/mediawiki/w/maintenance/sql.php /srv/mediawiki-staging/w/extensions/CheckUser/schema/mysql/patch-cu_log_event-def.sql (START) [01:52:32] !log [agent@test131] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/beta.json /srv/mediawiki/w/maintenance/sql.php /srv/mediawiki-staging/w/extensions/CheckUser/schema/mysql/patch-cu_log_event-def.sql (END - exit=0) [01:52:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:52:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:53:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.02, 11.48, 11.75 [02:01:45] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [02:05:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.82, 11.52, 11.13 [02:09:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.27, 10.99, 11.88 [02:09:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.48, 11.33, 11.13 [02:11:36] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 16.35, 13.20, 12.59 [02:15:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.06, 11.22, 11.12 [02:17:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.71, 10.83, 11.00 [02:19:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 14.04, 12.12, 11.45 [02:19:57] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.37, 3.52, 3.08 [02:20:58] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.97, 4.03, 3.22 [02:21:57] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.70, 3.18, 3.00 [02:22:55] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.24, 3.62, 3.16 [02:24:51] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.62, 3.40, 3.14 [02:25:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.99, 10.65, 11.10 [02:27:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.73, 11.30, 11.26 [02:29:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.17, 10.69, 11.06 [02:31:26] PROBLEM - cp23 APT on cp23 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [02:31:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.82, 10.30, 11.56 [02:33:27] RECOVERY - cp23 APT on cp23 is OK: APT OK: 36 packages available for upgrade (0 critical updates). [02:33:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 16.81, 11.95, 11.34 [02:35:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.89, 11.65, 11.33 [02:39:36] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 14.42, 11.57, 11.43 [02:39:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 15.50, 12.70, 11.70 [02:43:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.36, 11.34, 11.38 [02:49:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.55, 11.95, 11.58 [02:51:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.68, 11.21, 11.34 [02:53:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.69, 12.15, 11.68 [02:55:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.88, 11.87, 11.62 [02:57:01] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.31, 3.44, 2.91 [02:57:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.22, 12.17, 11.77 [02:58:58] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.37, 3.23, 2.91 [03:02:54] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.04, 4.33, 3.38 [03:04:51] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.14, 3.50, 3.19 [03:05:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.43, 11.50, 11.73 [03:06:48] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.56, 3.07, 3.08 [03:09:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.78, 10.34, 11.85 [03:11:45] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.1265469193 secs [03:15:57] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.22, 3.58, 3.28 [03:19:57] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.36, 3.27, 3.25 [03:21:41] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.43, 8.88, 10.02 [03:31:36] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.91, 9.01, 10.15 [03:33:38] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1365554035 secs [03:33:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.98, 10.73, 10.00 [03:33:45] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset 0.02125763893 secs [03:33:57] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.39, 3.24, 2.97 [03:35:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.98, 10.90, 10.72 [03:35:57] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.45, 3.06, 2.95 [03:37:36] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.60, 11.57, 10.99 [03:37:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.07, 10.75, 10.23 [03:39:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.64, 11.52, 10.58 [03:41:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.91, 11.15, 10.58 [03:43:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.21, 11.42, 10.74 [03:45:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.69, 11.12, 10.71 [03:49:16] !log [agent@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/Cargo:cargoRecreateData.php --wiki=solarwiki --table releases --replacement (END - exit=256) [03:49:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:49:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.05, 11.99, 11.13 [03:49:44] !log [agent@mwtask141] sudo -u www-data php /srv/mediawiki/w/extensions/Cargo/maintenance/cargoRecreateData.php --wiki=solarwiki --table releases --replacement (START) [03:49:45] !log [agent@mwtask141] sudo -u www-data php /srv/mediawiki/w/extensions/Cargo/maintenance/cargoRecreateData.php --wiki=solarwiki --table releases --replacement (END - exit=65280) [03:49:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:49:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:50:01] !log [agent@mwtask141] sudo -u www-data php /srv/mediawiki/w/extensions/Cargo/maintenance/cargoRecreateData.php --wiki=solarawiki --table releases --replacement (START) [03:50:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:51:09] !log [agent@mwtask141] sudo -u www-data php /srv/mediawiki/w/extensions/Cargo/maintenance/cargoRecreateData.php --wiki=solarawiki --table releases --replacement (END - exit=0) [03:51:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:51:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.89, 11.91, 11.95 [03:51:38] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset -0.008305132389 secs [03:51:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.25, 10.98, 10.85 [03:54:33] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.75, 3.33, 2.86 [03:56:30] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.25, 2.99, 2.80 [03:58:26] PROBLEM - test131 MediaWiki Rendering on test131 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:00:24] PROBLEM - test131 MediaWiki Rendering on test131 is WARNING: HTTP WARNING: HTTP/1.1 404 Not Found - 302 bytes in 0.636 second response time [04:00:24] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 3.85, 3.84, 1.74 [04:02:21] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 0.56, 2.61, 1.53 [04:05:41] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.31, 9.45, 10.12 [04:09:36] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.51, 9.22, 10.19 [04:19:45] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.1326162517 secs [04:34:32] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.73, 3.32, 2.82 [04:36:29] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.02, 2.76, 2.67 [04:42:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.32, 10.65, 9.40 [04:44:31] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 9.31, 10.00, 9.30 [04:51:11] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.76, 10.48, 9.73 [04:53:05] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.84, 10.00, 9.65 [04:55:45] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset 0.09155192971 secs [05:09:30] PROBLEM - open-justice.info - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'open-justice.info' expires in 15 day(s) (Sat 27 May 2023 05:00:42 GMT +0000). [05:10:13] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/7e80066f1c0c...ca92f53eb669 [05:10:14] [02miraheze/ssl] 07MirahezeSSLBot 03ca92f53 - Bot: Update SSL cert for open-justice.info [05:11:09] !log [agent@mwtask141] sudo -u www-data php /srv/mediawiki/w/extensions/Cargo/maintenance/cargoRecreateData.php --wiki=solarawiki --table releases --replacement (START) [05:11:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:11:22] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.30, 10.01, 9.25 [05:12:07] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.21, 10.22, 9.44 [05:12:18] !log [agent@mwtask141] sudo -u www-data php /srv/mediawiki/w/extensions/Cargo/maintenance/cargoRecreateData.php --wiki=solarawiki --table releases --replacement (END - exit=0) [05:12:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:13:50] !log [agent@mwtask141] sudo -u www-data php /srv/mediawiki/w/extensions/Cargo/maintenance/cargoRecreateData.php --wiki=solarawiki --table releases (START) [05:13:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:13:56] !log [agent@mwtask141] sudo -u www-data php /srv/mediawiki/w/extensions/Cargo/maintenance/cargoRecreateData.php --wiki=solarawiki --table releases (END - exit=0) [05:14:02] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.60, 10.26, 9.53 [05:14:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:15:19] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.68, 9.37, 9.19 [05:15:56] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.47, 9.78, 9.46 [05:19:57] !log [agent@mwtask141] sudo -u www-data php /srv/mediawiki/w/extensions/Cargo/maintenance/cargoRecreateData.php --wiki=solarawiki --table releases --replacement (START) [05:20:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:20:03] !log [agent@mwtask141] sudo -u www-data php /srv/mediawiki/w/extensions/Cargo/maintenance/cargoRecreateData.php --wiki=solarawiki --table releases --replacement (END - exit=0) [05:20:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:27:47] PROBLEM - eoa-wiki.ga - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'eoa-wiki.ga' expires in 15 day(s) (Sat 27 May 2023 05:15:34 GMT +0000). [05:35:10] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/ca92f53eb669...e003b2dabbf9 [05:35:13] [02miraheze/ssl] 07MirahezeSSLBot 03e003b2d - Bot: Update SSL cert for eoa-wiki.ga [05:36:00] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.31, 10.64, 9.58 [05:37:59] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.61, 9.96, 9.48 [05:39:01] RECOVERY - open-justice.info - LetsEncrypt on sslhost is OK: OK - Certificate 'open-justice.info' will expire on Wed 09 Aug 2023 04:10:03 GMT +0000. [05:40:50] [02mediawiki] 07wmfgerrit synchronize pull request 03#14708: Update 1.40 to latest - 13https://github.com/miraheze/mediawiki/pull/14708 [05:42:47] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.26, 10.81, 9.86 [05:44:48] [02mediawiki] 07wmfgerrit synchronize pull request 03#14708: Update 1.40 to latest - 13https://github.com/miraheze/mediawiki/pull/14708 [05:45:44] [02mediawiki] 07wmfgerrit synchronize pull request 03#14708: Update 1.40 to latest - 13https://github.com/miraheze/mediawiki/pull/14708 [05:46:31] PROBLEM - wiki.pulsus.cc - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.pulsus.cc' expires in 15 day(s) (Sat 27 May 2023 05:18:51 GMT +0000). [05:46:36] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 9.42, 10.12, 9.78 [05:46:50] [02mediawiki] 07wmfgerrit synchronize pull request 03#14708: Update 1.40 to latest - 13https://github.com/miraheze/mediawiki/pull/14708 [05:47:46] [02mediawiki] 07wmfgerrit synchronize pull request 03#14708: Update 1.40 to latest - 13https://github.com/miraheze/mediawiki/pull/14708 [05:48:28] [02mediawiki] 07wmfgerrit synchronize pull request 03#14708: Update 1.40 to latest - 13https://github.com/miraheze/mediawiki/pull/14708 [05:49:13] [02mediawiki] 07wmfgerrit synchronize pull request 03#14708: Update 1.40 to latest - 13https://github.com/miraheze/mediawiki/pull/14708 [05:50:04] [02mediawiki] 07wmfgerrit synchronize pull request 03#14708: Update 1.40 to latest - 13https://github.com/miraheze/mediawiki/pull/14708 [05:51:50] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/e003b2dabbf9...9bcd9cf60b4f [05:51:51] [02miraheze/ssl] 07MirahezeSSLBot 039bcd9cf - Bot: Update SSL cert for wiki.pulsus.cc [05:52:05] [02mediawiki] 07wmfgerrit synchronize pull request 03#14708: Update 1.40 to latest - 13https://github.com/miraheze/mediawiki/pull/14708 [05:56:35] RECOVERY - eoa-wiki.ga - LetsEncrypt on sslhost is OK: OK - Certificate 'eoa-wiki.ga' will expire on Wed 09 Aug 2023 04:35:02 GMT +0000. [05:59:45] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.1066450477 secs [05:59:56] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.92, 10.25, 9.76 [06:01:50] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.92, 9.99, 9.74 [06:11:36] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.66, 11.83, 10.51 [06:13:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.88, 10.53, 10.21 [06:16:02] RECOVERY - wiki.pulsus.cc - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.pulsus.cc' will expire on Wed 09 Aug 2023 04:51:43 GMT +0000. [06:17:36] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 9.98, 10.18, 10.14 [06:20:10] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.43, 7.60, 6.32 [06:23:45] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset 0.09843787551 secs [06:23:59] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.63, 7.72, 6.64 [06:25:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.81, 10.22, 10.14 [06:27:48] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.73, 6.58, 6.45 [06:29:36] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.50, 9.84, 10.05 [06:35:05] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - uk.religiononfire.mar.in.ua All nameservers failed to answer the query. [06:35:10] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.46, 2.60, 2.26 [06:37:06] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.59, 3.37, 2.58 [06:39:03] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.33, 2.58, 2.39 [07:04:55] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [07:09:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.55, 9.80, 8.91 [07:11:36] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.02, 10.87, 9.42 [07:13:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.95, 10.57, 9.49 [07:15:36] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.46, 10.00, 9.43 [07:36:42] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.religiononfire.mar.in.ua. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [07:41:45] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.453312248 secs [07:43:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.29, 10.15, 9.64 [07:47:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.41, 10.03, 9.55 [07:49:36] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.42, 11.12, 10.00 [07:51:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.30, 10.99, 10.08 [07:51:41] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.17, 10.15, 9.96 [07:57:36] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 9.97, 10.17, 10.01 [08:03:45] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset -0.05622237921 secs [08:06:32] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [08:11:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.65, 11.17, 10.41 [08:13:36] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.19, 10.10, 10.12 [08:39:21] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [08:43:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.12, 7.19, 6.55 [08:45:25] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.35, 6.70, 6.43 [08:51:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.40, 6.84, 6.55 [08:53:25] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.49, 6.52, 6.48 [08:55:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.80, 10.10, 9.08 [08:57:37] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 9.83, 9.99, 9.17 [09:00:46] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.97, 10.36, 9.66 [09:02:45] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 9.81, 9.75, 9.50 [09:05:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.36, 10.66, 9.82 [09:06:00] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 15.09, 6.74, 3.33 [09:06:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.53, 10.60, 10.01 [09:07:36] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.59, 11.19, 10.10 [09:08:31] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [09:08:39] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.05, 10.91, 10.18 [09:09:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.75, 11.14, 10.19 [09:11:36] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 14.78, 12.66, 10.87 [09:11:50] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 0.96, 3.40, 2.99 [09:12:36] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.61, 10.76, 10.30 [09:13:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.17, 6.80, 6.08 [09:14:34] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.48, 11.28, 10.51 [09:15:25] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.08, 6.01, 5.87 [09:15:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.36, 11.37, 10.79 [09:16:33] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.24, 10.79, 10.40 [09:20:30] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 9.56, 9.72, 10.03 [09:21:36] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.26, 9.61, 10.19 [09:39:36] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [09:41:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.77, 9.98, 9.52 [09:43:36] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 9.50, 9.58, 9.41 [09:47:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.47, 10.55, 9.89 [09:51:36] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.87, 9.94, 9.81 [09:54:15] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.95, 7.90, 6.55 [09:54:59] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.10, 10.34, 9.99 [09:56:10] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.44, 6.94, 6.36 [10:00:00] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.71, 6.77, 6.43 [10:02:53] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 9.68, 9.84, 9.89 [10:03:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.87, 10.29, 9.86 [10:05:36] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.61, 11.04, 10.17 [10:06:48] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.36, 12.74, 11.15 [10:07:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.44, 11.53, 10.48 [10:08:46] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [10:09:36] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.96, 12.60, 11.01 [10:12:43] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.11, 11.85, 11.31 [10:18:39] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.21, 11.28, 11.18 [10:20:37] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.73, 10.57, 10.95 [10:22:36] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.10, 11.11, 11.07 [10:24:34] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.32, 10.60, 10.88 [10:26:33] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 14.57, 12.51, 11.58 [10:38:23] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.66, 11.61, 11.79 [10:42:20] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.53, 11.65, 11.75 [10:44:19] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.03, 11.48, 11.68 [10:50:14] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.54, 12.14, 11.87 [10:52:12] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.96, 11.59, 11.71 [10:56:09] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.53, 11.50, 11.57 [10:58:08] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.57, 11.20, 11.43 [10:59:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.52, 7.68, 6.45 [11:00:06] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.65, 12.02, 11.70 [11:01:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.58, 6.95, 6.34 [11:03:25] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.03, 6.54, 6.25 [11:27:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.74, 7.12, 6.65 [11:31:25] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.10, 6.38, 6.45 [11:35:21] [02miraheze/GlobalNewFiles] 07translatewiki pushed 031 commit to 03master [+1/-0/±0] 13https://github.com/miraheze/GlobalNewFiles/compare/141dd4832a04...1b3b98ea804a [11:35:24] [02miraheze/GlobalNewFiles] 07translatewiki 031b3b98e - Localisation updates from https://translatewiki.net. [11:35:26] [02miraheze/ImportDump] 07translatewiki pushed 031 commit to 03master [+0/-0/±9] 13https://github.com/miraheze/ImportDump/compare/196f624b1ce3...22a619858e0a [11:35:29] [02miraheze/ImportDump] 07translatewiki 0322a6198 - Localisation updates from https://translatewiki.net. [11:35:32] [02miraheze/ErrorPages] 07translatewiki pushed 031 commit to 03master [+1/-0/±0] 13https://github.com/miraheze/ErrorPages/compare/d26c34941d0c...50f2ab567ac3 [11:35:33] [02miraheze/ErrorPages] 07translatewiki 0350f2ab5 - Localisation updates from https://translatewiki.net. [11:35:34] [02miraheze/CreateWiki] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/CreateWiki/compare/6ef5d4b20adb...c3abddf30d93 [11:35:35] [02miraheze/CreateWiki] 07translatewiki 03c3abddf - Localisation updates from https://translatewiki.net. [11:35:36] [02miraheze/ManageWiki] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ManageWiki/compare/26adf225bf00...cc66ceab864c [11:35:37] [02miraheze/ManageWiki] 07translatewiki 03cc66cea - Localisation updates from https://translatewiki.net. [11:37:05] miraheze/ErrorPages - translatewiki the build passed. [11:37:45] miraheze/GlobalNewFiles - translatewiki the build has errored. [11:42:08] miraheze/ManageWiki - translatewiki the build passed. [11:44:37] miraheze/CreateWiki - translatewiki the build passed. [11:47:01] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.19, 6.85, 6.46 [11:48:31] miraheze/ImportDump - translatewiki the build passed. [11:48:56] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.10, 6.66, 6.43 [11:54:40] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.21, 7.12, 6.72 [11:56:35] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.26, 6.69, 6.61 [12:00:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.57, 6.95, 6.74 [12:02:19] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.23, 6.56, 6.63 [12:11:09] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 10.07, 6.10, 3.20 [12:11:53] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.47, 7.78, 7.13 [12:12:10] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.43, 3.11, 2.33 [12:13:48] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.05, 8.59, 7.52 [12:14:10] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.10, 3.11, 2.43 [12:15:06] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 3.00, 5.85, 3.88 [12:15:43] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.76, 7.42, 7.21 [12:19:07] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.1337724924 secs [12:21:07] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.09882918 secs [12:21:16] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 10.09, 6.28, 3.43 [12:22:34] [02mediawiki] 07wmfgerrit synchronize pull request 03#14708: Update 1.40 to latest - 13https://github.com/miraheze/mediawiki/pull/14708 [12:23:25] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.57, 6.14, 6.72 [12:29:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.58, 7.19, 6.92 [12:30:10] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.81, 3.35, 2.85 [12:31:25] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.06, 6.73, 6.78 [12:32:10] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.39, 3.31, 2.89 [12:35:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.64, 6.91, 6.85 [12:37:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.21, 7.77, 7.20 [12:41:02] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.14, 3.65, 3.19 [12:41:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.20, 7.53, 7.26 [12:43:00] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.81, 3.77, 3.29 [12:43:15] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.04, 1.94, 3.70 [12:44:58] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.04, 3.77, 3.35 [12:45:15] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 0.85, 1.65, 3.38 [12:46:55] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.16, 3.56, 3.32 [12:53:07] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset -0.1050526202 secs [12:55:07] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset -0.08662122488 secs [12:55:25] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.23, 6.42, 6.74 [12:56:43] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.70, 3.31, 3.35 [13:05:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.72, 7.14, 6.89 [13:06:33] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.81, 3.03, 2.20 [13:08:30] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.04, 3.48, 2.47 [13:10:27] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.94, 2.81, 2.34 [13:13:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.36, 7.65, 7.51 [13:21:06] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 9.26, 6.26, 3.74 [13:23:05] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 5.98, 7.00, 4.39 [13:25:03] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 1.41, 4.96, 3.96 [13:29:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.71, 7.76, 7.39 [13:33:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.53, 7.49, 7.39 [13:35:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.98, 8.72, 7.87 [13:35:36] PROBLEM - mw142 Puppet on mw142 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 3 minutes ago with 2 failures. Failed resources (up to 3 shown): Exec[git_pull_JobRunner],Exec[git_pull_mathoid] [13:36:50] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.27, 4.63, 3.47 [13:37:05] PROBLEM - mw122 Puppet on mw122 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 2 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[git_pull_JobRunner],Exec[git_pull_3d2png],Exec[git_pull_femiwiki-deploy] [13:40:20] PROBLEM - mw141 Puppet on mw141 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 3 minutes ago with 2 failures. Failed resources (up to 3 shown): Exec[git_pull_mathoid],Exec[git_pull_3d2png] [13:40:31] !log [@mwtask141] starting deploy of {'config': True} to all [13:40:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [13:40:44] PROBLEM - mw132 Puppet on mw132 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 3 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[git_pull_JobRunner],Exec[git_pull_mathoid],Exec[git_pull_3d2png] [13:40:56] !log [@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 25s [13:41:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [13:42:40] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.34, 3.45, 3.42 [13:43:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 4.57, 7.05, 7.54 [13:43:41] PROBLEM - mw131 Puppet on mw131 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_mathoid] [13:43:54] PROBLEM - mwtask141 Puppet on mwtask141 is CRITICAL: CRITICAL: Puppet has 4 failures. Last run 2 minutes ago with 4 failures. Failed resources (up to 3 shown): Exec[git_pull_ErrorPages],Exec[git_pull_JobRunner],Exec[git_pull_mathoid],Exec[git_pull_femiwiki-deploy] [13:44:13] PROBLEM - mw121 Puppet on mw121 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 2 minutes ago with 2 failures. Failed resources (up to 3 shown): Exec[git_pull_mathoid],Exec[git_pull_3d2png] [13:44:37] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.80, 3.37, 3.40 [13:47:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.57, 7.89, 7.74 [13:50:30] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.06, 3.45, 3.46 [13:51:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.64, 7.97, 7.85 [13:52:26] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.03, 2.81, 3.22 [13:55:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 10.17, 8.36, 7.98 [13:57:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.38, 7.10, 7.56 [13:59:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.84, 8.17, 7.89 [14:01:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.83, 7.30, 7.61 [14:01:34] RECOVERY - mw142 Puppet on mw142 is OK: OK: Puppet is currently enabled, last run 48 seconds ago with 0 failures [14:05:22] RECOVERY - mw122 Puppet on mw122 is OK: OK: Puppet is currently enabled, last run 17 seconds ago with 0 failures [14:06:18] RECOVERY - mw141 Puppet on mw141 is OK: OK: Puppet is currently enabled, last run 37 seconds ago with 0 failures [14:06:50] RECOVERY - mw132 Puppet on mw132 is OK: OK: Puppet is currently enabled, last run 43 seconds ago with 0 failures [14:11:40] RECOVERY - mw131 Puppet on mw131 is OK: OK: Puppet is currently enabled, last run 51 seconds ago with 0 failures [14:11:50] RECOVERY - mwtask141 Puppet on mwtask141 is OK: OK: Puppet is currently enabled, last run 47 seconds ago with 0 failures [14:12:15] RECOVERY - mw121 Puppet on mw121 is OK: OK: Puppet is currently enabled, last run 23 seconds ago with 0 failures [14:23:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.51, 7.32, 7.13 [14:27:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.00, 7.17, 7.15 [14:32:22] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 8.20, 6.23, 3.40 [14:34:22] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 3.07, 4.88, 3.24 [14:37:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.16, 7.96, 7.56 [14:38:18] PROBLEM - lebork.info.pl - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'lebork.info.pl' expires in 15 day(s) (Sat 27 May 2023 14:21:47 GMT +0000). [14:39:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.92, 7.27, 7.36 [14:41:18] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/9bcd9cf60b4f...3072960bf253 [14:41:19] [02miraheze/ssl] 07MirahezeSSLBot 033072960 - Bot: Update SSL cert for lebork.info.pl [14:45:19] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.53, 3.41, 2.89 [14:47:14] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 3.13, 3.39, 2.95 [14:49:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 10.33, 7.69, 7.25 [14:51:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.66, 7.76, 7.34 [14:53:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.05, 7.85, 7.42 [14:54:18] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.34, 3.10, 2.20 [14:56:10] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.64, 3.52, 2.94 [14:57:07] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.1555478871 secs [14:58:11] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.81, 3.34, 2.62 [15:00:04] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.75, 2.97, 2.87 [15:02:07] PROBLEM - biblestrength.net - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for biblestrength.net could not be found [15:05:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.72, 7.63, 7.69 [15:07:08] RECOVERY - lebork.info.pl - LetsEncrypt on sslhost is OK: OK - Certificate 'lebork.info.pl' will expire on Wed 09 Aug 2023 13:41:09 GMT +0000. [15:09:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.25, 7.89, 7.79 [15:10:10] PROBLEM - biblestrength.net - LetsEncrypt on sslhost is CRITICAL: connect to address biblestrength.net and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [15:11:07] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.09352001548 secs [15:11:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.69, 7.45, 7.61 [15:17:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.76, 7.90, 7.73 [15:18:39] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.84, 4.28, 3.28 [15:20:38] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.78, 3.73, 3.21 [15:21:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.86, 7.73, 7.78 [15:22:33] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.28, 3.33, 3.13 [15:25:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.55, 7.72, 7.73 [15:28:49] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 5.32, 2.77, 1.82 [15:29:00] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [15:29:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.97, 7.61, 7.65 [15:30:48] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 2.37, 2.57, 1.87 [15:45:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.09, 7.47, 7.32 [15:47:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.85, 6.82, 7.10 [15:49:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.45, 7.29, 7.23 [15:50:59] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 11.09, 6.89, 4.20 [15:51:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.40, 6.73, 7.04 [15:54:55] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 3.83, 6.87, 4.98 [15:55:25] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.12, 6.32, 6.79 [15:56:54] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 2.09, 5.64, 4.78 [15:59:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.68, 7.30, 7.05 [16:01:07] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.1140527725 secs [16:01:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.19, 7.34, 7.10 [16:05:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.76, 7.31, 7.08 [16:06:29] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.59, 3.27, 2.81 [16:07:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.23, 7.28, 7.10 [16:08:26] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 3.07, 3.23, 2.85 [16:11:07] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.09887352586 secs [16:15:25] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.65, 6.29, 6.75 [16:21:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 14.25, 10.10, 8.10 [16:24:00] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [16:25:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.47, 7.74, 7.61 [16:27:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.21, 7.43, 7.48 [16:28:31] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: CRITICAL - load average: 4.62, 3.47, 2.83 [16:29:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.51, 7.20, 7.39 [16:30:30] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.75, 3.52, 2.92 [16:34:30] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 2.98, 3.38, 3.02 [16:35:28] PROBLEM - wiki.sukotuze.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.sukotuze.com' expires in 15 day(s) (Sat 27 May 2023 16:14:49 GMT +0000). [16:35:59] PROBLEM - swiftobject111 APT on swiftobject111 is CRITICAL: APT CRITICAL: 47 packages available for upgrade (1 critical updates). [16:36:45] PROBLEM - cp32 APT on cp32 is CRITICAL: APT CRITICAL: 36 packages available for upgrade (1 critical updates). [16:38:03] PROBLEM - mon141 APT on mon141 is CRITICAL: APT CRITICAL: 53 packages available for upgrade (1 critical updates). [16:38:14] PROBLEM - matomo121 APT on matomo121 is CRITICAL: APT CRITICAL: 42 packages available for upgrade (1 critical updates). [16:39:43] PROBLEM - os141 APT on os141 is CRITICAL: APT CRITICAL: 41 packages available for upgrade (1 critical updates). [16:40:15] PROBLEM - swiftproxy131 APT on swiftproxy131 is CRITICAL: APT CRITICAL: 47 packages available for upgrade (1 critical updates). [16:40:22] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/3072960bf253...25e2929abae9 [16:40:24] [02miraheze/ssl] 07MirahezeSSLBot 0325e2929 - Bot: Update SSL cert for wiki.sukotuze.com [16:41:08] PROBLEM - puppet141 APT on puppet141 is CRITICAL: APT CRITICAL: 42 packages available for upgrade (2 critical updates). [16:41:18] PROBLEM - cloud10 Puppet on cloud10 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [16:41:42] PROBLEM - cloud10 APT on cloud10 is CRITICAL: APT CRITICAL: 101 packages available for upgrade (1 critical updates). [16:41:44] PROBLEM - phab121 APT on phab121 is CRITICAL: APT CRITICAL: 41 packages available for upgrade (1 critical updates). [16:41:50] PROBLEM - mwtask141 Puppet on mwtask141 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [16:42:33] PROBLEM - bast121 APT on bast121 is CRITICAL: APT CRITICAL: 38 packages available for upgrade (1 critical updates). [16:42:46] PROBLEM - cloud12 APT on cloud12 is CRITICAL: APT CRITICAL: 100 packages available for upgrade (1 critical updates). [16:42:47] PROBLEM - db142 APT on db142 is CRITICAL: APT CRITICAL: 46 packages available for upgrade (1 critical updates). [16:42:59] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:43:25] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.79, 6.41, 6.78 [16:43:59] PROBLEM - swiftproxy111 APT on swiftproxy111 is CRITICAL: APT CRITICAL: 38 packages available for upgrade (1 critical updates). [16:44:23] PROBLEM - cp22 APT on cp22 is CRITICAL: APT CRITICAL: 36 packages available for upgrade (1 critical updates). [16:46:03] PROBLEM - mem141 APT on mem141 is CRITICAL: APT CRITICAL: 47 packages available for upgrade (1 critical updates). [16:46:35] PROBLEM - swiftobject113 APT on swiftobject113 is CRITICAL: APT CRITICAL: 47 packages available for upgrade (1 critical updates). [16:46:46] PROBLEM - os131 APT on os131 is CRITICAL: APT CRITICAL: 41 packages available for upgrade (1 critical updates). [16:47:07] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset -0.1105722785 secs [16:47:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.31, 6.82, 6.87 [16:48:23] PROBLEM - swiftobject112 APT on swiftobject112 is CRITICAL: APT CRITICAL: 47 packages available for upgrade (1 critical updates). [16:48:27] PROBLEM - jobchron121 APT on jobchron121 is CRITICAL: APT CRITICAL: 47 packages available for upgrade (1 critical updates). [16:49:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.19, 7.33, 7.04 [16:50:21] PROBLEM - ns2 APT on ns2 is CRITICAL: APT CRITICAL: 38 packages available for upgrade (1 critical updates). [16:51:16] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 9.237 second response time [16:51:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.07, 6.74, 6.86 [16:51:49] PROBLEM - db121 APT on db121 is CRITICAL: APT CRITICAL: 55 packages available for upgrade (1 critical updates). [16:52:08] PROBLEM - cloud11 APT on cloud11 is CRITICAL: APT CRITICAL: 45 packages available for upgrade (1 critical updates). [16:52:22] PROBLEM - swiftobject121 APT on swiftobject121 is CRITICAL: APT CRITICAL: 47 packages available for upgrade (1 critical updates). [16:52:26] PROBLEM - swiftobject122 APT on swiftobject122 is CRITICAL: APT CRITICAL: 47 packages available for upgrade (1 critical updates). [16:52:29] PROBLEM - cloud13 APT on cloud13 is CRITICAL: APT CRITICAL: 100 packages available for upgrade (1 critical updates). [16:52:31] PROBLEM - db101 APT on db101 is CRITICAL: APT CRITICAL: 55 packages available for upgrade (1 critical updates). [16:53:25] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.91, 6.20, 6.65 [16:54:34] PROBLEM - cloud14 APT on cloud14 is CRITICAL: APT CRITICAL: 110 packages available for upgrade (1 critical updates). [16:55:01] PROBLEM - cp33 APT on cp33 is CRITICAL: APT CRITICAL: 36 packages available for upgrade (1 critical updates). [16:55:07] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset -0.09300488234 secs [16:55:38] PROBLEM - swiftac111 APT on swiftac111 is CRITICAL: APT CRITICAL: 38 packages available for upgrade (1 critical updates). [16:55:57] PROBLEM - prometheus131 APT on prometheus131 is CRITICAL: APT CRITICAL: 38 packages available for upgrade (1 critical updates). [16:56:27] PROBLEM - mem131 APT on mem131 is CRITICAL: APT CRITICAL: 47 packages available for upgrade (1 critical updates). [16:57:32] PROBLEM - ns1 APT on ns1 is CRITICAL: APT CRITICAL: 37 packages available for upgrade (1 critical updates). [16:58:22] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 8.80, 5.37, 3.22 [16:59:00] PROBLEM - cp23 APT on cp23 is CRITICAL: APT CRITICAL: 36 packages available for upgrade (1 critical updates). [17:00:14] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.96, 6.95, 6.81 [17:00:27] PROBLEM - reports121 APT on reports121 is CRITICAL: APT CRITICAL: 45 packages available for upgrade (1 critical updates). [17:01:11] PROBLEM - bast141 APT on bast141 is CRITICAL: APT CRITICAL: 38 packages available for upgrade (1 critical updates). [17:02:09] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.54, 6.31, 6.59 [17:02:15] PROBLEM - db131 APT on db131 is CRITICAL: APT CRITICAL: 55 packages available for upgrade (1 critical updates). [17:02:22] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 4.06, 6.08, 4.10 [17:03:22] PROBLEM - db112 APT on db112 is CRITICAL: APT CRITICAL: 55 packages available for upgrade (1 critical updates). [17:04:36] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 9.23, 6.11, 4.08 [17:04:44] PROBLEM - graylog131 APT on graylog131 is CRITICAL: APT CRITICAL: 44 packages available for upgrade (1 critical updates). [17:05:00] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [17:05:26] RECOVERY - wiki.sukotuze.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.sukotuze.com' will expire on Wed 09 Aug 2023 15:40:13 GMT +0000. [17:05:29] PROBLEM - ldap141 APT on ldap141 is CRITICAL: APT CRITICAL: 38 packages available for upgrade (1 critical updates). [17:07:54] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.14, 7.16, 6.92 [17:09:18] RECOVERY - cloud10 Puppet on cloud10 is OK: OK: Puppet is currently enabled, last run 3 seconds ago with 0 failures [17:09:48] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.92, 6.74, 6.80 [17:10:00] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [17:11:44] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/ssl/compare/25e2929abae9...6c8b4bfff8a6 [17:11:45] [02miraheze/ssl] 07MirahezeSSLBot 036c8b4bf - Bot: Add SSL cert for dlfm-wiki.top [17:11:50] RECOVERY - mwtask141 Puppet on mwtask141 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [17:12:14] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/ssl/compare/6c8b4bfff8a6...3512cc94d6e8 [17:12:17] [02miraheze/ssl] 07MirahezeSSLBot 033512cc9 - Bot: Add SSL cert for o.kaoyu666.cn [17:12:23] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.17, 3.15, 3.57 [17:16:17] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.38, 2.35, 3.16 [17:16:31] [02dns] 07MacFan4000 opened pull request 03#411: Add bushcraftwiki.com zone - 13https://github.com/miraheze/dns/pull/411 [17:27:10] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.14, 7.22, 6.81 [17:31:03] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.50, 6.82, 6.79 [17:33:00] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.06, 6.75, 6.77 [17:38:00] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [17:42:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.26, 7.13, 6.77 [17:44:42] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.68, 6.54, 6.60 [17:45:48] PROBLEM - cp22 Current Load on cp22 is WARNING: WARNING - load average: 3.46, 2.69, 1.75 [17:47:46] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 1.19, 2.11, 1.65 [17:50:30] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.90, 7.58, 7.07 [17:52:35] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.99, 4.62, 3.33 [17:54:34] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.11, 3.69, 3.15 [17:56:32] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.04, 3.30, 3.08 [17:58:17] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.66, 6.17, 6.63 [18:13:00] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [18:14:49] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [18:19:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 12.67, 10.09, 7.91 [18:20:21] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.45, 6.20, 4.99 [18:20:34] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 8.61, 5.69, 3.20 [18:22:21] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 6.38, 5.94, 5.03 [18:22:34] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 4.35, 5.25, 3.36 [18:29:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.79, 7.49, 7.71 [18:33:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 10.60, 8.46, 7.95 [18:35:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.54, 7.24, 7.57 [18:39:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.60, 7.15, 7.39 [18:41:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.82, 6.97, 7.29 [18:43:34] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [18:49:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.57, 7.62, 7.47 [18:50:40] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.39, 3.51, 3.18 [18:50:46] [02mw-config] 07Universal-Omega opened pull request 03#5228: Update Cosmos search config help messages to mention Codex not WVUI - 13https://github.com/miraheze/mw-config/pull/5228 [18:52:38] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 3.14, 3.39, 3.17 [18:56:05] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [18:56:34] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.83, 3.86, 3.36 [18:57:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.05, 7.48, 7.65 [19:00:31] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.03, 3.11, 3.18 [19:11:05] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [19:33:45] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.72, 6.21, 6.64 [19:40:38] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 9.73, 6.10, 3.25 [19:41:29] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:42:35] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 4.69, 5.72, 3.47 [19:43:31] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 5.848 second response time [19:47:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 12.73, 7.77, 6.92 [19:51:45] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.31, 6.74, 6.71 [20:10:29] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.60, 3.48, 2.89 [20:11:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.40, 7.70, 7.08 [20:12:28] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.20, 2.91, 2.75 [20:13:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.10, 7.41, 7.06 [20:15:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.36, 7.79, 7.24 [20:17:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.47, 7.25, 7.10 [20:29:12] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.37, 3.43, 2.91 [20:31:10] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.88, 3.83, 3.14 [20:31:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.89, 7.57, 7.14 [20:33:08] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 7.01, 5.30, 3.78 [20:35:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.84, 7.20, 7.07 [20:41:02] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.91, 3.60, 3.65 [20:43:00] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.54, 3.91, 3.75 [20:43:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.92, 7.76, 7.22 [20:45:58] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.1159750521 secs [20:47:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.14, 6.92, 7.04 [20:51:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.44, 7.58, 7.26 [20:52:52] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.37, 3.83, 3.98 [20:53:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.99, 6.89, 7.04 [20:58:46] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.56, 2.43, 3.29 [21:11:45] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.66, 6.35, 6.77 [21:17:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.22, 7.55, 7.12 [21:19:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.27, 7.19, 7.05 [21:23:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.48, 7.46, 7.16 [21:25:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.22, 7.42, 7.18 [21:26:19] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.08, 3.53, 3.29 [21:28:18] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.31, 3.15, 3.18 [21:29:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.28, 7.94, 7.39 [21:31:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.35, 7.29, 7.22 [21:37:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.04, 7.37, 7.22 [21:39:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.35, 7.03, 7.12 [21:43:57] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset 0.092625916 secs [21:45:45] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.81, 6.26, 6.75 [21:49:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.04, 7.51, 7.16 [21:55:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.47, 7.60, 7.37 [22:07:00] [02mw-config] 07Amanda-Catherine commented on pull request 03#5225: Userrights config for nycsubwaywiki - 13https://github.com/miraheze/mw-config/pull/5225#issuecomment-1544753936 [22:08:44] [02mw-config] 07AgentIsai commented on pull request 03#5225: Userrights config for nycsubwaywiki - 13https://github.com/miraheze/mw-config/pull/5225#issuecomment-1544755524 [22:09:45] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.36, 6.26, 6.77 [22:15:36] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.89, 3.62, 2.97 [22:17:34] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.52, 3.79, 3.13 [22:19:34] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.42, 4.86, 3.59 [22:20:04] [02MirahezeMagic] 07Amanda-Catherine opened pull request 03#425: Update group permissions - 13https://github.com/miraheze/MirahezeMagic/pull/425 [22:21:34] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.03, 3.98, 3.42 [22:27:34] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 3.23, 3.25, 3.24 [22:30:35] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.98, 6.51, 6.54 [22:32:33] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.55, 6.06, 6.37 [22:36:29] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.63, 4.49, 3.67 [22:38:28] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.19, 3.88, 2.94 [22:38:28] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.01, 3.98, 3.59 [22:46:28] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.78, 2.78, 2.82 [22:54:15] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.11, 2.73, 3.21 [22:55:46] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.68, 6.82, 6.72 [22:57:45] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.25, 6.14, 6.48 [23:05:27] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 7.58, 5.99, 3.37 [23:07:24] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 1.50, 4.23, 3.04 [23:10:27] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.09, 7.71, 6.99 [23:12:24] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.36, 6.98, 6.82 [23:14:20] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.76, 7.59, 7.05 [23:16:17] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.78, 6.88, 6.85 [23:20:12] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 13.61, 9.05, 7.60 [23:22:08] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.80, 7.95, 7.36 [23:24:45] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.38, 3.41, 3.08 [23:26:43] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.51, 3.00, 2.97 [23:35:46] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.87, 7.40, 7.13 [23:36:35] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.72, 3.70, 3.26 [23:37:42] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.25, 4.31, 3.34 [23:38:33] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.59, 3.86, 3.39 [23:39:39] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.24, 3.90, 3.31 [23:39:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.28, 7.56, 7.28 [23:41:37] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.87, 3.08, 3.08 [23:44:28] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.01, 4.13, 3.60 [23:45:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.08, 7.56, 7.36 [23:46:26] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.77, 3.72, 3.52 [23:47:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.94, 7.09, 7.22 [23:49:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.78, 7.86, 7.47 [23:51:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.84, 7.60, 7.41 [23:53:19] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 8.46, 5.38, 3.95 [23:54:19] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.97, 4.09, 3.72 [23:55:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 11.24, 8.14, 7.57 [23:57:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.20, 7.20, 7.29 [23:59:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.78, 7.85, 7.49