[00:01:17] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.79, 10.66, 10.27 [00:01:33] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.88, 3.97, 3.92 [00:03:17] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.00, 10.25, 10.17 [00:03:36] PROBLEM - os141 Current Load on os141 is CRITICAL: CRITICAL - load average: 4.18, 2.56, 1.35 [00:05:10] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.93, 3.98, 3.60 [00:05:32] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 2.51, 2.44, 1.45 [00:07:11] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.77, 4.54, 3.82 [00:07:46] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.106258899 secs [00:09:17] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.81, 10.98, 10.48 [00:10:51] PROBLEM - mw121 Disk Space on mw121 is WARNING: DISK WARNING - free space: / 1333 MB (6% inode=81%); [00:12:51] RECOVERY - mw121 Disk Space on mw121 is OK: DISK OK - free space: / 5167 MB (23% inode=81%); [00:13:17] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.81, 11.97, 11.09 [00:15:33] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.43, 3.44, 3.86 [00:15:42] [02mw-config] 07AgentIsai closed pull request 03#5222: Initial multiversion support - 13https://github.com/miraheze/mw-config/pull/5222 [00:15:45] [02miraheze/mw-config] 07AgentIsai pushed 031 commit to 03master [+8/-0/±1] 13https://github.com/miraheze/mw-config/compare/c883ea006b61...2bfb070d0e84 [00:15:48] [02miraheze/mw-config] 07Universal-Omega 032bfb070 - Initial multiversion support (#5222) [00:16:53] miraheze/mw-config - AgentIsai the build passed. [00:17:33] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.43, 3.76, 3.92 [00:17:37] !log [agent@test131] starting deploy of {'config': True} to all [00:17:38] !log [agent@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [00:17:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:17:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:19:45] !log [agent@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [00:19:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:20:04] !log [agent@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 19s [00:20:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:21:11] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.52, 3.75, 3.95 [00:21:15] PROBLEM - cp33 APT on cp33 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [00:23:10] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 3.80, 4.24, 4.13 [00:23:19] RECOVERY - cp33 APT on cp33 is OK: APT OK: 36 packages available for upgrade (0 critical updates). [00:23:33] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.06, 3.29, 3.75 [00:23:46] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.09298616648 secs [00:24:05] PROBLEM - os141 Current Load on os141 is WARNING: WARNING - load average: 3.62, 2.93, 2.32 [00:25:11] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.16, 3.78, 3.97 [00:26:01] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 2.58, 2.75, 2.32 [00:27:17] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 9.09, 9.29, 10.14 [00:27:26] !log [@test131] starting deploy of {'config': True} to all [00:27:27] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 1s [00:27:33] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.55, 3.71, 3.77 [00:27:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:27:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:29:33] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.70, 3.26, 3.60 [00:30:41] PROBLEM - cp23 APT on cp23 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [00:31:33] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.85, 2.79, 3.38 [00:32:00] !log [agent@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/sql.php --wiki=loginwiki --wikidb=mhglobal ALTER TABLE cw_wikis ADD COLUMN wiki_version TEXT DEFAULT NULL (END - exit=256) [00:32:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:33:49] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 9.52, 5.64, 3.12 [00:35:11] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 7.38, 4.77, 4.08 [00:35:33] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.74, 3.13, 3.43 [00:35:41] RECOVERY - cp23 APT on cp23 is OK: APT OK: 36 packages available for upgrade (0 critical updates). [00:36:00] !log [agent@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/sql.php --wiki=loginwiki --wikidb=testglobal [00:36:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:36:08] !log [agent@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/sql.php --wiki=loginwiki --wikidb=testglobal --query ALTER TABLE cw_wikis ADD COLUMN wiki_version TEXT DEFAULT NULL; [00:36:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:37:33] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.21, 2.79, 3.26 [00:37:48] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 0.66, 2.83, 2.54 [00:38:49] PROBLEM - os141 Current Load on os141 is CRITICAL: CRITICAL - load average: 5.19, 3.63, 2.82 [00:39:04] [02mw-config] 07Universal-Omega opened pull request 03#5223: Enable some multiversion functionality - 13https://github.com/miraheze/mw-config/pull/5223 [00:40:45] PROBLEM - os141 Current Load on os141 is WARNING: WARNING - load average: 3.98, 3.77, 2.98 [00:41:08] miraheze/mw-config - Universal-Omega the build passed. [00:41:09] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.09, 10.55, 10.05 [00:41:11] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.05, 3.73, 3.97 [00:43:08] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.21, 10.28, 10.04 [00:44:29] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.86, 3.52, 3.46 [00:44:37] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 2.48, 3.05, 2.86 [00:45:07] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 5.80, 8.74, 9.51 [00:47:11] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.31, 3.59, 3.74 [00:48:23] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.81, 3.98, 3.60 [00:49:11] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.83, 3.55, 3.70 [00:50:17] [02mw-config] 07Universal-Omega synchronize pull request 03#5223: Enable some multiversion functionality - 13https://github.com/miraheze/mw-config/pull/5223 [00:50:19] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.22, 3.23, 3.37 [00:51:11] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 8.38, 4.80, 4.10 [00:51:18] miraheze/mw-config - Universal-Omega the build passed. [00:59:11] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.83, 3.49, 3.92 [00:59:19] [02mw-config] 07AgentIsai closed pull request 03#5223: Enable some multiversion functionality - 13https://github.com/miraheze/mw-config/pull/5223 [00:59:20] [02miraheze/mw-config] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/2bfb070d0e84...6cf7c15e0f9e [00:59:21] [02miraheze/mw-config] 07Universal-Omega 036cf7c15 - Enable some multiversion functionality (#5223) [00:59:38] !log [agent@test131] starting deploy of {'config': True} to all [00:59:39] !log [agent@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [00:59:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:59:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:59:48] !log [agent@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [00:59:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:59:56] !log [agent@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 8s [01:00:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:00:19] miraheze/mw-config - AgentIsai the build passed. [01:01:10] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.43, 4.11, 4.10 [01:03:11] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.28, 3.39, 3.84 [01:03:59] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.29, 4.47, 3.70 [01:05:10] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.78, 3.92, 3.97 [01:07:10] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.50, 3.29, 3.73 [01:08:48] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 14.86, 10.89, 9.17 [01:11:11] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.65, 3.12, 3.49 [01:12:45] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.85, 11.09, 9.68 [01:13:10] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.45, 3.60, 3.66 [01:18:41] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.10, 9.41, 9.47 [01:19:11] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.94, 4.08, 3.83 [01:19:33] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.74, 3.53, 4.00 [01:21:11] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.10, 3.71, 3.73 [01:21:33] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.05, 4.24, 4.20 [01:23:11] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.43, 4.04, 3.84 [01:27:31] !log [@test131] starting deploy of {'config': True} to all [01:27:32] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [01:27:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:27:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:30:31] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.63, 11.89, 10.45 [01:32:43] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.38, 6.98, 6.33 [01:34:40] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.59, 6.67, 6.31 [01:36:26] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.19, 11.77, 10.90 [01:36:30] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.92, 10.79, 9.56 [01:38:25] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.57, 12.05, 11.10 [01:38:25] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.74, 9.82, 9.36 [01:39:10] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.58, 3.54, 3.95 [01:42:15] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.23, 11.29, 10.11 [01:43:11] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 7.09, 4.24, 4.05 [01:47:59] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.88, 11.63, 10.61 [01:51:33] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.23, 2.99, 3.81 [01:51:48] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.62, 11.77, 10.90 [01:54:13] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.77, 11.47, 12.00 [01:55:37] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.59, 9.51, 10.15 [01:56:12] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.19, 11.81, 12.06 [01:57:33] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.06, 3.47, 3.76 [01:59:27] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.84, 10.09, 10.33 [01:59:33] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 4.00, 3.86, 3.87 [02:00:09] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.79, 11.11, 11.75 [02:03:33] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.73, 2.63, 3.35 [02:04:06] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.29, 11.62, 11.76 [02:05:10] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.90, 2.92, 3.96 [02:06:05] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.75, 11.47, 11.68 [02:07:33] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.28, 4.44, 3.96 [02:09:33] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.58, 3.99, 3.87 [02:10:02] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 15.18, 12.58, 12.02 [02:12:28] RECOVERY - cloud11 IPMI Sensors on cloud11 is OK: IPMI Status: OK [02:15:11] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.62, 2.48, 3.28 [02:15:58] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.37, 11.31, 11.72 [02:16:28] PROBLEM - cloud11 IPMI Sensors on cloud11 is CRITICAL: IPMI Status: Critical [Cntlr 2 Bay 8 = Critical] [02:17:33] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.12, 2.69, 3.25 [02:18:55] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 14.27, 11.43, 10.83 [02:20:55] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.12, 10.44, 10.54 [02:21:33] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.95, 3.49, 3.46 [02:22:03] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.45, 7.45, 6.00 [02:23:33] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.74, 3.10, 3.31 [02:23:52] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.03, 11.07, 11.31 [02:24:03] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.71, 6.50, 5.81 [02:25:11] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.53, 3.51, 3.38 [02:25:51] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.30, 10.87, 11.22 [02:26:55] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.43, 11.55, 10.91 [02:27:10] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.31, 3.88, 3.53 [02:29:10] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.90, 3.42, 3.39 [02:30:55] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.64, 11.46, 11.03 [02:31:11] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.10, 4.10, 3.64 [02:33:10] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.35, 3.86, 3.61 [02:34:55] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 14.92, 12.63, 11.53 [02:35:10] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.03, 3.19, 3.40 [02:36:55] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.02, 11.83, 11.36 [02:37:33] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.00, 4.69, 3.60 [02:37:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.21, 11.61, 11.37 [02:38:55] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.47, 12.77, 11.79 [02:39:23] miraheze/MirahezeMagic - AgentIsai the build has errored. [02:39:33] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.74, 3.97, 3.45 [02:41:28] [02miraheze/MirahezeMagic] 07AgentIsai pushed 031 commit to 03vanishuser [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/fe61d5484994...fcd1815063d9 [02:41:29] [02miraheze/MirahezeMagic] 07AgentIsai 03fcd1815 - Use MediaWiki\Skin [02:41:32] [02MirahezeMagic] 07AgentIsai synchronize pull request 03#421: [DMG] Testing for VanishUser - 13https://github.com/miraheze/MirahezeMagic/pull/421 [02:41:33] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.46, 3.01, 3.16 [02:44:00] [02MirahezeMagic] 07AgentIsai synchronize pull request 03#421: [DMG] Testing for VanishUser - 13https://github.com/miraheze/MirahezeMagic/pull/421 [02:44:02] [02miraheze/MirahezeMagic] 07AgentIsai pushed 031 commit to 03vanishuser [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/fcd1815063d9...463a96a67b8a [02:44:03] [02miraheze/MirahezeMagic] 07AgentIsai 03463a96a - fix [02:44:22] miraheze/MirahezeMagic - AgentIsai the build has errored. [02:44:55] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.32, 11.11, 11.48 [02:46:16] [02MirahezeMagic] 07AgentIsai synchronize pull request 03#421: [DMG] Testing for VanishUser - 13https://github.com/miraheze/MirahezeMagic/pull/421 [02:46:20] [02miraheze/MirahezeMagic] 07AgentIsai pushed 031 commit to 03vanishuser [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/463a96a67b8a...099edb2fc82f [02:46:22] [02miraheze/MirahezeMagic] 07AgentIsai 03099edb2 - Alphabetize [02:47:33] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.81, 3.48, 3.28 [02:47:38] miraheze/MirahezeMagic - AgentIsai the build has errored. [02:48:01] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.15, 4.67, 3.85 [02:49:33] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.10, 3.78, 3.42 [02:50:35] miraheze/MirahezeMagic - AgentIsai the build has errored. [02:53:33] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.57, 3.72, 3.51 [02:53:56] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.95, 3.54, 3.58 [02:54:26] miraheze/MirahezeMagic - AgentIsai the build has errored. [02:55:33] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.24, 3.31, 3.39 [02:55:54] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.41, 4.58, 3.95 [02:57:53] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.80, 3.85, 3.76 [02:57:56] miraheze/MirahezeMagic - AgentIsai the build has errored. [02:59:26] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.44, 10.53, 11.80 [02:59:33] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.57, 3.38, 3.44 [03:01:33] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.22, 2.93, 3.26 [03:01:46] RECOVERY - db131 Backups SQL on db131 is OK: FILE_AGE OK: /var/log/sql-backup.log is 105 seconds old and 275 bytes [03:01:50] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.81, 4.27, 3.84 [03:04:14] [02miraheze/MirahezeMagic] 07AgentIsai pushed 031 commit to 03vanishuser [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/099edb2fc82f...7a27e02f91bd [03:04:16] [02miraheze/MirahezeMagic] 07AgentIsai 037a27e02 - Use ConfigFactory [03:04:18] [02MirahezeMagic] 07AgentIsai synchronize pull request 03#421: [DMG] Testing for VanishUser - 13https://github.com/miraheze/MirahezeMagic/pull/421 [03:04:46] [02miraheze/MirahezeMagic] 07AgentIsai pushed 031 commit to 03vanishuser [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/7a27e02f91bd...6028fdcd15f2 [03:04:47] [02miraheze/MirahezeMagic] 07AgentIsai 036028fdc - Use Html [03:04:48] [02MirahezeMagic] 07AgentIsai synchronize pull request 03#421: [DMG] Testing for VanishUser - 13https://github.com/miraheze/MirahezeMagic/pull/421 [03:05:20] miraheze/MirahezeMagic - AgentIsai the build has errored. [03:05:33] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.82, 4.22, 3.72 [03:07:45] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.78, 3.38, 3.71 [03:08:26] miraheze/MirahezeMagic - AgentIsai the build has errored. [03:08:55] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 9.19, 9.34, 10.12 [03:09:33] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.20, 3.77, 3.63 [03:09:38] [02MirahezeMagic] 07AgentIsai synchronize pull request 03#421: [DMG] Testing for VanishUser - 13https://github.com/miraheze/MirahezeMagic/pull/421 [03:09:39] [02miraheze/MirahezeMagic] 07AgentIsai pushed 031 commit to 03vanishuser [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/6028fdcd15f2...c4f339d16d49 [03:09:42] [02miraheze/MirahezeMagic] 07AgentIsai 03c4f339d - use GlobalVarConfig [03:11:33] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.61, 3.05, 3.39 [03:13:41] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 3.00, 2.84, 3.35 [03:17:17] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.91, 9.27, 10.10 [03:20:08] miraheze/MirahezeMagic - AgentIsai the build passed. [03:21:17] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.14, 10.33, 10.38 [03:25:17] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.08, 9.42, 10.02 [03:47:43] [02miraheze/MirahezeMagic] 07AgentIsai pushed 031 commit to 03vanishuser [+3/-3/±2] 13https://github.com/miraheze/MirahezeMagic/compare/c4f339d16d49...02098bcfa82a [03:47:45] [02miraheze/MirahezeMagic] 07AgentIsai 0302098bc - Move special pages to folder and change hook handler [03:47:48] [02MirahezeMagic] 07AgentIsai synchronize pull request 03#421: [DMG] Testing for VanishUser - 13https://github.com/miraheze/MirahezeMagic/pull/421 [03:51:13] PROBLEM - wikiyri.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wikiyri.org' expires in 7 day(s) (Fri 12 May 2023 03:48:20 GMT +0000). [03:51:27] miraheze/MirahezeMagic - AgentIsai the build has errored. [03:51:31] PROBLEM - bn.wikiyri.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'bn.wikiyri.org' expires in 7 day(s) (Fri 12 May 2023 03:51:17 GMT +0000). [03:53:11] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.72, 4.10, 3.03 [03:59:10] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.71, 3.39, 3.15 [04:05:00] PROBLEM - hi.famepedia.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'hi.famepedia.org' expires in 7 day(s) (Fri 12 May 2023 03:46:10 GMT +0000). [04:07:09] PROBLEM - www.zetapedia.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'www.zetapedia.com' expires in 7 day(s) (Fri 12 May 2023 03:46:27 GMT +0000). [04:10:06] PROBLEM - entw.ml - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'entw.ml' expires in 7 day(s) (Fri 12 May 2023 03:46:58 GMT +0000). [04:11:46] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.171687305 secs [04:16:04] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.81, 3.93, 3.28 [04:19:34] [02MirahezeMagic] 07AgentIsai synchronize pull request 03#421: [DMG] Testing for VanishUser - 13https://github.com/miraheze/MirahezeMagic/pull/421 [04:19:36] [02miraheze/MirahezeMagic] 07AgentIsai pushed 031 commit to 03vanishuser [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/02098bcfa82a...c8f0e32b3eb7 [04:19:38] [02miraheze/MirahezeMagic] 07AgentIsai 03c8f0e32 - Update extension.json [04:23:46] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.09472748637 secs [04:27:55] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.87, 3.39, 3.56 [04:29:53] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.39, 2.66, 3.26 [04:29:55] miraheze/MirahezeMagic - AgentIsai the build passed. [04:34:10] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.70, 4.03, 2.83 [04:38:04] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.82, 3.85, 3.00 [04:41:26] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.27, 9.95, 8.54 [04:41:42] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.36, 3.56, 3.25 [04:41:57] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.86, 3.02, 2.87 [04:43:25] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.22, 8.97, 8.35 [04:43:41] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.20, 2.94, 3.05 [04:49:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.66, 10.52, 9.14 [04:49:47] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.26, 3.67, 3.18 [04:51:18] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.88, 10.05, 9.15 [04:53:40] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.46, 3.10, 3.11 [04:56:42] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 9.73, 5.18, 2.55 [04:58:26] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.95, 4.06, 3.42 [04:58:42] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 2.83, 4.41, 2.61 [04:59:33] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.52, 3.25, 3.14 [05:00:25] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.10, 3.60, 3.33 [05:01:17] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.21, 10.50, 9.56 [05:01:33] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 3.28, 3.19, 3.13 [05:02:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.63, 4.23, 3.60 [05:03:17] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.67, 10.33, 9.63 [05:04:22] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.34, 3.60, 3.44 [05:04:55] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.55, 11.09, 9.22 [05:06:20] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.16, 2.79, 3.17 [05:06:55] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.80, 10.30, 9.17 [05:07:17] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.60, 9.80, 9.60 [05:11:17] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.35, 11.43, 10.33 [05:13:17] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.21, 11.66, 10.58 [05:14:55] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.52, 9.63, 9.40 [05:16:11] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.04, 3.45, 3.08 [05:17:17] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.72, 11.54, 10.71 [05:18:09] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 3.31, 3.27, 3.05 [05:19:17] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.37, 10.48, 10.43 [05:23:17] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.98, 9.60, 10.16 [05:26:01] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.15, 3.52, 3.08 [05:27:03] PROBLEM - cp33 APT on cp33 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:28:00] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.65, 3.25, 3.04 [05:29:17] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.09, 10.19, 10.15 [05:29:17] RECOVERY - cp33 APT on cp33 is OK: APT OK: 36 packages available for upgrade (0 critical updates). [05:31:17] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.68, 10.19, 10.15 [05:33:17] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.97, 9.66, 9.95 [05:36:26] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.83, 4.31, 3.18 [05:38:23] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.10, 3.77, 3.11 [05:40:19] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.57, 3.04, 2.92 [05:42:11] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 15.31, 11.75, 10.54 [05:44:10] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.24, 10.74, 10.33 [05:50:05] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.71, 9.81, 10.09 [05:54:36] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.02, 3.16, 2.84 [05:58:33] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.24, 2.89, 2.82 [06:00:55] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.75, 10.21, 10.15 [06:06:51] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.04, 11.21, 10.56 [06:08:49] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.09, 11.18, 10.63 [06:10:48] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 14.45, 12.38, 11.14 [06:12:29] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.75, 10.07, 9.63 [06:18:13] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.73, 9.52, 9.63 [06:22:03] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.90, 11.62, 10.43 [06:23:58] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.87, 11.64, 10.59 [06:26:37] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.66, 11.93, 11.89 [06:29:49] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 3.63, 2.71, 1.52 [06:31:49] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 4.66, 3.38, 1.91 [06:32:31] PROBLEM - cp23 APT on cp23 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [06:33:31] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.25, 11.09, 10.64 [06:33:49] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 3.43, 3.97, 2.36 [06:34:31] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.71, 11.26, 11.34 [06:34:32] RECOVERY - cp23 APT on cp23 is OK: APT OK: 36 packages available for upgrade (0 critical updates). [06:35:48] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 1.86, 3.40, 2.35 [06:43:04] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.74, 11.90, 11.46 [06:47:11] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.28, 3.90, 2.75 [06:49:10] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.97, 3.14, 2.61 [06:58:55] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 14.98, 12.02, 11.36 [07:01:36] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.90, 3.51, 2.93 [07:04:09] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.40, 10.77, 11.92 [07:04:55] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.52, 11.86, 11.53 [07:07:20] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.71, 3.26, 3.03 [07:10:05] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 16.69, 12.51, 12.19 [07:11:13] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.70, 3.56, 3.20 [07:14:02] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.21, 11.69, 12.00 [07:17:13] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.06, 2.91, 3.07 [07:19:58] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.70, 12.06, 11.98 [07:20:55] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.12, 11.35, 11.10 [07:22:55] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.63, 10.98, 10.99 [07:24:55] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.41, 11.38, 11.13 [07:26:55] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.07, 11.01, 11.03 [07:28:55] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.08, 11.40, 11.17 [07:48:46] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query en.religiononfire.mar.in.ua. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [08:00:55] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.50, 10.37, 11.76 [08:02:55] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.39, 11.02, 11.79 [08:06:55] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.23, 10.97, 11.63 [08:09:22] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.09, 10.79, 11.95 [08:11:21] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 14.80, 11.81, 12.14 [08:12:55] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.88, 11.54, 11.53 [08:13:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.56, 11.42, 11.98 [08:14:55] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.70, 11.14, 11.41 [08:16:55] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.37, 11.56, 11.53 [08:17:17] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.94, 12.14, 12.12 [08:17:33] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [08:22:55] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.00, 11.23, 11.40 [08:23:17] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.99, 11.65, 11.91 [08:32:55] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.43, 10.97, 11.01 [08:34:55] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.35, 11.00, 11.02 [08:35:17] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.66, 10.95, 11.02 [08:37:17] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.34, 10.83, 10.98 [08:41:17] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 17.22, 13.15, 11.80 [08:48:55] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.83, 11.39, 11.04 [09:00:55] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.21, 10.92, 11.56 [09:07:17] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.20, 10.36, 11.63 [09:10:55] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.37, 8.66, 10.14 [09:11:55] PROBLEM - www.lab612.at - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - www.lab612.at All nameservers failed to answer the query. [09:17:17] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.35, 8.36, 10.07 [09:36:55] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.97, 10.84, 9.79 [09:38:04] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.62, 9.79, 9.56 [09:40:02] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.11, 10.67, 9.92 [09:40:55] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.55, 9.13, 9.37 [09:41:21] RECOVERY - www.lab612.at - reverse DNS on sslhost is OK: SSL OK - www.lab612.at reverse DNS resolves to cp23.miraheze.org - CNAME OK [09:44:00] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.46, 10.99, 10.25 [09:49:55] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 9.20, 9.81, 9.98 [10:06:56] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1120895147 secs [10:13:35] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.18, 10.24, 9.72 [10:15:34] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.21, 10.75, 9.95 [10:17:32] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.16, 9.32, 9.53 [10:24:25] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.08, 11.19, 10.32 [10:26:24] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.60, 10.74, 10.27 [10:28:56] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.07864436507 secs [10:32:55] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.68, 10.48, 9.95 [10:34:18] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.07, 10.93, 10.50 [10:34:55] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.17, 10.92, 10.14 [10:36:17] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.28, 11.18, 10.69 [10:36:55] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.72, 11.15, 10.32 [10:38:42] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 10.44, 6.96, 3.68 [10:38:55] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 14.54, 12.22, 10.81 [10:40:14] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 17.45, 13.60, 11.71 [10:40:42] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 2.39, 5.10, 3.40 [10:43:37] PROBLEM - cloud10 Puppet on cloud10 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [11:02:55] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.97, 10.83, 11.74 [11:04:55] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.09, 12.02, 12.08 [11:06:55] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.72, 11.32, 11.83 [11:09:37] RECOVERY - cloud10 Puppet on cloud10 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [11:10:55] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.90, 12.51, 12.21 [11:14:55] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.22, 11.76, 12.00 [11:16:55] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.98, 12.04, 12.05 [11:17:17] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query en.religiononfire.mar.in.ua. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [11:17:24] PROBLEM - cp33 APT on cp33 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [11:19:50] RECOVERY - cp33 APT on cp33 is OK: APT OK: 36 packages available for upgrade (0 critical updates). [11:28:42] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 10.18, 5.64, 2.92 [11:30:42] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 1.86, 3.98, 2.64 [11:30:55] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.32, 11.40, 11.91 [11:32:55] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.37, 12.33, 12.20 [12:11:13] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.67, 3.05, 2.65 [12:13:13] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.11, 3.47, 2.85 [12:15:13] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.70, 3.60, 2.99 [12:15:17] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [12:29:13] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.03, 2.91, 3.06 [12:39:19] PROBLEM - cp33 APT on cp33 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [12:39:32] PROBLEM - os141 Current Load on os141 is CRITICAL: CRITICAL - load average: 4.57, 2.85, 1.50 [12:40:42] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 8.79, 7.62, 4.75 [12:41:32] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 2.51, 2.85, 1.68 [12:41:37] RECOVERY - cp33 APT on cp33 is OK: APT OK: 36 packages available for upgrade (0 critical updates). [12:42:42] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 2.87, 6.10, 4.57 [12:48:42] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 6.91, 6.16, 4.95 [12:50:03] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.86, 6.55, 5.86 [12:50:42] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 3.21, 5.35, 4.83 [12:52:03] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.49, 6.02, 5.75 [12:53:57] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.46, 4.04, 3.01 [13:03:49] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.28, 3.34, 3.40 [13:17:33] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.47, 3.50, 2.57 [13:19:33] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.71, 2.80, 2.43 [13:39:17] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.39, 3.35, 3.02 [13:41:15] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.00, 2.93, 2.92 [13:49:11] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.10, 3.50, 3.09 [13:51:11] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.88, 2.76, 2.86 [13:57:10] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.88, 3.84, 3.34 [13:57:46] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.149705857 secs [14:01:11] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.59, 4.93, 3.85 [14:03:10] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.79, 3.62, 3.50 [14:05:10] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.16, 3.87, 3.60 [14:07:11] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.80, 3.26, 3.39 [14:12:20] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.15, 3.42, 3.06 [14:12:33] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: CRITICAL - load average: 4.22, 3.44, 2.84 [14:13:19] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:14:17] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.92, 3.94, 3.28 [14:14:31] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.77, 3.52, 2.95 [14:16:13] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.59, 3.45, 3.18 [14:16:29] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 2.30, 3.08, 2.85 [14:18:10] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.20, 3.02, 3.06 [14:19:31] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 5.160 second response time [14:19:46] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset -0.008758336306 secs [14:20:03] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.49, 3.88, 3.47 [14:22:41] [02MirahezeMagic] 07AgentIsai synchronize pull request 03#421: [DMG] Testing for VanishUser - 13https://github.com/miraheze/MirahezeMagic/pull/421 [14:22:43] [02miraheze/MirahezeMagic] 07AgentIsai pushed 031 commit to 03vanishuser [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/c8f0e32b3eb7...61b94d0f8135 [14:22:46] [02miraheze/MirahezeMagic] 07AgentIsai 0361b94d0 - Remove VanishUser mention [14:23:11] [02MirahezeMagic] 07AgentIsai synchronize pull request 03#421: [DMG] Testing for VanishUser - 13https://github.com/miraheze/MirahezeMagic/pull/421 [14:23:14] [02miraheze/MirahezeMagic] 07AgentIsai pushed 031 commit to 03vanishuser [+0/-1/±0] 13https://github.com/miraheze/MirahezeMagic/compare/61b94d0f8135...311adff2cac5 [14:23:16] [02miraheze/MirahezeMagic] 07AgentIsai 03311adff - Delete SpecialVanishUser.php [14:24:00] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.93, 3.89, 3.51 [14:24:53] [02MirahezeMagic] 07AgentIsai synchronize pull request 03#421: [DMG] Testing for VanishUser - 13https://github.com/miraheze/MirahezeMagic/pull/421 [14:24:55] [02miraheze/MirahezeMagic] 07AgentIsai pushed 031 commit to 03vanishuser [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/311adff2cac5...56ee7764808e [14:24:57] [02miraheze/MirahezeMagic] 07AgentIsai 0356ee776 - Remove VanishUser [14:26:03] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.19, 6.66, 6.11 [14:27:18] [02miraheze/MirahezeMagic] 07AgentIsai pushed 031 commit to 03vanishuser [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/56ee7764808e...ad689fda52bc [14:27:21] [02miraheze/MirahezeMagic] 07AgentIsai 03ad689fd - Namespace [14:27:23] [02MirahezeMagic] 07AgentIsai synchronize pull request 03#421: [DMG] Testing for VanishUser - 13https://github.com/miraheze/MirahezeMagic/pull/421 [14:27:32] [02miraheze/MirahezeMagic] 07AgentIsai pushed 031 commit to 03vanishuser [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/ad689fda52bc...adcaafc05059 [14:27:34] [02miraheze/MirahezeMagic] 07AgentIsai 03adcaafc - Namespace [14:27:37] [02MirahezeMagic] 07AgentIsai synchronize pull request 03#421: [DMG] Testing for VanishUser - 13https://github.com/miraheze/MirahezeMagic/pull/421 [14:27:44] [02miraheze/MirahezeMagic] 07AgentIsai pushed 031 commit to 03vanishuser [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/adcaafc05059...bbac3cff05fb [14:27:47] [02miraheze/MirahezeMagic] 07AgentIsai 03bbac3cf - Namespace [14:27:49] [02MirahezeMagic] 07AgentIsai synchronize pull request 03#421: [DMG] Testing for VanishUser - 13https://github.com/miraheze/MirahezeMagic/pull/421 [14:28:03] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.13, 6.37, 6.06 [14:28:05] [02miraheze/MirahezeMagic] 07AgentIsai pushed 031 commit to 03vanishuser [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/bbac3cff05fb...2c29a216c203 [14:28:07] PROBLEM - cp33 APT on cp33 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [14:28:07] [02miraheze/MirahezeMagic] 07AgentIsai 032c29a21 - Namespace [14:28:10] [02MirahezeMagic] 07AgentIsai synchronize pull request 03#421: [DMG] Testing for VanishUser - 13https://github.com/miraheze/MirahezeMagic/pull/421 [14:28:16] [02miraheze/MirahezeMagic] 07AgentIsai pushed 031 commit to 03vanishuser [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/2c29a216c203...697b295379c4 [14:28:19] [02miraheze/MirahezeMagic] 07AgentIsai 03697b295 - Namespace [14:28:21] [02MirahezeMagic] 07AgentIsai synchronize pull request 03#421: [DMG] Testing for VanishUser - 13https://github.com/miraheze/MirahezeMagic/pull/421 [14:28:27] [02miraheze/MirahezeMagic] 07AgentIsai pushed 031 commit to 03vanishuser [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/697b295379c4...3cdbb5174ee5 [14:28:28] [02miraheze/MirahezeMagic] 07AgentIsai 033cdbb51 - Namespace [14:28:30] [02MirahezeMagic] 07AgentIsai synchronize pull request 03#421: [DMG] Testing for VanishUser - 13https://github.com/miraheze/MirahezeMagic/pull/421 [14:29:07] [02miraheze/MirahezeMagic] 07AgentIsai pushed 031 commit to 03vanishuser [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/3cdbb5174ee5...4e5907e7d26b [14:29:10] [02miraheze/MirahezeMagic] 07AgentIsai 034e5907e - NS [14:29:11] [02MirahezeMagic] 07AgentIsai synchronize pull request 03#421: [DMG] Testing for VanishUser - 13https://github.com/miraheze/MirahezeMagic/pull/421 [14:29:13] @Agent if you could review https://github.com/AgentIsai/Share/pull/5 that'd be great [14:29:16] [02miraheze/MirahezeMagic] 07AgentIsai pushed 031 commit to 03vanishuser [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/4e5907e7d26b...6160f208ce0d [14:29:19] [02miraheze/MirahezeMagic] 07AgentIsai 036160f20 - NS [14:29:22] [02MirahezeMagic] 07AgentIsai synchronize pull request 03#421: [DMG] Testing for VanishUser - 13https://github.com/miraheze/MirahezeMagic/pull/421 [14:29:27] [02miraheze/MirahezeMagic] 07AgentIsai pushed 031 commit to 03vanishuser [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/6160f208ce0d...aebdfe0e0a77 [14:29:29] [02miraheze/MirahezeMagic] 07AgentIsai 03aebdfe0 - NS [14:29:31] [02MirahezeMagic] 07AgentIsai synchronize pull request 03#421: [DMG] Testing for VanishUser - 13https://github.com/miraheze/MirahezeMagic/pull/421 [14:29:36] [02miraheze/MirahezeMagic] 07AgentIsai pushed 031 commit to 03vanishuser [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/aebdfe0e0a77...bdc2e744ab45 [14:29:37] [02miraheze/MirahezeMagic] 07AgentIsai 03bdc2e74 - NS [14:29:39] [02MirahezeMagic] 07AgentIsai synchronize pull request 03#421: [DMG] Testing for VanishUser - 13https://github.com/miraheze/MirahezeMagic/pull/421 [14:29:44] [02miraheze/MirahezeMagic] 07AgentIsai pushed 031 commit to 03vanishuser [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/bdc2e744ab45...6f6265c9c3dd [14:29:47] [02miraheze/MirahezeMagic] 07AgentIsai 036f6265c - NS [14:29:50] [02MirahezeMagic] 07AgentIsai synchronize pull request 03#421: [DMG] Testing for VanishUser - 13https://github.com/miraheze/MirahezeMagic/pull/421 [14:29:54] [02miraheze/MirahezeMagic] 07AgentIsai pushed 031 commit to 03vanishuser [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/6f6265c9c3dd...1f41919a6aca [14:29:56] [02miraheze/MirahezeMagic] 07AgentIsai 031f41919 - NS [14:29:57] [02MirahezeMagic] 07AgentIsai synchronize pull request 03#421: [DMG] Testing for VanishUser - 13https://github.com/miraheze/MirahezeMagic/pull/421 [14:30:09] RECOVERY - cp33 APT on cp33 is OK: APT OK: 36 packages available for upgrade (0 critical updates). [14:30:22] [02miraheze/MirahezeMagic] 07AgentIsai pushed 031 commit to 03vanishuser [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/1f41919a6aca...5173a8f179a5 [14:30:25] [02miraheze/MirahezeMagic] 07AgentIsai 035173a8f - NS [14:30:26] [02MirahezeMagic] 07AgentIsai synchronize pull request 03#421: [DMG] Testing for VanishUser - 13https://github.com/miraheze/MirahezeMagic/pull/421 [14:30:40] [02miraheze/MirahezeMagic] 07AgentIsai pushed 031 commit to 03vanishuser [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/5173a8f179a5...4667c2ef941b [14:30:42] [02miraheze/MirahezeMagic] 07AgentIsai 034667c2e - Namespace [14:30:45] [02MirahezeMagic] 07AgentIsai synchronize pull request 03#421: [DMG] Testing for VanishUser - 13https://github.com/miraheze/MirahezeMagic/pull/421 [14:30:49] [02miraheze/MirahezeMagic] 07AgentIsai pushed 031 commit to 03vanishuser [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/4667c2ef941b...6b8e163a4512 [14:30:50] [02miraheze/MirahezeMagic] 07AgentIsai 036b8e163 - Namespace [14:30:51] [02MirahezeMagic] 07AgentIsai synchronize pull request 03#421: [DMG] Testing for VanishUser - 13https://github.com/miraheze/MirahezeMagic/pull/421 [14:31:50] [02miraheze/MirahezeMagic] 07AgentIsai pushed 031 commit to 03vanishuser [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/6b8e163a4512...e341285a1c5c [14:31:51] [02miraheze/MirahezeMagic] 07AgentIsai 03e341285 - Add maintenance namespace [14:31:54] [02MirahezeMagic] 07AgentIsai synchronize pull request 03#421: [DMG] Testing for VanishUser - 13https://github.com/miraheze/MirahezeMagic/pull/421 [14:31:54] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.26, 3.92, 3.85 [14:34:42] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 7.88, 5.60, 3.49 [14:36:42] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 8.21, 7.02, 4.30 [14:37:49] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.92, 2.87, 3.38 [14:38:03] [02MirahezeMagic] 07AgentIsai edited pull request 03#421: Namespace MirahezeMagic - 13https://github.com/miraheze/MirahezeMagic/pull/421 [14:38:25] [02MirahezeMagic] 07AgentIsai edited pull request 03#421: Namespace MirahezeMagic - 13https://github.com/miraheze/MirahezeMagic/pull/421 [14:38:42] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 3.30, 5.62, 4.12 [14:39:44] miraheze/MirahezeMagic - AgentIsai the build passed. [14:41:54] Orange_Star: my apologies, I didn't realise you sent in that PR [14:43:03] merged [14:44:46] [02MirahezeMagic] 07AgentIsai synchronize pull request 03#421: Namespace MirahezeMagic - 13https://github.com/miraheze/MirahezeMagic/pull/421 [14:44:48] [02miraheze/MirahezeMagic] 07AgentIsai pushed 031 commit to 03vanishuser [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/e341285a1c5c...48389c009f6c [14:44:50] [02miraheze/MirahezeMagic] 07AgentIsai 0348389c0 - Remove VanishUser mention [14:48:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 4.00, 3.55, 3.07 [14:50:00] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:50:24] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.05, 4.20, 3.36 [14:53:04] miraheze/MirahezeMagic - AgentIsai the build passed. [14:54:11] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 9.223 second response time [14:55:33] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.47, 4.29, 3.47 [14:56:05] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [14:56:14] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.69, 3.65, 3.44 [15:07:24] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.32, 3.83, 3.92 [15:11:48] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.00, 2.91, 3.27 [15:15:18] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.35, 3.68, 3.75 [15:17:57] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:19:15] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.26, 3.75, 3.84 [15:19:57] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 4.918 second response time [15:23:12] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 8.88, 5.32, 4.34 [15:23:13] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.42, 6.70, 6.27 [15:25:10] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.30, 6.63, 6.28 [15:27:07] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.71, 6.43, 6.25 [15:40:19] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.42, 3.37, 3.05 [15:41:10] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.31, 2.77, 3.79 [15:42:16] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.35, 2.71, 2.85 [15:45:11] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.22, 3.76, 3.93 [15:46:12] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 9.70, 4.54, 3.46 [15:51:11] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.32, 3.81, 3.93 [15:52:42] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 8.18, 5.93, 3.57 [15:54:42] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 7.33, 6.50, 4.08 [15:55:56] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.11, 3.66, 3.60 [15:56:42] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 5.98, 6.48, 4.38 [15:57:11] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.37, 4.16, 3.94 [15:59:49] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.17, 3.83, 3.67 [16:01:11] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.14, 3.36, 3.68 [16:01:46] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.84, 3.40, 3.53 [16:02:01] PROBLEM - wiki.hylormc.tk - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.hylormc.tk' expires in 7 day(s) (Fri 12 May 2023 15:45:37 GMT +0000). [16:03:43] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.13, 4.64, 3.97 [16:05:11] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.76, 2.78, 3.39 [16:05:18] PROBLEM - en.wikiyri.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'en.wikiyri.org' expires in 7 day(s) (Fri 12 May 2023 15:47:14 GMT +0000). [16:05:40] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.16, 3.69, 3.70 [16:11:33] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.36, 2.79, 3.30 [16:20:18] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:25:02] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.02, 3.86, 3.45 [16:30:58] PROBLEM - os141 Current Load on os141 is WARNING: WARNING - load average: 3.63, 3.28, 2.64 [16:32:50] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 6.051 second response time [16:32:54] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 1.05, 2.55, 2.45 [16:32:55] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.51, 3.65, 3.77 [16:33:33] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.61, 3.94, 3.07 [16:34:54] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 7.47, 4.59, 4.06 [16:35:33] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.90, 3.48, 3.01 [16:36:52] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.77, 3.78, 3.83 [16:37:33] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.52, 2.80, 2.82 [16:40:28] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:40:49] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.98, 3.52, 3.64 [16:42:48] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.45, 3.48, 3.62 [16:43:33] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.15, 3.83, 3.22 [16:46:45] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.41, 2.59, 3.24 [16:51:33] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.19, 3.65, 3.53 [16:55:33] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.01, 2.93, 3.28 [16:56:00] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [17:03:31] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 7.044 second response time [17:03:33] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 3.39, 4.37, 3.80 [17:07:33] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.39, 3.51, 3.59 [17:09:33] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.63, 2.84, 3.33 [17:13:33] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.08, 4.12, 3.75 [17:13:44] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:15:33] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.23, 3.42, 3.54 [17:19:33] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 3.21, 3.10, 3.37 [17:20:01] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 9.278 second response time [17:27:30] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [17:42:02] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:44:30] PROBLEM - cp33 APT on cp33 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [17:46:13] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 9.525 second response time [17:46:38] RECOVERY - cp33 APT on cp33 is OK: APT OK: 36 packages available for upgrade (0 critical updates). [17:57:43] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.97, 4.30, 3.20 [18:01:40] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.17, 3.40, 3.12 [18:09:31] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.45, 3.97, 3.53 [18:11:30] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.70, 4.45, 3.74 [18:13:06] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:13:18] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.81, 5.94, 5.52 [18:13:28] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.87, 3.83, 3.59 [18:15:15] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.76, 5.81, 5.55 [18:15:27] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.51, 4.22, 3.77 [18:17:25] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.15, 3.49, 3.57 [18:17:33] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.61, 3.50, 2.75 [18:18:56] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1442681551 secs [18:19:24] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.66, 4.17, 3.81 [18:19:33] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.74, 2.72, 2.55 [18:21:03] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.62, 7.01, 6.08 [18:23:00] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.79, 6.78, 6.10 [18:27:33] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.55, 3.79, 3.01 [18:28:56] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.0771574378 secs [18:31:33] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.90, 3.95, 3.34 [18:33:33] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.75, 4.63, 3.65 [18:34:01] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 7.573 second response time [18:37:10] PROBLEM - mw122 MediaWiki Rendering on mw122 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:37:11] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.16, 2.73, 3.81 [18:39:12] RECOVERY - mw122 MediaWiki Rendering on mw122 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 3.592 second response time [18:41:33] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.14, 3.42, 3.55 [18:42:25] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:43:10] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.49, 2.07, 3.18 [18:43:33] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.76, 2.83, 3.31 [18:54:57] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 7.637 second response time [18:56:08] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 7.90, 4.51, 3.50 [19:00:04] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.92, 3.94, 3.52 [19:02:03] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.05, 3.24, 3.30 [19:06:29] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.82, 3.62, 3.04 [19:08:56] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.2065476775 secs [19:10:22] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 8.75, 5.67, 3.92 [19:10:42] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.20, 6.24, 5.60 [19:12:39] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.27, 5.51, 5.41 [19:13:57] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:16:00] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 7.880 second response time [19:16:13] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.40, 3.85, 3.65 [19:18:09] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.82, 4.77, 4.00 [19:20:06] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.42, 3.81, 3.74 [19:20:24] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:21:40] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.45, 2.84, 2.32 [19:22:03] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.22, 4.00, 3.81 [19:23:40] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 2.98, 2.88, 2.40 [19:24:00] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.67, 3.94, 3.82 [19:25:42] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.62, 3.95, 3.40 [19:29:50] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.07, 3.40, 3.59 [19:30:54] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 9.765 second response time [19:31:38] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.79, 3.05, 3.22 [19:31:47] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.93, 3.16, 3.48 [19:33:44] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.43, 2.93, 3.36 [19:36:56] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.03931170702 secs [19:37:40] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.57, 3.21, 3.43 [19:39:37] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.17, 3.05, 3.36 [19:48:56] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset -0.1322203279 secs [19:49:26] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.96, 4.67, 3.58 [19:51:24] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.80, 3.87, 3.42 [19:52:56] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset -0.08202248812 secs [19:53:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.20, 3.88, 3.46 [20:05:14] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.69, 3.88, 3.84 [20:11:10] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.94, 2.70, 3.30 [20:16:02] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.93, 4.40, 3.55 [20:16:56] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1035842896 secs [20:17:59] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.79, 3.71, 3.40 [20:19:56] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.36, 4.12, 3.58 [20:22:56] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.09772953391 secs [20:24:46] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.21, 7.29, 6.26 [20:26:43] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.79, 7.18, 6.39 [20:27:10] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.89, 3.87, 3.26 [20:28:40] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.50, 6.58, 6.29 [20:29:10] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.30, 3.53, 3.21 [20:29:46] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - uk.religiononfire.mar.in.ua All nameservers failed to answer the query. [20:31:11] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 7.28, 4.04, 3.38 [20:33:33] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.87, 3.63, 3.76 [20:35:11] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.42, 3.47, 3.35 [20:37:11] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.34, 4.17, 3.60 [20:39:11] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.74, 3.22, 3.33 [20:39:33] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.59, 2.81, 3.35 [20:42:30] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [20:43:46] PROBLEM - wiki.yermocapital.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.yermocapital.com' expires in 7 day(s) (Fri 12 May 2023 20:26:44 GMT +0000). [20:49:10] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.36, 3.81, 3.45 [20:51:11] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.04, 3.71, 3.44 [20:53:11] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.99, 3.04, 3.22 [20:54:19] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.27, 4.31, 3.59 [20:57:11] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.33, 3.43, 3.34 [20:58:12] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.07, 3.53, 3.46 [20:59:10] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.08, 3.13, 3.25 [20:59:25] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [21:02:06] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.25, 3.75, 3.56 [21:09:53] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.68, 4.00, 3.85 [21:10:02] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:11:49] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.38, 5.00, 4.23 [21:12:07] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 9.101 second response time [21:15:43] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.53, 3.67, 3.84 [21:16:06] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 3.84, 4.02, 3.61 [21:19:37] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.20, 2.68, 3.38 [21:20:04] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.51, 3.74, 3.62 [21:24:01] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.35, 3.84, 3.68 [21:26:00] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [21:26:34] that alert has been going on for a few days now [21:39:49] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.18, 3.93, 3.99 [21:41:00] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [21:41:47] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.09, 4.26, 4.11 [21:45:44] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.57, 3.68, 3.94 [21:53:38] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.87, 3.72, 3.71 [21:57:35] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.44, 3.39, 3.60 [21:58:44] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - uk.religiononfire.mar.in.ua All nameservers failed to answer the query. [21:59:33] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.35, 2.78, 3.37 [22:01:42] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.20, 4.27, 3.47 [22:07:33] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.05, 3.91, 3.65 [22:13:33] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.97, 2.94, 3.31 [22:28:07] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.05, 3.73, 3.26 [22:28:23] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [22:29:35] [02ssl] 07MacFan4000 created branch 03MacFan4000-patch-1 - 13https://github.com/miraheze/ssl [22:29:37] [02miraheze/ssl] 07MacFan4000 pushed 031 commit to 03MacFan4000-patch-1 [+0/-1/±0] 13https://github.com/miraheze/ssl/commit/c961bcc7c851 [22:29:40] [02miraheze/ssl] 07MacFan4000 03c961bcc - Delete wiki.erag.eu.org.crt [22:29:45] [02ssl] 07MacFan4000 opened pull request 03#648: Remove wiki.erag.eu.org - 13https://github.com/miraheze/ssl/pull/648 [22:30:05] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.83, 2.95, 3.03 [22:30:36] [02miraheze/ssl] 07MacFan4000 pushed 031 commit to 03MacFan4000-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/c961bcc7c851...c45a9d299d05 [22:30:38] [02miraheze/ssl] 07MacFan4000 03c45a9d2 - Update certs.yaml [22:30:39] [02ssl] 07MacFan4000 synchronize pull request 03#648: Remove wiki.erag.eu.org - 13https://github.com/miraheze/ssl/pull/648 [22:32:34] [02ssl] 07MacFan4000 closed pull request 03#648: Remove wiki.erag.eu.org - 13https://github.com/miraheze/ssl/pull/648 [22:32:36] [02miraheze/ssl] 07MacFan4000 deleted branch 03MacFan4000-patch-1 [22:32:39] [02ssl] 07MacFan4000 deleted branch 03MacFan4000-patch-1 - 13https://github.com/miraheze/ssl [22:32:40] [02miraheze/ssl] 07MacFan4000 pushed 031 commit to 03master [+0/-1/±1] 13https://github.com/miraheze/ssl/compare/a44d1d41a69d...0127d132cfd0 [22:32:42] [02miraheze/ssl] 07MacFan4000 030127d13 - Remove wiki.erag.eu.org (#648) [22:37:53] PROBLEM - mw122 Puppet on mw122 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/etc/ssl/private/wiki.erag.eu.org.key] [22:39:02] PROBLEM - mw132 Puppet on mw132 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/etc/ssl/private/wiki.erag.eu.org.key] [22:39:34] PROBLEM - mw141 Puppet on mw141 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/etc/ssl/private/wiki.erag.eu.org.key] [22:40:56] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset -0.1027873755 secs [22:42:22] PROBLEM - cp32 Puppet on cp32 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/etc/ssl/private/wiki.erag.eu.org.key] [22:42:56] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset -0.0824252069 secs [22:44:39] PROBLEM - mwtask141 Puppet on mwtask141 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 3 minutes ago with 2 failures. Failed resources (up to 3 shown): File[/etc/ssl/localcerts/wiki.erag.eu.org.crt],File[/etc/ssl/private/wiki.erag.eu.org.key] [22:46:55] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.32, 9.89, 11.80 [22:50:13] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.66, 3.57, 3.21 [22:52:10] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 3.04, 3.21, 3.11 [22:54:39] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.92, 11.97, 11.90 [22:56:04] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 3.40, 4.35, 3.70 [22:56:33] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.51, 11.62, 11.80 [22:58:27] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 14.28, 12.54, 12.10 [23:00:04] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.73, 3.65, 3.59 [23:00:08] [02mw-config] 07Amanda-Catherine opened pull request 03#5224: nycsubwaywiki: condense all restricted user groups into one - 13https://github.com/miraheze/mw-config/pull/5224 [23:00:20] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.47, 11.90, 11.91 [23:00:57] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.2962132394 secs [23:01:09] miraheze/mw-config - Amanda-Catherine the build passed. [23:02:14] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.52, 12.66, 12.22 [23:05:44] RECOVERY - mw122 Puppet on mw122 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [23:06:02] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.77, 11.49, 11.86 [23:06:04] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.16, 3.29, 3.41 [23:06:16] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.53, 10.31, 11.67 [23:07:07] RECOVERY - mw132 Puppet on mw132 is OK: OK: Puppet is currently enabled, last run 31 seconds ago with 0 failures [23:07:38] RECOVERY - mw141 Puppet on mw141 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [23:08:04] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.45, 3.08, 3.32 [23:08:10] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.87, 11.72, 12.03 [23:09:20] [02mw-config] 07Amanda-Catherine synchronize pull request 03#5224: nycsubwaywiki: condense all restricted user groups into one - 13https://github.com/miraheze/mw-config/pull/5224 [23:09:42] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.99, 3.82, 3.09 [23:09:53] RECOVERY - cp32 Puppet on cp32 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [23:10:05] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.80, 11.68, 11.99 [23:10:20] miraheze/mw-config - Amanda-Catherine the build passed. [23:11:37] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.45, 3.25, 2.96 [23:12:37] RECOVERY - mwtask141 Puppet on mwtask141 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [23:13:38] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.95, 11.08, 11.47 [23:13:53] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.50, 12.12, 12.05 [23:14:57] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.03836500645 secs [23:15:31] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.10, 10.22, 11.11 [23:16:58] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.85, 3.57, 3.45 [23:17:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.69, 11.86, 11.98 [23:18:56] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.37, 3.76, 3.52 [23:20:54] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.76, 3.16, 3.34 [23:23:07] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.40, 8.16, 9.83 [23:27:14] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.52, 11.52, 11.60 [23:29:08] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.68, 10.70, 11.29 [23:40:11] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.nj.cn.eu.org All nameservers failed to answer the query. [23:43:00] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.46, 11.80, 11.23 [23:45:00] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.62, 11.30, 11.12 [23:49:01] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 15.20, 11.94, 11.32 [23:51:43] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 14.37, 12.77, 10.93 [23:59:18] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.81, 11.70, 11.19