[00:01:54] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.51, 10.77, 10.60 [00:02:59] PROBLEM - cp33 APT on cp33 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [00:02:59] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.16, 7.06, 6.67 [00:03:25] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.13, 3.46, 3.15 [00:04:54] RECOVERY - cp33 APT on cp33 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [00:06:49] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.38, 10.15, 10.48 [00:09:17] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.93, 3.20, 3.14 [00:10:43] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.78, 10.55, 10.48 [00:10:48] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.12, 6.53, 6.60 [00:16:06] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.58, 10.90, 10.00 [00:18:06] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 16.28, 12.53, 10.68 [00:19:47] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.19, 3.10, 4.00 [00:20:40] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.81, 7.50, 6.92 [00:23:46] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.77, 4.48, 4.30 [00:24:44] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.religiononfire.mar.in.ua. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [00:24:58] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.24, 4.46, 3.69 [00:26:03] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 9.33, 6.08, 3.53 [00:26:57] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.94, 3.82, 3.56 [00:28:04] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 3.93, 5.65, 3.73 [00:28:52] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.16, 3.86, 3.60 [00:30:49] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.28, 3.60, 3.53 [00:31:49] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.37, 3.14, 3.77 [00:32:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.78, 7.86, 7.60 [00:32:46] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.23, 3.10, 3.35 [00:35:45] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.1905437112 secs [00:36:05] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 10.23, 6.70, 4.59 [00:36:46] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.74, 8.73, 7.97 [00:39:39] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.18, 3.37, 3.42 [00:39:46] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.54, 2.83, 3.39 [00:40:40] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.59, 7.41, 7.64 [00:41:36] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.61, 3.65, 3.50 [00:43:34] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.91, 3.33, 3.40 [00:44:48] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.27, 7.59, 7.65 [00:46:03] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 3.91, 6.89, 6.30 [00:46:40] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.49, 7.19, 7.50 [00:48:03] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 1.64, 5.00, 5.68 [00:50:40] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.54, 7.71, 7.59 [00:52:41] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.59, 7.62, 7.57 [00:53:28] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [00:54:37] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.09, 4.19, 3.49 [00:54:41] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.44, 8.73, 8.00 [00:56:04] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 11.51, 8.31, 6.63 [00:56:40] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.53, 7.78, 7.74 [01:00:17] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 1.71, 5.86, 6.12 [01:01:14] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 4.12, 2.73, 1.54 [01:02:09] PROBLEM - cp23 APT on cp23 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:02:35] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.40, 3.85, 3.77 [01:02:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.49, 7.68, 7.58 [01:03:14] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 1.38, 2.31, 1.54 [01:03:53] RECOVERY - cp23 APT on cp23 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [01:04:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 3.97, 4.09, 3.88 [01:04:43] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.12, 7.15, 7.39 [01:08:07] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.90, 4.01, 3.41 [01:08:20] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.10, 3.94, 3.87 [01:09:45] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset 0.0579496026 secs [01:12:15] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.76, 2.72, 3.38 [01:15:30] PROBLEM - os131 Disk Space on os131 is WARNING: DISK WARNING - free space: / 21877 MB (10% inode=99%); [01:16:22] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.97, 3.87, 3.64 [01:19:27] RECOVERY - os131 Disk Space on os131 is OK: DISK OK - free space: / 22191 MB (11% inode=99%); [01:19:44] PROBLEM - cp33 APT on cp33 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:22:07] RECOVERY - cp33 APT on cp33 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [01:24:43] PROBLEM - os141 Disk Space on os141 is WARNING: DISK WARNING - free space: / 20987 MB (10% inode=99%); [01:26:48] RECOVERY - os141 Disk Space on os141 is OK: DISK OK - free space: / 22283 MB (11% inode=99%); [01:32:43] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.35, 6.38, 6.74 [01:41:16] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.92, 3.04, 3.76 [01:41:27] PROBLEM - os131 Disk Space on os131 is WARNING: DISK WARNING - free space: / 21841 MB (10% inode=99%); [01:41:32] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.07, 6.92, 6.89 [01:43:30] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.35, 7.84, 7.25 [01:43:46] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.08, 3.29, 3.95 [01:45:52] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.04, 7.46, 7.20 [01:47:34] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.21, 7.80, 7.35 [01:47:47] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.19, 3.51, 3.86 [01:49:12] PROBLEM - os141 Disk Space on os141 is WARNING: DISK WARNING - free space: / 21904 MB (10% inode=99%); [01:49:49] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.89, 3.55, 3.83 [01:51:01] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.64, 2.46, 3.21 [01:51:48] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.96, 3.99, 3.93 [01:59:34] RECOVERY - os131 Disk Space on os131 is OK: DISK OK - free space: / 22025 MB (11% inode=99%); [02:01:28] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.41, 7.97, 7.96 [02:03:27] PROBLEM - os131 Disk Space on os131 is WARNING: DISK WARNING - free space: / 21885 MB (10% inode=99%); [02:05:16] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.32, 8.40, 8.10 [02:06:02] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.70, 2.89, 3.77 [02:11:07] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.63, 7.93, 7.99 [02:12:58] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.92, 8.96, 8.38 [02:13:58] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.26, 2.66, 3.35 [02:18:07] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.26, 10.80, 11.85 [02:26:22] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 8.01, 6.08, 3.54 [02:28:14] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 3.44, 4.98, 3.45 [02:33:06] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.70, 3.32, 2.98 [02:35:03] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.37, 2.92, 2.87 [02:38:52] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.21, 10.78, 11.76 [02:39:46] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.74, 3.82, 3.03 [02:42:47] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.90, 11.28, 11.70 [02:43:51] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.51, 3.62, 3.13 [02:44:14] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.72, 9.25, 10.07 [02:44:50] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.67, 10.97, 11.54 [02:45:49] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.41, 3.74, 3.22 [02:49:55] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.49, 3.87, 3.43 [02:56:47] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.01, 3.61, 3.17 [02:56:49] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.60, 8.69, 10.03 [02:57:58] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.05, 3.72, 3.50 [02:58:44] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.25, 3.59, 3.23 [02:59:49] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.55, 3.26, 3.36 [03:00:40] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 3.08, 3.31, 3.16 [03:01:32] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1094450951 secs [03:04:33] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 13.66, 12.21, 7.11 [03:04:43] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 7.47, 4.86, 3.74 [03:10:36] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.12, 3.62, 3.58 [03:11:28] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.07798734307 secs [03:12:33] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.06, 3.09, 3.38 [03:12:46] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.49, 7.52, 7.93 [03:12:46] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.40, 10.42, 9.84 [03:14:40] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.04, 7.82, 7.99 [03:14:43] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.44, 9.61, 9.61 [03:16:40] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.44, 7.45, 7.85 [03:18:40] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.67, 7.58, 7.82 [03:20:48] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.45, 7.55, 7.78 [03:31:45] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.1442747712 secs [03:33:58] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.41, 2.75, 3.90 [03:37:58] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.13, 1.86, 3.27 [03:39:32] !log [agent@mwtask141] sudo -u www-data php cargoRecreateData.php --wiki projectsekaiwiki --table cards [03:39:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:42:43] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.97, 7.80, 7.57 [03:43:46] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset 0.07770994306 secs [03:44:42] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.27, 7.11, 7.34 [03:48:31] !log [agent@mwtask141] sudo -u www-data php cargoRecreateData.php --wiki projectsekaiwiki --table cards [03:48:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:48:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 10.23, 7.65, 7.44 [03:50:38] PROBLEM - os141 PowerDNS Recursor on os141 is CRITICAL: CRITICAL - Plugin timed out while executing system call [03:50:48] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 7.77, 5.29, 3.28 [03:52:39] RECOVERY - os141 PowerDNS Recursor on os141 is OK: DNS OK: 0.283 seconds response time. miraheze.org returns 109.228.51.216,217.174.247.33,2a00:da00:1800:326::1,2a00:da00:1800:328::1 [03:52:45] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 4.04, 4.28, 3.13 [04:00:46] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.63, 7.83, 7.93 [04:06:42] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.08, 7.46, 7.64 [04:10:33] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.53, 3.32, 2.61 [04:10:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.62, 7.67, 7.78 [04:12:33] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.67, 3.13, 2.61 [04:14:33] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.56, 2.58, 2.48 [04:16:14] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 9.60, 6.12, 3.83 [04:18:11] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 4.85, 5.80, 4.01 [04:20:40] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.01, 7.55, 7.46 [04:24:43] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.84, 9.60, 8.51 [04:26:56] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.57, 7.98, 7.68 [04:26:59] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.41, 9.05, 8.44 [04:34:43] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.87, 7.19, 7.29 [04:38:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.64, 6.96, 7.18 [04:50:07] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 7.57, 5.95, 3.72 [04:52:03] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 5.20, 5.51, 3.83 [04:54:42] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.31, 5.98, 6.60 [04:55:32] [02miraheze/RemovePII] 07AgentIsai created branch 03vanishuser 13https://github.com/miraheze/RemovePII/commit/05ab972dbf13f904f4736119f067356334c35c72 [04:55:35] [02RemovePII] 07AgentIsai created branch 03vanishuser - 13https://github.com/miraheze/RemovePII [04:57:20] [02miraheze/RemovePII] 07AgentIsai pushed 031 commit to 03vanishuser [+0/-0/±1] 13https://github.com/miraheze/RemovePII/compare/05ab972dbf13...9b5b53048aec [04:57:23] [02miraheze/RemovePII] 07AgentIsai 039b5b530 - Alias [04:57:43] [02miraheze/RemovePII] 07AgentIsai pushed 031 commit to 03vanishuser [+0/-0/±1] 13https://github.com/miraheze/RemovePII/compare/9b5b53048aec...18c70cb36194 [04:57:46] [02miraheze/RemovePII] 07AgentIsai 0318c70cb - Add vanishuser log messages [04:58:29] [02miraheze/RemovePII] 07AgentIsai pushed 031 commit to 03vanishuser [+1/-0/±0] 13https://github.com/miraheze/RemovePII/compare/18c70cb36194...4f2a0a8e978a [04:58:30] [02miraheze/RemovePII] 07AgentIsai 034f2a0a8 - Create Special:VanishUser [04:59:37] [02miraheze/RemovePII] 07AgentIsai pushed 031 commit to 03vanishuser [+0/-0/±1] 13https://github.com/miraheze/RemovePII/compare/4f2a0a8e978a...9888100d5a4f [04:59:39] [02miraheze/RemovePII] 07AgentIsai 039888100 - Fix indent [05:02:40] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.37, 6.87, 6.75 [05:03:58] [02miraheze/RemovePII] 07AgentIsai pushed 031 commit to 03vanishuser [+0/-0/±1] 13https://github.com/miraheze/RemovePII/compare/9888100d5a4f...b123329f5996 [05:04:01] [02miraheze/RemovePII] 07AgentIsai 03b123329 - Fix reason message [05:04:42] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.04, 6.54, 6.64 [05:06:41] [02RemovePII] 07AgentIsai opened pull request 03#68: Create Special:VanishUser - 13https://github.com/miraheze/RemovePII/pull/68 [05:08:19] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 6.93, 6.12, 4.97 [05:08:19] [02miraheze/RemovePII] 07github-actions[bot] pushed 031 commit to 03vanishuser [+0/-0/±1] 13https://github.com/miraheze/RemovePII/compare/b123329f5996...2b38732ba381 [05:08:19] [02miraheze/RemovePII] 07github-actions 032b38732 - CI: lint code to MediaWiki standards [05:08:19] [02RemovePII] 07github-actions[bot] synchronize pull request 03#68: Create Special:VanishUser - 13https://github.com/miraheze/RemovePII/pull/68 [05:08:33] miraheze/RemovePII - AgentIsai the build has errored. [05:08:42] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.46, 6.94, 6.83 [05:09:08] [02RemovePII] 07AgentIsai synchronize pull request 03#68: Create Special:VanishUser - 13https://github.com/miraheze/RemovePII/pull/68 [05:09:08] [02miraheze/RemovePII] 07AgentIsai pushed 031 commit to 03vanishuser [+0/-0/±1] 13https://github.com/miraheze/RemovePII/compare/2b38732ba381...2c8137691338 [05:09:08] [02miraheze/RemovePII] 07AgentIsai 032c81376 - Comma [05:10:04] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 9.07, 6.52, 5.21 [05:10:40] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.66, 6.60, 6.72 [05:12:14] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 4.43, 5.97, 5.20 [05:16:43] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.83, 6.59, 6.64 [05:18:01] PROBLEM - db112 Disk Space on db112 is WARNING: DISK WARNING - free space: / 14597 MB (10% inode=99%); [05:18:50] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.42, 6.40, 6.55 [05:20:48] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.47, 10.30, 9.13 [05:22:49] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 9.93, 9.93, 9.12 [05:25:31] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1216321588 secs [05:27:33] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.09825071692 secs [05:29:52] [02RemovePII] 07Universal-Omega commented on pull request 03#68: Create Special:VanishUser - 13https://github.com/miraheze/RemovePII/pull/68#issuecomment-1528661287 [05:37:26] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 6.92, 6.32, 5.76 [05:40:37] PROBLEM - os141 Current Load on os141 is CRITICAL: CRITICAL - load average: 4.70, 3.26, 2.00 [05:41:19] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 6.46, 6.78, 6.09 [05:42:25] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 1.50, 2.56, 1.90 [05:52:02] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 10.05, 7.62, 6.88 [05:53:35] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.62, 7.09, 6.77 [05:54:47] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - uk.religiononfire.mar.in.ua All nameservers failed to answer the query. [05:55:03] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.18, 7.55, 6.98 [05:57:00] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.90, 7.70, 7.10 [06:01:20] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 11.03, 8.44, 7.44 [06:02:22] PROBLEM - db142 Disk Space on db142 is WARNING: DISK WARNING - free space: / 36499 MB (10% inode=97%); [06:08:47] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.64, 7.12, 7.34 [06:09:04] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.93, 10.08, 9.23 [06:14:49] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 9.32, 10.05, 9.54 [06:16:40] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.76, 6.05, 6.74 [06:23:29] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [06:31:57] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.68, 10.34, 9.45 [06:33:58] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.76, 10.91, 9.75 [06:35:58] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.59, 11.15, 10.01 [06:42:04] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.78, 10.92, 10.15 [06:44:06] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.75, 9.25, 9.63 [06:44:20] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.85, 6.99, 6.68 [06:46:18] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.53, 6.29, 6.45 [06:50:01] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.84, 7.01, 6.76 [06:51:59] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.24, 6.51, 6.61 [07:01:01] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.93, 3.43, 2.81 [07:05:00] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.11, 3.38, 2.95 [07:16:47] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.70, 10.48, 9.14 [07:19:35] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.41, 10.70, 9.65 [07:22:33] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.39, 7.54, 6.93 [07:22:45] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.79, 9.00, 9.03 [07:23:26] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.84, 9.97, 9.60 [07:24:20] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.17, 6.83, 6.73 [07:30:17] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.14, 6.68, 6.73 [07:35:35] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.53, 10.14, 9.67 [07:39:36] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 9.52, 10.19, 9.82 [07:43:32] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.75, 12.10, 10.70 [07:44:21] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 9.20, 5.93, 3.61 [07:45:34] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.01, 10.26, 9.89 [07:46:16] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 3.19, 4.93, 3.53 [07:47:31] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 16.09, 12.35, 10.69 [07:48:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.84, 6.91, 6.65 [07:50:53] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.32, 6.66, 6.59 [08:01:32] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.30, 11.68, 11.85 [08:03:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.61, 12.16, 11.99 [08:11:31] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.56, 7.06, 6.55 [08:15:27] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.24, 6.65, 6.50 [08:19:20] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.19, 7.34, 6.77 [08:23:18] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.08, 7.45, 6.97 [08:25:14] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.97, 7.88, 7.17 [08:25:48] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.88, 3.48, 2.32 [08:27:11] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.15, 7.39, 7.07 [08:27:23] PROBLEM - tudien.como.vn - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'tudien.como.vn' expires in 15 day(s) (Mon 15 May 2023 08:13:54 GMT +0000). [08:27:49] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.16, 2.80, 2.20 [08:29:12] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.80, 7.69, 7.20 [08:31:03] PROBLEM - wiki.drycleaninglobby.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.drycleaninglobby.net' expires in 15 day(s) (Mon 15 May 2023 08:17:34 GMT +0000). [08:31:11] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.94, 7.39, 7.15 [08:32:45] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/a3a2463e6ccb...c153d9ab30b5 [08:32:59] [02miraheze/ssl] 07MirahezeSSLBot 03c153d9a - Bot: Update SSL cert for tudien.como.vn [08:37:01] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.46, 6.33, 6.76 [08:39:32] [02miraheze/CreateWiki] 07redbluegreenhat pushed 033 commits to 03master [+0/-0/±3] 13https://github.com/miraheze/CreateWiki/compare/8c8113459d35...544413b1d4a1 [08:39:32] [02miraheze/CreateWiki] 07redbluegreenhat 0327d9e83 - Merge pull request https://github.com/miraheze/CreateWiki/pull/398 [08:39:41] [02miraheze/CreateWiki] 07redbluegreenhat 03544413b - eslint: Ignore possible timing attacks [08:39:51] [02CreateWiki] 07redbluegreenhat closed pull request 03#398: Update mediawiki/mediawiki-phan-config requirement from 0.12.0 to 0.12.1 - 13https://github.com/miraheze/CreateWiki/pull/398 [08:39:52] [02miraheze/CreateWiki] 07redbluegreenhat deleted branch 03dependabot/composer/mediawiki/mediawiki-phan-config-0.12.1 [08:39:52] [02CreateWiki] 07redbluegreenhat deleted branch 03dependabot/composer/mediawiki/mediawiki-phan-config-0.12.1 - 13https://github.com/miraheze/CreateWiki [08:40:02] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/c153d9ab30b5...420519cdf8d0 [08:40:02] [02miraheze/ssl] 07MirahezeSSLBot 03420519c - Bot: Update SSL cert for wiki.drycleaninglobby.net [08:42:14] PROBLEM - db121 Backups SQL on db121 is WARNING: FILE_AGE WARNING: /var/log/sql-backup.log is 864180 seconds old and 1753 bytes [08:43:23] [02miraheze/CreateWiki] 07redbluegreenhat pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/CreateWiki/compare/544413b1d4a1...0dee3c4a4112 [08:43:26] [02miraheze/CreateWiki] 07redbluegreenhat 030dee3c4 - Typo [08:43:43] miraheze/CreateWiki - dependabot[bot] the build has errored. [08:45:55] [02miraheze/CreateWiki] 07redbluegreenhat pushed 032 commits to 03master [+0/-0/±4] 13https://github.com/miraheze/CreateWiki/compare/0dee3c4a4112...7d711e565e18 [08:45:55] [02miraheze/CreateWiki] 07redbluegreenhat 037d711e5 - Merge pull request https://github.com/miraheze/CreateWiki/pull/401 [08:45:55] [02CreateWiki] 07redbluegreenhat closed pull request 03#401: Bump eslint-config-wikimedia from 0.24.0 to 0.25.0 - 13https://github.com/miraheze/CreateWiki/pull/401 [08:45:55] [02miraheze/CreateWiki] 07redbluegreenhat deleted branch 03dependabot/npm_and_yarn/eslint-config-wikimedia-0.25.0 [08:45:55] [02CreateWiki] 07redbluegreenhat deleted branch 03dependabot/npm_and_yarn/eslint-config-wikimedia-0.25.0 - 13https://github.com/miraheze/CreateWiki [08:46:22] miraheze/CreateWiki - redbluegreenhat the build has errored. [08:50:40] PROBLEM - cp32 NTP time on cp32 is WARNING: NTP WARNING: Offset 0.2821708024 secs [08:53:22] miraheze/CreateWiki - redbluegreenhat the build passed. [08:53:31] 1 down, 1 more to go [08:56:31] miraheze/RemovePII - dependabot[bot] the build has errored. [08:57:09] heh [08:57:10] RECOVERY - tudien.como.vn - LetsEncrypt on sslhost is OK: OK - Certificate 'tudien.como.vn' will expire on Fri 28 Jul 2023 07:32:33 GMT +0000. [08:58:55] [02miraheze/IncidentReporting] 07redbluegreenhat pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/IncidentReporting/compare/d828b9b121f0...a5a83ba2a04c [08:58:56] [02IncidentReporting] 07redbluegreenhat closed pull request 03#60: Update mediawiki/mediawiki-phan-config requirement from 0.12.0 to 0.12.1 - 13https://github.com/miraheze/IncidentReporting/pull/60 [08:58:56] [02miraheze/IncidentReporting] 07redbluegreenhat deleted branch 03dependabot/composer/mediawiki/mediawiki-phan-config-0.12.1 [08:58:56] [02IncidentReporting] 07redbluegreenhat deleted branch 03dependabot/composer/mediawiki/mediawiki-phan-config-0.12.1 - 13https://github.com/miraheze/IncidentReporting [08:59:56] RECOVERY - wiki.drycleaninglobby.net - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.drycleaninglobby.net' will expire on Fri 28 Jul 2023 07:39:47 GMT +0000. [09:01:20] miraheze/IncidentReporting - redbluegreenhat the build passed. [09:02:26] [02RemovePII] 07redbluegreenhat commented on pull request 03#67: Update mediawiki/mediawiki-phan-config requirement from 0.12.0 to 0.12.1 - 13https://github.com/miraheze/RemovePII/pull/67#issuecomment-1528725814 [09:02:37] RECOVERY - cp32 NTP time on cp32 is OK: NTP OK: Offset -0.03250950575 secs [09:16:43] PROBLEM - os141 Disk Space on os141 is CRITICAL: DISK CRITICAL - free space: / 11667 MB (5% inode=99%); [09:18:44] PROBLEM - os141 Disk Space on os141 is WARNING: DISK WARNING - free space: / 16563 MB (8% inode=99%); [09:44:44] PROBLEM - os141 Disk Space on os141 is CRITICAL: DISK CRITICAL - free space: / 11978 MB (5% inode=99%); [09:45:31] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.184214592 secs [09:48:43] PROBLEM - os141 Disk Space on os141 is WARNING: DISK WARNING - free space: / 16676 MB (8% inode=99%); [10:01:28] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.08860701323 secs [10:09:28] PROBLEM - os131 Disk Space on os131 is CRITICAL: DISK CRITICAL - free space: / 11306 MB (5% inode=99%); [10:11:26] PROBLEM - phab121 Puppet on phab121 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [10:11:41] PROBLEM - os131 Disk Space on os131 is WARNING: DISK WARNING - free space: / 16291 MB (8% inode=99%); [10:27:52] RECOVERY - cloud11 IPMI Sensors on cloud11 is OK: IPMI Status: OK [10:31:41] PROBLEM - cloud11 IPMI Sensors on cloud11 is CRITICAL: IPMI Status: Critical [Cntlr 2 Bay 8 = Critical] [10:39:15] RECOVERY - phab121 Puppet on phab121 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [11:00:03] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 8.22, 5.58, 3.21 [11:02:04] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 1.87, 4.10, 2.96 [11:51:39] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.56, 6.61, 5.95 [11:53:27] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.54, 6.62, 6.04 [11:57:20] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.77, 7.59, 6.60 [12:01:19] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.19, 6.31, 6.31 [12:08:27] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.81, 6.56, 6.33 [12:11:20] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.32, 6.24, 6.26 [12:23:36] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.45, 7.71, 6.79 [12:43:20] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.44, 7.21, 6.69 [12:49:14] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.45, 7.64, 7.20 [12:53:09] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.04, 7.79, 7.34 [12:55:07] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.57, 7.03, 7.09 [12:56:03] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 13.71, 7.66, 4.06 [12:58:03] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 4.60, 6.42, 4.06 [12:58:09] “Would you like to discover how to profit from the cryptocurrency market if YES I’m willing to help 11 people how to earn $47,000 in just 72hours Dm me to know How [12:59:03] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.77, 5.85, 6.60 [13:20:40] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.64, 7.26, 6.68 [13:22:40] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.42, 6.48, 6.46 [13:24:31] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.86, 3.07, 2.45 [13:24:44] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.religiononfire.mar.in.ua. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [13:26:29] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.83, 2.99, 2.50 [13:26:40] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.99, 7.03, 6.72 [13:28:41] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.24, 6.45, 6.55 [13:36:40] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.11, 7.20, 6.89 [13:38:41] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.75, 6.54, 6.68 [13:40:34] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.130848676 secs [13:44:34] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.09122878313 secs [13:44:51] PROBLEM - cp22 Current Load on cp22 is WARNING: WARNING - load average: 3.46, 2.19, 1.48 [13:46:51] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 1.77, 2.03, 1.52 [13:50:40] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.46, 8.91, 7.53 [13:56:55] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.22, 3.46, 2.97 [13:58:52] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.15, 3.68, 3.10 [14:00:40] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.48, 7.42, 7.49 [14:00:50] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.46, 2.89, 2.88 [14:02:00] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [14:02:40] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 7.60, 8.06, 7.75 [14:04:40] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.69, 7.26, 7.50 [14:05:52] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:06:40] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 10.14, 7.89, 7.66 [14:12:08] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 9.392 second response time [14:22:02] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.43, 3.24, 2.89 [14:22:44] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [14:24:01] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 2.55, 3.09, 2.89 [14:24:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.69, 3.27, 2.79 [14:26:21] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.38, 4.27, 3.19 [14:26:40] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.59, 6.95, 7.94 [14:28:19] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.78, 3.51, 3.04 [14:30:17] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.81, 3.07, 2.95 [14:38:40] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.74, 5.47, 6.66 [14:44:40] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.97, 6.73, 6.88 [14:46:40] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.75, 6.47, 6.77 [14:53:48] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 4.00, 3.69, 2.94 [14:56:32] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - en.religiononfire.mar.in.ua All nameservers failed to answer the query. [14:57:00] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [14:57:46] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.45, 2.76, 2.76 [15:03:35] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 14.30, 7.66, 4.43 [15:05:09] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.87, 3.11, 2.56 [15:06:25] PROBLEM - os141 Current Load on os141 is CRITICAL: CRITICAL - load average: 5.27, 3.23, 1.86 [15:07:06] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.68, 3.73, 2.83 [15:08:25] PROBLEM - os141 Current Load on os141 is WARNING: WARNING - load average: 3.89, 3.30, 2.05 [15:09:03] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.75, 3.96, 3.04 [15:10:25] PROBLEM - os141 Current Load on os141 is CRITICAL: CRITICAL - load average: 4.30, 3.76, 2.38 [15:10:40] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.66, 6.78, 6.37 [15:11:46] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.16, 4.07, 3.14 [15:12:25] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 1.18, 2.90, 2.24 [15:12:40] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.44, 5.86, 6.08 [15:12:57] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.92, 3.05, 2.89 [15:13:21] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 4.72, 6.94, 6.07 [15:13:41] PROBLEM - cp33 APT on cp33 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [15:13:46] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.56, 3.82, 3.17 [15:15:18] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 1.23, 4.92, 5.43 [15:15:39] RECOVERY - cp33 APT on cp33 is OK: APT OK: 36 packages available for upgrade (0 critical updates). [15:15:46] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.75, 2.91, 2.91 [15:22:34] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.1289515793 secs [15:23:46] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.98, 4.46, 3.53 [15:25:51] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [15:26:34] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.09227612615 secs [15:27:46] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.81, 3.81, 3.50 [15:28:40] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.96, 7.79, 6.61 [15:29:46] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.54, 3.37, 3.37 [15:30:40] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.61, 7.12, 6.50 [15:32:40] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.07, 7.36, 6.63 [15:33:49] [02miraheze/IncidentReporting] 07redbluegreenhat pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/IncidentReporting/compare/a5a83ba2a04c...4660ebf90ae4 [15:33:50] [02miraheze/IncidentReporting] 07redbluegreenhat 034660ebf - eslint: Do not check for possible timing attacks [15:35:51] miraheze/IncidentReporting - redbluegreenhat the build has errored. [15:36:40] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.14, 6.84, 6.62 [15:37:26] [02miraheze/IncidentReporting] 07redbluegreenhat pushed 032 commits to 03master [+0/-0/±4] 13https://github.com/miraheze/IncidentReporting/compare/4660ebf90ae4...cef05e8c08b7 [15:37:28] [02miraheze/IncidentReporting] 07redbluegreenhat 03cef05e8 - Merge pull request https://github.com/miraheze/IncidentReporting/pull/61 [15:37:31] [02IncidentReporting] 07redbluegreenhat closed pull request 03#61: Bump eslint-config-wikimedia from 0.24.0 to 0.25.0 - 13https://github.com/miraheze/IncidentReporting/pull/61 [15:37:32] [02miraheze/IncidentReporting] 07redbluegreenhat deleted branch 03dependabot/npm_and_yarn/eslint-config-wikimedia-0.25.0 [15:37:33] [02IncidentReporting] 07redbluegreenhat deleted branch 03dependabot/npm_and_yarn/eslint-config-wikimedia-0.25.0 - 13https://github.com/miraheze/IncidentReporting [15:38:40] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.42, 6.42, 6.50 [15:41:11] miraheze/IncidentReporting - redbluegreenhat the build passed. [15:41:21] nice [15:42:29] miraheze/RemovePII - dependabot[bot] the build has errored. [15:43:46] I keep re-running jobs like I'm trying to get a compiler to somehow magically solve errors, then keep getting surprised when it doesn't work [15:44:07] I'm honestly at a loss here [15:50:03] Orange_Star: what you at a loss with [15:50:22] with https://github.com/miraheze/RemovePII/pull/67 [15:50:31] I want to get the CI to pass before merging [15:52:43] PROBLEM - os141 Disk Space on os141 is CRITICAL: DISK CRITICAL - free space: / 11665 MB (5% inode=99%); [15:53:12] [02RemovePII] 07RhinosF1 opened pull request 03#69: Update composer.json TEST BLANK PR CI - 13https://github.com/miraheze/RemovePII/pull/69 [15:53:32] Orange_Star: that error seems impossible [15:53:45] Why would phan do that [15:53:49] the CI says that [15:54:16] if I go further back the logs I can see it installing normally, but it just doesn't wanna work [15:54:22] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.16, 7.16, 6.87 [15:54:43] PROBLEM - os141 Disk Space on os141 is WARNING: DISK WARNING - free space: / 12569 MB (6% inode=99%); [15:55:20] the same thing happened to your PR [15:55:43] miraheze/RemovePII - RhinosF1 the build has errored. [15:56:04] Orange_Star: so something has changed [15:56:22] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.69, 6.37, 6.62 [15:56:34] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset -0.109616518 secs [15:57:14] [02RemovePII] 07RhinosF1 synchronize pull request 03#69: Update composer.json TEST BLANK PR CI - 13https://github.com/miraheze/RemovePII/pull/69 [15:57:33] I'm marking it experimental [15:57:44] To see if it's landed in 1.40 or if its 1.41 [15:57:52] If its 1.41, that's not too worrying [16:00:34] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset -0.0962407589 secs [16:01:40] The bot didn't work but RhinosF1 the build passed [16:01:48] something changed in master then [16:03:13] Orange_Star: that's less worrying [16:03:26] I don't think we can expect social profile to work with 1.41 [16:03:45] that's gonna be bad news for some large wikis [16:03:52] hopefully it won't be the case [16:05:23] I've messaged Ashley [16:05:31] We have 6 months to figure that out [16:05:36] I'm very sure it'll be fixed [16:05:42] Ashley is extremely responsive [16:05:54] Orange_Star: why are we not running CI against php82 [16:06:06] We're going to be moving to bullseye after 1.40 [16:06:09] because no-one maintains the workflows [16:06:13] So in 8 weeks time [16:06:16] Which is php82 [16:06:33] Orange_Star: that's not good [16:06:45] You're not wrong [16:06:57] not bullseye [16:06:59] Bookworm [16:07:08] Orange_Star: Debian release is like 5 weeks time [16:07:15] We should be scared [16:09:27] PROBLEM - os131 Disk Space on os131 is CRITICAL: DISK CRITICAL - free space: / 11929 MB (5% inode=99%); [16:09:29] I don't have write access to https://github.com/miraheze/.github it seems [16:09:43] I'll write a PR and leave it for someone on infra then [16:14:58] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.38, 7.08, 6.69 [16:15:27] PROBLEM - os131 Disk Space on os131 is WARNING: DISK WARNING - free space: / 12172 MB (6% inode=99%); [16:16:43] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:16:56] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.37, 6.39, 6.48 [16:20:47] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 3.282 second response time [16:21:27] PROBLEM - os131 Disk Space on os131 is CRITICAL: DISK CRITICAL - free space: / 11916 MB (5% inode=99%); [16:22:43] PROBLEM - os141 Disk Space on os141 is CRITICAL: DISK CRITICAL - free space: / 11952 MB (5% inode=99%); [16:25:27] PROBLEM - os131 Disk Space on os131 is WARNING: DISK WARNING - free space: / 12286 MB (6% inode=99%); [16:30:43] PROBLEM - os141 Disk Space on os141 is WARNING: DISK WARNING - free space: / 12124 MB (6% inode=99%); [16:31:02] phab and icingaweb2 don’t yet support PHP 8.2, thus those servers can’t be upgraded to bookworm unless we can still keep the older PHP versions [16:33:27] PROBLEM - os131 Disk Space on os131 is CRITICAL: DISK CRITICAL - free space: / 11953 MB (5% inode=99%); [16:34:41] [02.github] 07redbluegreenhat opened pull request 03#31: Add PHP 8.2 tests - 13https://github.com/miraheze/.github/pull/31 [16:34:43] PROBLEM - os141 Disk Space on os141 is CRITICAL: DISK CRITICAL - free space: / 11981 MB (5% inode=99%); [16:35:52] icinga supports PHP 8, is 8.2 a big bump up from 8? [16:36:00] icingaweb2 that is [16:36:19] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 9.09, 6.00, 4.14 [16:36:21] Yes there are some breakages [16:36:52] Even PHP 8.1 caused a massive amount of breakages in Phabricator [16:37:25] Phorge has been working on the fixes though, so I think full support isn’t too far off [16:38:12] https://github.com/Icinga/icingaweb2/issues/4918 [16:38:16] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 7.83, 6.67, 4.61 [16:38:25] PROBLEM - os141 Current Load on os141 is CRITICAL: CRITICAL - load average: 4.59, 3.23, 1.91 [16:38:32] support in icingaweb2 is in progress [16:40:13] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 9.81, 7.51, 5.15 [16:40:43] PROBLEM - os141 Disk Space on os141 is WARNING: DISK WARNING - free space: / 12287 MB (6% inode=99%); [16:42:10] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 4.90, 6.82, 5.20 [16:42:26] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 2.01, 3.14, 2.22 [16:44:07] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 2.75, 5.52, 4.92 [16:46:43] PROBLEM - os141 Disk Space on os141 is CRITICAL: DISK CRITICAL - free space: / 11966 MB (5% inode=99%); [16:46:51] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.56, 3.94, 2.89 [16:48:00] PROBLEM - cp33 APT on cp33 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [16:48:48] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.50, 3.86, 3.00 [16:50:45] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.26, 4.30, 3.27 [16:52:36] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.45, 4.03, 3.22 [16:52:53] RECOVERY - cp33 APT on cp33 is OK: APT OK: 36 packages available for upgrade (0 critical updates). [16:56:36] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.59, 3.84, 3.48 [16:57:25] MacFan4000: yeah phorge isn't far off [16:57:29] we have a year or so [16:57:36] if we want to hit our targets [16:57:59] but we don't want mixed mw* for long or test to be running a different php version to prod [16:58:06] when it comes to mw releases [16:58:12] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.23, 6.93, 6.53 [16:58:28] 8 -> 8.2 shouldn't be awful [16:58:33] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.64, 3.04, 3.22 [16:59:38] mediawiki doesn't even have confirmed 8.2 support on the table [17:02:07] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.30, 6.05, 6.32 [17:02:25] Orange_Star: (or any SRE MacFan4000 / Agent ) Has social profile been tested on 1.40? [17:02:26] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.99, 3.69, 3.85 [17:02:30] I know that there is breakage in icingaweb from having tried it [17:02:44] I don’t know [17:02:52] it's not on https://phabricator.miraheze.org/T10680 [17:03:20] It's there, as SocialProfile [17:03:28] and looks like no-one has taken a look yet [17:05:32] testwiki has it on [17:05:35] so that's fine [17:05:39] testwikibeta even [17:06:01] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.45, 8.08, 7.06 [17:07:58] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.20, 7.32, 6.90 [17:08:19] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.87, 2.71, 3.35 [17:09:56] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.08, 6.32, 6.60 [17:11:15] when Orange_Star back, https://phabricator.wikimedia.org/T335626 [17:11:46] [02RemovePII] 07RhinosF1 closed pull request 03#69: Update composer.json TEST BLANK PR CI - 13https://github.com/miraheze/RemovePII/pull/69 [17:12:01] [02RemovePII] 07RhinosF1 commented on pull request 03#67: Update mediawiki/mediawiki-phan-config requirement from 0.12.0 to 0.12.1 - 13https://github.com/miraheze/RemovePII/pull/67#issuecomment-1528831764 [17:17:58] [02.github] 07RhinosF1 commented on pull request 03#31: Add PHP 8.2 tests - 13https://github.com/miraheze/.github/pull/31#issuecomment-1528832725 [17:27:56] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.16, 3.66, 3.18 [17:29:29] [02mw-config] 07brandon-wm opened pull request 03#5217: T10777: Set options for zhtranswiki - 13https://github.com/miraheze/mw-config/pull/5217 [17:29:54] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.54, 3.29, 3.10 [17:30:31] miraheze/mw-config - brandon-wm the build passed. [17:35:54] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 10.06, 5.75, 3.84 [17:45:38] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.39, 3.90, 3.87 [17:46:16] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: CRITICAL - load average: 4.39, 3.65, 3.03 [17:48:03] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.34, 7.39, 6.84 [17:48:15] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.34, 3.56, 3.08 [17:49:32] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.30, 2.53, 3.31 [17:50:01] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.71, 8.52, 7.34 [17:50:15] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: CRITICAL - load average: 4.47, 3.84, 3.23 [17:52:15] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 2.53, 3.44, 3.17 [17:54:14] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 2.07, 3.04, 3.06 [17:57:52] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 4.78, 7.02, 7.43 [18:05:11] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.10, 3.85, 3.34 [18:07:08] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.67, 3.69, 3.35 [18:09:05] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.42, 2.88, 3.09 [18:15:32] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.83, 5.95, 6.64 [18:19:26] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.27, 7.08, 6.96 [18:21:23] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.32, 7.45, 7.12 [18:24:04] [02mw-config] 07AgentIsai closed pull request 03#5217: T10777: Set options for zhtranswiki - 13https://github.com/miraheze/mw-config/pull/5217 [18:24:06] [02miraheze/mw-config] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/7db95141a297...315f45465a6f [18:24:08] [02miraheze/mw-config] 07brandon-wm 03315f454 - T10777: Set options for zhtranswiki (#5217) [18:25:29] miraheze/mw-config - AgentIsai the build passed. [18:25:43] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.38, 3.51, 3.13 [18:26:54] !log [@test131] starting deploy of {'config': True} to all [18:26:55] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 1s [18:27:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:27:40] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.74, 2.93, 2.96 [18:27:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:36:33] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.65, 4.96, 3.78 [18:37:05] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.99, 6.39, 6.73 [18:41:10] !log [@mwtask141] starting deploy of {'config': True} to all [18:41:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:41:20] !log [@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 10s [18:41:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:42:20] [02mw-config] 07AgentIsai synchronize pull request 03#5216: T10773: Add $wgUFAllowedNamespaces - 13https://github.com/miraheze/mw-config/pull/5216 [18:42:33] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.92, 3.59, 3.66 [18:42:37] [02mw-config] 07AgentIsai closed pull request 03#5216: T10773: Add $wgUFAllowedNamespaces - 13https://github.com/miraheze/mw-config/pull/5216 [18:42:40] [02miraheze/mw-config] 07AgentIsai pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/mw-config/compare/315f45465a6f...6c061678c160 [18:42:43] [02miraheze/mw-config] 07brandon-wm 036c06167 - T10773: Add $wgUFAllowedNamespaces (#5216) [18:43:16] miraheze/mw-config - AgentIsai the build passed. [18:43:39] miraheze/mw-config - AgentIsai the build passed. [18:45:52] [02miraheze/mw-config] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/6c061678c160...471692736230 [18:45:54] [02miraheze/mw-config] 07AgentIsai 034716927 - T10782: Add more defaults for zhtranswiki [18:46:53] miraheze/mw-config - AgentIsai the build passed. [18:47:14] !log [agent@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [18:47:23] !log [agent@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 8s [18:47:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:47:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:48:33] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.14, 2.87, 3.31 [18:57:04] !log [@test131] starting deploy of {'config': True} to all [18:57:05] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 1s [18:57:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:57:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:04:03] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 8.65, 5.68, 3.25 [19:06:03] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 4.55, 5.91, 3.68 [19:11:45] !log [agent@mwtask141] swift download miraheze-intercriaturaswiki-local-public -D intercriaturaswiki-images --object-threads=1 [19:11:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:19:13] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.98, 7.03, 6.61 [19:19:54] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.41, 3.25, 2.68 [19:21:51] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.60, 2.57, 2.49 [19:25:07] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.42, 6.80, 6.68 [19:32:56] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.90, 7.24, 6.88 [19:34:08] PROBLEM - mwtask141 Disk Space on mwtask141 is WARNING: DISK WARNING - free space: / 7483 MB (10% inode=91%); [19:36:51] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.33, 6.76, 6.77 [19:40:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 7.70, 8.21, 7.40 [19:42:43] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 4.94, 7.20, 7.14 [19:46:08] PROBLEM - mwtask141 Disk Space on mwtask141 is CRITICAL: DISK CRITICAL - free space: / 4114 MB (5% inode=91%); [19:48:06] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.61, 10.19, 11.77 [19:50:06] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.16, 10.53, 11.68 [19:50:40] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.42, 8.07, 7.45 [19:52:01] RECOVERY - db112 Disk Space on db112 is OK: DISK OK - free space: / 15363 MB (11% inode=99%); [19:52:06] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.03, 10.15, 11.39 [19:52:35] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 13.78, 8.21, 5.27 [19:54:40] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.72, 7.42, 7.39 [19:55:46] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.75, 3.35, 2.49 [19:56:43] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.66, 10.32, 11.71 [19:58:40] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 13.50, 9.03, 7.93 [20:01:46] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.94, 3.42, 2.95 [20:02:05] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 5.14, 8.06, 9.93 [20:02:40] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.57, 7.61, 7.65 [20:03:34] PROBLEM - mw141 MediaWiki Rendering on mw141 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:03:47] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.87, 2.92, 2.82 [20:04:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 14.53, 9.69, 10.63 [20:05:30] RECOVERY - mw141 MediaWiki Rendering on mw141 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.230 second response time [20:06:14] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 2.97, 7.78, 7.84 [20:07:01] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 9.74, 5.42, 3.33 [20:07:47] PROBLEM - mwtask141 HTTPS on mwtask141 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:08:05] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.45, 10.07, 10.27 [20:08:14] PROBLEM - mwtask141 MediaWiki Rendering on mwtask141 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:08:43] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.76, 11.60, 11.31 [20:10:05] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 9.50, 9.90, 10.18 [20:12:06] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 1.12, 4.40, 6.47 [20:12:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 17.10, 14.06, 12.31 [20:14:01] RECOVERY - mwtask141 HTTPS on mwtask141 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 553 bytes in 0.009 second response time [20:14:05] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.32, 10.75, 10.46 [20:14:20] PROBLEM - mwtask141 MediaWiki Rendering on mwtask141 is WARNING: HTTP WARNING: HTTP/1.1 404 Not Found - 6664 bytes in 0.076 second response time [20:16:25] PROBLEM - mwtask141 MediaWiki Rendering on mwtask141 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:18:06] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 14.87, 11.36, 10.65 [20:18:20] PROBLEM - mwtask141 MediaWiki Rendering on mwtask141 is WARNING: HTTP WARNING: HTTP/1.1 404 Not Found - 6664 bytes in 0.034 second response time [20:19:35] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.31, 3.97, 3.15 [20:20:25] PROBLEM - mwtask141 MediaWiki Rendering on mwtask141 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:20:27] PROBLEM - mwtask141 Puppet on mwtask141 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 3 minutes ago with 3 failures. Failed resources (up to 3 shown): File[/etc/apt/trusted.gpg.d/puppetlabs.gpg],File[/usr/local/bin/puppet-enabled],File[/etc/sysctl.d] [20:22:20] PROBLEM - mwtask141 MediaWiki Rendering on mwtask141 is WARNING: HTTP WARNING: HTTP/1.1 404 Not Found - 6664 bytes in 0.042 second response time [20:23:28] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.61, 3.64, 3.25 [20:24:05] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.92, 11.27, 10.95 [20:24:40] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.84, 5.87, 6.60 [20:25:25] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.35, 3.29, 3.17 [20:25:28] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.14, 4.18, 3.27 [20:26:52] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 1.14, 2.62, 3.96 [20:29:24] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.18, 3.56, 3.26 [20:30:50] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 0.63, 1.52, 3.20 [20:31:22] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.68, 3.14, 3.13 [20:32:05] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.54, 11.92, 11.39 [20:35:16] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.19, 3.41, 3.31 [20:36:40] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.34, 6.90, 6.64 [20:37:14] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 3.26, 3.38, 3.31 [20:37:40] PROBLEM - cp33 APT on cp33 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [20:38:40] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.77, 6.04, 6.35 [20:39:10] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.33, 3.96, 3.40 [20:39:37] RECOVERY - mwtask141 MediaWiki Rendering on mwtask141 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.264 second response time [20:41:07] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.86, 3.20, 3.19 [20:42:20] RECOVERY - mwtask141 Puppet on mwtask141 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [20:43:12] RECOVERY - cp33 APT on cp33 is OK: APT OK: 36 packages available for upgrade (0 critical updates). [20:45:04] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.41, 3.41, 3.32 [20:45:24] PROBLEM - mwtask141 MediaWiki Rendering on mwtask141 is WARNING: HTTP WARNING: HTTP/1.1 404 Not Found - 6664 bytes in 0.033 second response time [20:45:40] RECOVERY - cloud11 IPMI Sensors on cloud11 is OK: IPMI Status: OK [20:46:08] RECOVERY - mwtask141 Disk Space on mwtask141 is OK: DISK OK - free space: / 12751 MB (17% inode=91%); [20:47:19] RECOVERY - mwtask141 MediaWiki Rendering on mwtask141 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.296 second response time [20:49:00] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.45, 3.73, 3.45 [20:49:31] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.04, 7.15, 6.66 [20:49:40] PROBLEM - cloud11 IPMI Sensors on cloud11 is CRITICAL: IPMI Status: Critical [Cntlr 2 Bay 8 = Critical] [20:50:58] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.39, 2.78, 3.13 [20:51:29] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.03, 6.71, 6.55 [20:52:06] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.48, 11.37, 11.99 [20:54:51] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.56, 4.74, 3.84 [20:55:14] PROBLEM - wiki.yuanpi.eu.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.yuanpi.eu.org could not be found [20:55:22] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.58, 6.98, 6.72 [20:55:51] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - en.religiononfire.mar.in.ua All nameservers failed to answer the query. [20:57:20] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.14, 6.43, 6.56 [20:58:06] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.33, 11.90, 12.02 [21:02:06] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.64, 11.40, 11.82 [21:02:43] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.92, 3.92, 3.90 [21:04:06] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.68, 12.50, 12.18 [21:08:06] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.15, 11.61, 11.88 [21:08:37] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.32, 2.85, 3.40 [21:10:06] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.94, 12.21, 12.05 [21:14:28] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.91, 4.02, 3.67 [21:14:51] PROBLEM - cp22 Current Load on cp22 is WARNING: WARNING - load average: 3.96, 2.13, 1.27 [21:16:34] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.1665705442 secs [21:16:51] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 2.42, 2.27, 1.44 [21:17:26] PROBLEM - cp22 APT on cp22 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [21:18:05] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.09, 10.95, 11.64 [21:18:24] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.23, 3.43, 3.52 [21:19:23] RECOVERY - cp22 APT on cp22 is OK: APT OK: 36 packages available for upgrade (0 critical updates). [21:20:06] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.24, 11.28, 11.66 [21:22:06] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.40, 10.34, 11.26 [21:22:20] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.96, 3.16, 3.38 [21:22:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.48, 11.19, 11.99 [21:24:06] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.54, 10.77, 11.27 [21:25:11] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [21:26:05] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.12, 9.98, 10.91 [21:26:14] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.98, 3.93, 3.64 [21:32:05] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.90, 9.09, 10.19 [21:32:08] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.16, 3.65, 3.72 [21:32:37] PROBLEM - cp32 NTP time on cp32 is WARNING: NTP WARNING: Offset 0.2719163001 secs [21:34:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.31, 11.19, 11.28 [21:36:04] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.01, 2.76, 3.33 [21:36:05] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.17, 10.88, 10.67 [21:40:05] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.94, 11.56, 11.07 [21:40:34] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset -0.0103700459 secs [21:41:55] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.67, 3.29, 3.42 [21:43:53] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.96, 2.77, 3.21 [21:48:05] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.43, 10.99, 10.86 [21:52:37] RECOVERY - cp32 NTP time on cp32 is OK: NTP OK: Offset -0.02279880643 secs [21:54:05] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.72, 11.64, 11.24 [21:54:38] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.33, 3.54, 3.25 [21:54:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.59, 11.30, 11.91 [21:56:36] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.79, 3.25, 3.19 [22:00:30] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.36, 3.60, 3.42 [22:02:06] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.49, 8.84, 10.07 [22:03:21] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.33, 4.04, 3.31 [22:04:26] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.30, 3.14, 3.29 [22:05:18] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.36, 3.88, 3.34 [22:07:15] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.15, 3.31, 3.20 [22:08:20] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.61, 3.72, 3.48 [22:10:40] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.22, 6.99, 6.11 [22:10:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.94, 11.48, 11.09 [22:12:16] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.51, 3.87, 3.63 [22:12:40] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.60, 6.09, 5.89 [22:12:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.50, 11.53, 11.19 [22:13:08] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.43, 3.82, 3.38 [22:15:05] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 3.18, 3.31, 3.23 [22:20:08] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.22, 2.72, 3.26 [22:20:42] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 9.79, 9.31, 10.11 [22:24:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.00, 10.02, 10.22 [22:26:59] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.91, 10.31, 9.95 [22:32:43] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 14.36, 11.16, 10.54 [22:36:39] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.03, 3.86, 3.16 [22:36:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.75, 11.57, 10.85 [22:36:56] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.87, 11.54, 10.69 [22:38:36] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.58, 3.20, 2.99 [22:38:43] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 14.73, 12.57, 11.30 [22:38:55] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.70, 11.50, 10.81 [22:42:54] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.19, 11.27, 10.81 [22:43:46] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.64, 4.22, 3.54 [22:44:53] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.81, 10.50, 10.58 [22:45:46] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.36, 3.40, 3.32 [22:50:51] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 16.20, 12.69, 11.36 [22:52:43] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.24, 11.58, 11.92 [22:52:51] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.45, 11.43, 11.07 [22:55:36] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.90, 3.86, 3.62 [22:57:34] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.94, 4.85, 4.02 [23:00:48] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.27, 9.38, 10.18 [23:01:29] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.53, 3.47, 3.68 [23:02:43] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.05, 8.38, 10.09 [23:04:47] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.57, 10.17, 10.36 [23:08:01] PROBLEM - db112 Disk Space on db112 is WARNING: DISK WARNING - free space: / 14601 MB (10% inode=99%); [23:08:45] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.55, 9.47, 10.06 [23:09:21] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.50, 2.88, 3.30 [23:12:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.94, 10.75, 10.33 [23:14:42] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 9.32, 9.87, 10.04 [23:18:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.15, 11.14, 10.55 [23:20:43] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.80, 11.66, 10.82 [23:22:43] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.00, 11.92, 11.01 [23:23:15] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.07, 3.60, 3.16 [23:24:03] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.50, 4.01, 3.43 [23:25:12] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.32, 4.18, 3.41 [23:27:09] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.27, 3.48, 3.25 [23:28:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.53, 11.76, 11.30 [23:29:06] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.50, 2.84, 3.05 [23:34:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.44, 11.35, 11.18 [23:36:34] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.67, 11.13, 10.41 [23:36:57] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 3.85, 4.02, 3.44 [23:37:49] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.02, 3.67, 3.97 [23:38:54] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.46, 3.38, 3.27 [23:40:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.38, 11.66, 11.37 [23:42:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 18.91, 14.47, 12.44 [23:43:46] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 7.07, 5.02, 4.33 [23:47:46] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.83, 3.43, 3.85 [23:48:31] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 15.49, 12.95, 11.51 [23:48:45] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.65, 4.11, 3.59 [23:52:39] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.39, 3.93, 3.64 [23:53:46] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.25, 4.27, 4.07 [23:58:33] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.60, 3.11, 3.37