[00:02:20] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.69, 11.63, 11.68 [00:03:28] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.22, 1.83, 1.62 [00:03:48] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.61, 1.75, 1.53 [00:04:20] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.96, 11.59, 11.68 [00:05:26] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.89, 1.39, 1.48 [00:06:38] PROBLEM - cp20 Stunnel HTTP for mw111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:06:41] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:06:47] PROBLEM - cp21 Stunnel HTTP for mw111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:06:58] PROBLEM - cp31 Stunnel HTTP for mw111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:07:02] PROBLEM - cp30 Stunnel HTTP for mw111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:07:42] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.28, 1.70, 1.58 [00:08:40] RECOVERY - cp20 Stunnel HTTP for mw111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 7.289 second response time [00:08:44] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 28487 bytes in 7.793 second response time [00:08:49] RECOVERY - cp21 Stunnel HTTP for mw111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 6.330 second response time [00:09:03] RECOVERY - cp31 Stunnel HTTP for mw111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 7.798 second response time [00:09:09] RECOVERY - cp30 Stunnel HTTP for mw111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 9.708 second response time [00:11:31] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.54, 1.72, 1.98 [00:14:55] PROBLEM - mwtask111 Current Load on mwtask111 is WARNING: WARNING - load average: 3.60, 3.65, 2.57 [00:16:55] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 2.30, 3.15, 2.52 [00:17:31] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.53, 1.95, 1.97 [00:25:31] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.40, 1.85, 1.96 [00:31:31] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.32, 1.79, 1.86 [00:32:17] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.18, 10.94, 9.43 [00:34:16] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.07, 10.14, 9.32 [00:35:31] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.71, 1.84, 1.87 [00:40:20] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 10.03, 9.47, 10.14 [00:40:50] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.62, 1.79, 1.50 [00:42:47] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.67, 1.79, 1.53 [00:44:45] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.58, 1.70, 1.53 [00:47:31] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.74, 2.19, 1.96 [00:48:39] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.82, 1.85, 1.62 [00:50:39] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.53, 1.70, 1.59 [00:54:20] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.00, 10.01, 9.76 [00:58:20] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.46, 9.85, 9.75 [01:06:20] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.35, 10.32, 10.01 [01:08:20] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 10.08, 10.11, 9.96 [01:10:20] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.92, 1.74, 1.53 [01:12:17] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 5.91, 2.81, 1.92 [01:23:44] PROBLEM - cloud11 Puppet on cloud11 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [01:25:58] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.53, 1.93, 1.95 [01:26:51] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-3 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/db71e11b8d36 [01:26:53] [02miraheze/mw-config] 07Universal-Omega 03db71e11 - Rename `$wgCosmosRailBlacklistedNamespaces` to `$wgCosmosRailDisabledNamespaces` [01:26:54] [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-3 - 13https://github.com/miraheze/mw-config [01:26:55] [url] Page not found · GitHub · GitHub | github.com [01:27:36] [02mw-config] 07Universal-Omega opened pull request 03#4816: Rename `$wgCosmosRailBlacklistedNamespaces` to `$wgCosmosRailDisabledNamespaces` - 13https://github.com/miraheze/mw-config/pull/4816 [01:27:37] ... [01:28:46] miraheze/mw-config - Universal-Omega the build passed. [01:31:50] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.87, 1.27, 1.65 [01:34:59] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.68, 10.85, 10.20 [01:40:48] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.64, 11.58, 10.68 [01:40:49] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.35, 10.22, 9.49 [01:42:44] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.73, 11.52, 10.76 [01:42:47] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 9.71, 9.96, 9.47 [01:48:33] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.32, 11.62, 10.91 [01:48:40] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.19, 10.31, 9.78 [01:49:44] RECOVERY - cloud11 Puppet on cloud11 is OK: OK: Puppet is currently enabled, last run 12 seconds ago with 0 failures [01:50:38] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 9.82, 9.95, 9.70 [01:52:26] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.24, 11.17, 10.95 [01:54:45] PROBLEM - mw122 MediaWiki Rendering on mw122 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:55:39] PROBLEM - cp21 Stunnel HTTP for mw122 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:56:47] RECOVERY - mw122 MediaWiki Rendering on mw122 is OK: HTTP OK: HTTP/1.1 200 OK - 28479 bytes in 6.587 second response time [01:57:37] RECOVERY - cp21 Stunnel HTTP for mw122 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 3.321 second response time [01:58:15] PROBLEM - cp21 Stunnel HTTP for mw111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:58:27] PROBLEM - cp31 Stunnel HTTP for mw111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:58:34] PROBLEM - cp30 Stunnel HTTP for mw111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:58:51] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:00:42] RECOVERY - cp30 Stunnel HTTP for mw111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 9.324 second response time [02:01:31] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.85, 1.75, 2.00 [02:02:23] RECOVERY - cp21 Stunnel HTTP for mw111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 6.681 second response time [02:02:57] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 28479 bytes in 6.725 second response time [02:04:43] RECOVERY - cp31 Stunnel HTTP for mw111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 4.111 second response time [02:05:19] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.11, 10.34, 9.97 [02:07:17] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.73, 10.98, 10.23 [02:08:15] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.71, 1.64, 1.43 [02:09:15] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.04, 10.43, 10.12 [02:11:13] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.79, 9.18, 9.70 [02:12:15] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.03, 1.36, 1.37 [02:13:31] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.43, 1.43, 1.69 [02:14:02] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk [02:17:31] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 3.31, 2.01, 1.83 [02:22:15] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.15, 1.63, 1.43 [02:22:20] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.76, 9.15, 10.10 [02:24:15] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.57, 1.51, 1.40 [02:27:31] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.51, 1.89, 1.95 [02:27:46] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.54, 9.31, 8.87 [02:29:45] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.92, 9.01, 8.80 [02:32:20] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.36, 9.95, 9.98 [02:32:47] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.36, 9.87, 9.29 [02:34:20] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.34, 10.45, 10.14 [02:34:45] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.36, 11.04, 9.78 [02:36:20] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.72, 10.74, 10.30 [02:36:43] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.71, 10.67, 9.79 [02:41:31] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.46, 2.11, 1.98 [02:42:21] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.06, 9.92, 8.97 [02:44:15] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.74, 9.34, 8.86 [02:48:31] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.54, 9.80, 9.97 [02:50:20] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.79, 8.87, 9.77 [02:51:38] PROBLEM - cp31 Stunnel HTTP for mw122 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:51:39] PROBLEM - cp20 Stunnel HTTP for mw121 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:51:41] PROBLEM - cp21 Stunnel HTTP for mw122 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:51:41] PROBLEM - cp21 Stunnel HTTP for mw111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:51:41] PROBLEM - db121 Current Load on db121 is CRITICAL: CRITICAL - load average: 9.55, 6.61, 3.82 [02:51:42] PROBLEM - cp31 Stunnel HTTP for mw121 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:51:42] PROBLEM - cp30 Stunnel HTTP for mw121 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:51:42] PROBLEM - cp21 Stunnel HTTP for mw121 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:52:07] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:52:20] PROBLEM - mw102 MediaWiki Rendering on mw102 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:52:38] PROBLEM - cp20 Stunnel HTTP for mw111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:52:45] PROBLEM - mw122 MediaWiki Rendering on mw122 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:52:51] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:53:06] PROBLEM - cp20 Stunnel HTTP for mw122 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:53:07] PROBLEM - cp30 Stunnel HTTP for mw122 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:53:37] RECOVERY - cp21 Stunnel HTTP for mw121 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 0.980 second response time [02:53:39] RECOVERY - cp20 Stunnel HTTP for mw121 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 0.193 second response time [02:53:39] RECOVERY - cp31 Stunnel HTTP for mw121 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.866 second response time [02:53:40] RECOVERY - cp30 Stunnel HTTP for mw121 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.343 second response time [02:53:46] PROBLEM - cp30 Stunnel HTTP for mw111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:54:05] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 28479 bytes in 0.347 second response time [02:54:20] RECOVERY - mw102 MediaWiki Rendering on mw102 is OK: HTTP OK: HTTP/1.1 200 OK - 28479 bytes in 2.306 second response time [02:54:36] RECOVERY - cp20 Stunnel HTTP for mw111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 3.313 second response time [02:54:39] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 4.34, 2.81, 1.73 [02:54:48] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 28479 bytes in 2.308 second response time [02:55:41] RECOVERY - db121 Current Load on db121 is OK: OK - load average: 4.51, 6.26, 4.40 [02:55:42] RECOVERY - cp21 Stunnel HTTP for mw111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 0.023 second response time [02:55:43] RECOVERY - cp30 Stunnel HTTP for mw111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.336 second response time [02:57:22] RECOVERY - cp20 Stunnel HTTP for mw122 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 8.973 second response time [02:57:22] RECOVERY - cp30 Stunnel HTTP for mw122 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 8.190 second response time [02:57:53] RECOVERY - cp21 Stunnel HTTP for mw122 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 7.519 second response time [02:57:58] RECOVERY - cp31 Stunnel HTTP for mw122 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 8.068 second response time [02:58:18] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.68, 10.01, 9.66 [02:58:51] RECOVERY - mw122 MediaWiki Rendering on mw122 is OK: HTTP OK: HTTP/1.1 200 OK - 28479 bytes in 0.982 second response time [02:59:02] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk [03:00:16] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 9.33, 9.82, 9.65 [03:00:39] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.13, 1.81, 1.64 [03:02:39] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.02, 1.50, 1.54 [03:04:10] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.26, 10.30, 9.89 [03:04:20] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.46, 10.86, 10.04 [03:06:08] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 9.91, 10.03, 9.83 [03:06:20] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.77, 11.41, 10.36 [03:10:20] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.33, 10.14, 10.11 [03:34:20] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.62, 10.70, 9.86 [03:36:20] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.77, 10.26, 9.84 [03:40:20] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.36, 10.00, 9.84 [03:43:31] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.23, 1.58, 1.89 [03:47:31] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.31, 1.85, 1.92 [03:48:20] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.67, 10.03, 9.85 [03:52:20] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.24, 9.02, 9.53 [03:53:31] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.41, 1.95, 1.99 [04:01:31] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.26, 1.84, 1.87 [04:05:31] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.87, 1.83, 1.85 [04:15:31] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.11, 1.78, 1.80 [04:36:14] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.07, 10.55, 9.20 [04:38:11] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.38, 10.73, 9.42 [04:44:00] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.28, 9.80, 9.49 [05:15:31] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.54, 1.77, 2.00 [05:17:31] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.95, 2.32, 2.18 [05:21:44] PROBLEM - cloud11 Puppet on cloud11 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [05:33:31] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.49, 1.69, 1.96 [05:47:31] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.71, 1.96, 1.84 [05:51:44] RECOVERY - cloud11 Puppet on cloud11 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [05:55:31] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.50, 1.88, 1.91 [06:04:10] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.96, 3.57, 3.92 [06:05:31] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 3.03, 2.02, 1.90 [06:10:10] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.18, 3.91, 3.94 [06:12:10] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.71, 3.84, 3.91 [06:14:10] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.96, 4.29, 4.07 [06:22:33] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.42, 9.74, 8.40 [06:22:39] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.96, 1.44, 1.10 [06:24:39] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.95, 1.26, 1.07 [06:26:25] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.77, 10.00, 8.83 [06:38:20] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.67, 10.12, 9.40 [06:40:20] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.52, 9.75, 9.37 [07:02:20] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.17, 9.62, 9.05 [07:03:50] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 5.60, 2.28, 1.45 [07:04:20] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.75, 10.94, 9.59 [07:05:47] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.35, 1.73, 1.34 [07:07:44] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.57, 1.60, 1.33 [07:08:20] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.25, 11.19, 10.05 [07:12:20] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.35, 9.01, 9.45 [07:15:51] PROBLEM - cp31 Stunnel HTTP for mwtask111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:16:20] PROBLEM - mwtask111 MediaWiki Rendering on mwtask111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:16:21] PROBLEM - cp21 Stunnel HTTP for mwtask111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:16:46] PROBLEM - mwtask111 APT on mwtask111 is CRITICAL: CRITICAL - Plugin timed out after 10 seconds [07:16:48] PROBLEM - cp20 Stunnel HTTP for mwtask111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:17:20] PROBLEM - mwtask111 Current Load on mwtask111 is CRITICAL: CRITICAL - load average: 16.57, 8.03, 4.26 [07:17:28] PROBLEM - cp30 Stunnel HTTP for mwtask111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:20:01] PROBLEM - mwtask111 PowerDNS Recursor on mwtask111 is CRITICAL: CRITICAL - Plugin timed out while executing system call [07:20:20] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.23, 10.80, 9.99 [07:20:30] PROBLEM - mwtask111 SSH on mwtask111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:20:48] PROBLEM - mwtask111 Puppet on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [07:21:41] RECOVERY - mwtask111 APT on mwtask111 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [07:21:58] RECOVERY - mwtask111 PowerDNS Recursor on mwtask111 is OK: DNS OK: 0.439 seconds response time. miraheze.org returns 198.244.148.90,2001:41d0:801:2000::1b80,2001:41d0:801:2000::4c25,51.195.220.68 [07:22:01] RECOVERY - cp31 Stunnel HTTP for mwtask111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15956 bytes in 4.363 second response time [07:22:20] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.38, 10.22, 9.88 [07:22:28] RECOVERY - mwtask111 SSH on mwtask111 is OK: SSH OK - OpenSSH_8.4p1 Debian-5+deb11u1 (protocol 2.0) [07:22:31] RECOVERY - cp21 Stunnel HTTP for mwtask111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15956 bytes in 0.041 second response time [07:22:36] RECOVERY - mwtask111 MediaWiki Rendering on mwtask111 is OK: HTTP OK: HTTP/1.1 200 OK - 28487 bytes in 0.929 second response time [07:22:43] RECOVERY - mwtask111 Puppet on mwtask111 is OK: OK: Puppet is currently enabled, last run 29 minutes ago with 0 failures [07:22:57] RECOVERY - cp20 Stunnel HTTP for mwtask111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15956 bytes in 0.029 second response time [07:23:40] RECOVERY - cp30 Stunnel HTTP for mwtask111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15956 bytes in 0.319 second response time [07:24:20] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.78, 9.81, 9.77 [07:35:08] PROBLEM - mwtask111 Current Load on mwtask111 is WARNING: WARNING - load average: 0.34, 1.58, 3.96 [07:35:45] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.45, 1.89, 1.19 [07:37:45] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.76, 1.68, 1.21 [07:38:57] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 0.19, 0.79, 3.12 [07:42:11] PROBLEM - www.fortis.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - www.fortis.wiki All nameservers failed to answer the query. [07:50:39] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.65, 2.41, 1.52 [07:54:39] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.82, 1.58, 1.38 [08:12:06] PROBLEM - www.fortis.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.fortis.wiki could not be found [08:22:10] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.07, 3.61, 3.97 [08:26:10] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.36, 4.14, 4.10 [08:54:31] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.70, 3.38, 4.00 [08:56:31] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 3.64, 3.64, 4.03 [09:04:10] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.80, 3.54, 3.90 [09:04:31] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.87, 3.65, 3.95 [09:08:10] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.05, 3.58, 3.83 [09:08:31] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.17, 3.64, 3.86 [09:10:10] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.01, 3.44, 3.75 [09:10:31] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.21, 3.67, 3.86 [09:12:10] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.02, 3.62, 3.77 [09:12:31] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.65, 4.17, 4.02 [09:16:10] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.94, 3.65, 3.78 [09:20:10] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.31, 3.76, 3.78 [09:22:10] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.05, 3.64, 3.75 [09:24:10] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.77, 4.37, 4.02 [09:26:10] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.68, 3.69, 3.81 [09:30:10] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 7.10, 4.96, 4.25 [09:55:20] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 1.75, 2.05, 1.53 [09:57:14] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.24, 1.82, 1.51 [09:59:08] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 0.72, 1.43, 1.40 [10:35:31] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.21, 1.63, 1.94 [10:45:31] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.55, 1.50, 1.70 [10:49:31] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.79, 2.15, 1.92 [10:53:31] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.31, 1.80, 1.84 [10:57:31] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.03, 1.77, 1.81 [10:59:31] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.68, 1.75, 1.80 [11:17:31] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.33, 2.01, 1.89 [11:38:18] !log [universalomega@mwtask111] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/databases.json /srv/mediawiki/w/maintenance/migrateRevisionActorTemp.php --sleep=5 (END - exit=0) [11:38:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:13:21] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.28, 1.85, 1.48 [12:13:21] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.12, 10.47, 9.63 [12:15:15] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.50, 1.71, 1.47 [12:15:18] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.33, 9.64, 9.43 [12:15:31] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.45, 1.75, 1.98 [12:17:10] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 1.01, 1.56, 1.45 [12:17:31] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.39, 2.07, 2.08 [12:19:12] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.60, 10.82, 9.99 [12:27:02] CosmicAlpha|AWAY: wooooooooo ^ [12:27:31] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.25, 1.72, 2.00 [12:30:50] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.07, 10.88, 10.42 [12:32:46] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.27, 10.64, 10.39 [12:34:43] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.84, 11.27, 10.64 [12:38:35] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.93, 11.00, 10.75 [12:39:31] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.35, 1.44, 1.67 [12:44:24] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.62, 9.36, 10.15 [12:46:20] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.07, 1.77, 1.75 [12:54:20] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.53, 10.10, 10.04 [12:56:20] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.56, 9.31, 9.77 [12:57:47] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.46, 1.96, 1.99 [13:02:58] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 19.72, 10.52, 4.79 [13:07:31] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.57, 2.16, 2.03 [13:18:20] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.24, 10.82, 10.20 [13:20:58] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.01, 1.60, 1.10 [13:22:20] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.20, 10.48, 10.25 [13:22:55] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.14, 1.49, 1.13 [13:27:45] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 3.64, 3.62, 1.95 [13:30:30] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 0.69, 1.22, 1.94 [13:32:20] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.41, 9.68, 10.09 [13:33:46] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.36, 1.56, 1.55 [13:36:17] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.18, 10.57, 9.42 [13:36:20] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.88, 10.66, 10.40 [13:36:30] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 0.88, 1.15, 1.67 [13:36:55] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.52, 3.20, 2.86 [13:38:15] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 9.20, 9.83, 9.27 [13:40:55] RECOVERY - matomo101 Current Load on matomo101 is OK: OK - load average: 2.41, 2.97, 2.87 [13:48:20] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.95, 9.52, 10.03 [13:58:20] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.24, 10.77, 10.18 [14:00:20] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.77, 10.79, 10.26 [14:07:31] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.47, 1.64, 1.93 [14:10:20] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.08, 9.77, 10.11 [14:16:20] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.83, 10.49, 10.29 [14:17:31] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.15, 1.72, 1.80 [14:22:20] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.46, 10.05, 10.18 [14:25:31] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.56, 1.92, 1.93 [14:31:31] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.09, 1.85, 1.88 [14:33:31] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.97, 1.80, 1.85 [14:41:31] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.25, 1.88, 1.85 [14:45:31] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.73, 1.78, 1.81 [14:47:31] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.39, 2.02, 1.90 [14:48:05] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.45, 3.18, 2.97 [14:50:38] PROBLEM - cp20 Stunnel HTTP for mw111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:50:51] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:51:14] PROBLEM - cp31 Stunnel HTTP for mw111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:51:56] RECOVERY - matomo101 Current Load on matomo101 is OK: OK - load average: 2.41, 2.95, 2.94 [14:52:00] PROBLEM - cp21 Stunnel HTTP for mw111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:52:10] PROBLEM - cp30 Stunnel HTTP for mw111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:52:42] RECOVERY - cp20 Stunnel HTTP for mw111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 9.370 second response time [14:52:59] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.58, 2.00, 1.48 [14:54:56] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.61, 1.52, 1.37 [14:54:58] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 28487 bytes in 7.170 second response time [14:55:24] RECOVERY - cp31 Stunnel HTTP for mw111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 7.091 second response time [14:56:02] RECOVERY - cp21 Stunnel HTTP for mw111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 0.820 second response time [14:56:16] RECOVERY - cp30 Stunnel HTTP for mw111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 2.085 second response time [14:57:47] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.55, 10.55, 9.73 [14:59:02] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk [15:01:40] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.56, 9.51, 9.50 [15:02:15] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.95, 1.49, 1.24 [15:04:12] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.39, 1.45, 1.26 [15:04:47] PROBLEM - mw122 MediaWiki Rendering on mw122 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:06:47] RECOVERY - mw122 MediaWiki Rendering on mw122 is OK: HTTP OK: HTTP/1.1 200 OK - 28487 bytes in 4.996 second response time [15:06:56] PROBLEM - cp31 Stunnel HTTP for mw111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:08:53] RECOVERY - cp31 Stunnel HTTP for mw111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.306 second response time [15:09:27] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.65, 10.53, 10.03 [15:11:24] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.22, 11.19, 10.34 [15:13:20] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.32, 10.72, 10.25 [15:19:59] PROBLEM - cp20 Stunnel HTTP for mw111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:20:07] PROBLEM - cp21 Stunnel HTTP for mw111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:21:05] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.41, 9.96, 10.11 [15:21:16] PROBLEM - cp31 Stunnel HTTP for mw111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:21:21] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:21:26] PROBLEM - cp30 Stunnel HTTP for mw111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:23:16] RECOVERY - cp31 Stunnel HTTP for mw111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 3.072 second response time [15:23:20] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 28487 bytes in 4.827 second response time [15:23:26] RECOVERY - cp30 Stunnel HTTP for mw111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 3.002 second response time [15:23:59] RECOVERY - cp20 Stunnel HTTP for mw111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 0.020 second response time [15:24:08] RECOVERY - cp21 Stunnel HTTP for mw111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.026 second response time [15:34:36] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.37, 10.01, 9.02 [15:35:58] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.29, 9.82, 9.32 [15:36:35] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.35, 9.45, 8.94 [15:37:56] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.83, 9.37, 9.21 [15:38:36] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.47, 10.80, 10.27 [15:40:32] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.32, 9.70, 9.93 [15:42:54] RhinosF1: FINALLY!!! Only 3 weeks also. [15:43:26] CosmicAlpha|AWAY: question. Is there a way to find the average time for a wiki request [15:43:31] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.76, 1.80, 2.00 [15:43:38] Easily [15:43:42] To be handled [15:44:04] RhinosF1: I don't know what you mean? [15:45:14] CosmicAlpha|AWAY: the average time from a wiki request being submitted to either being accepted or declined [15:46:06] RhinosF1: well that is not tracked except in logs, so maybe using logs for the request, but no, not easily. [15:47:14] CosmicAlpha|AWAY: ok [15:47:31] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 3.23, 2.23, 2.11 [15:55:31] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.24, 1.76, 1.98 [16:00:30] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.69, 1.66, 1.25 [16:02:30] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 0.82, 1.28, 1.16 [16:04:12] [02miraheze/MirahezeMagic] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-3 [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/commit/bbcf433c94c1 [16:04:14] [02miraheze/MirahezeMagic] 07Universal-Omega 03bbcf433 - CI: fix support for MediaWiki 1.39+ [16:04:15] [02MirahezeMagic] 07Universal-Omega created branch 03Universal-Omega-patch-3 - 13https://github.com/miraheze/MirahezeMagic [16:04:16] [url] Page not found · GitHub · GitHub | github.com [16:04:17] [02MirahezeMagic] 07Universal-Omega opened pull request 03#352: CI: fix support for MediaWiki 1.39+ - 13https://github.com/miraheze/MirahezeMagic/pull/352 [16:04:17] [url] Page not found · GitHub · GitHub | github.com [16:04:54] [02MirahezeMagic] 07Universal-Omega synchronize pull request 03#352: CI: fix support for MediaWiki 1.39+ - 13https://github.com/miraheze/MirahezeMagic/pull/352 [16:04:54] ... [16:04:55] [02miraheze/MirahezeMagic] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-3 [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/bbcf433c94c1...ffd5cda0978d [16:04:56] [url] Comparing bbcf433c94c1...ffd5cda0978d · miraheze/MirahezeMagic · GitHub | github.com [16:04:57] [02miraheze/MirahezeMagic] 07Universal-Omega 03ffd5cda - Update globals.php [16:05:31] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.15, 1.35, 1.64 [16:13:31] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.78, 1.60, 1.62 [16:14:11] miraheze/MirahezeMagic - Universal-Omega the build passed. [16:14:29] [02MirahezeMagic] 07Universal-Omega closed pull request 03#352: CI: fix support for MediaWiki 1.39+ - 13https://github.com/miraheze/MirahezeMagic/pull/352 [16:14:29] [url] Page not found · GitHub · GitHub | github.com [16:14:30] [02miraheze/MirahezeMagic] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/dc0123480194...13ad30653323 [16:14:31] [url] Comparing dc0123480194...13ad30653323 · miraheze/MirahezeMagic · GitHub | github.com [16:14:32] [02miraheze/MirahezeMagic] 07Universal-Omega 0313ad306 - CI: fix support for MediaWiki 1.39+ (#352) [16:14:33] [02miraheze/MirahezeMagic] 07Universal-Omega deleted branch 03Universal-Omega-patch-3 [16:14:35] [02MirahezeMagic] 07Universal-Omega deleted branch 03Universal-Omega-patch-3 - 13https://github.com/miraheze/MirahezeMagic [16:14:35] [url] Page not found · GitHub · GitHub | github.com [16:15:04] [02MirahezeMagic] 07Universal-Omega closed pull request 03#351: Bump actions/cache from 3.0.4 to 3.0.5 - 13https://github.com/miraheze/MirahezeMagic/pull/351 [16:15:05] [url] Page not found · GitHub · GitHub | github.com [16:15:06] [02MirahezeMagic] 07dependabot[bot] commented on pull request 03#351: Bump actions/cache from 3.0.4 to 3.0.5 - 13https://github.com/miraheze/MirahezeMagic/pull/351#issuecomment-1186560512 [16:15:06] [url] Bump actions/cache from 3.0.4 to 3.0.5 by dependabot[bot] · Pull Request #351 · miraheze/MirahezeMagic · GitHub | github.com [16:15:08] [02miraheze/MirahezeMagic] 07Universal-Omega deleted branch 03dependabot/github_actions/actions/cache-3.0.5 [16:15:09] [02MirahezeMagic] 07Universal-Omega deleted branch 03dependabot/github_actions/actions/cache-3.0.5 - 13https://github.com/miraheze/MirahezeMagic [16:15:10] [url] Page not found · GitHub · GitHub | github.com [16:15:23] [02.github] 07Universal-Omega closed pull request 03#13: Bump actions/cache from 3.0.4 to 3.0.5 in /workflow-templates - 13https://github.com/miraheze/.github/pull/13 [16:15:24] [url] Page not found · GitHub · GitHub | github.com [16:15:25] [02miraheze/.github] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/.github/compare/269d4fbb7e8c...d35934b582bd [16:15:25] [url] Comparing 269d4fbb7e8c...d35934b582bd · miraheze/.github · GitHub | github.com [16:15:26] [02miraheze/.github] 07dependabot[bot] 03d35934b - Bump actions/cache from 3.0.4 to 3.0.5 in /workflow-templates (#13) [16:15:28] [02miraheze/.github] 07Universal-Omega deleted branch 03dependabot/github_actions/workflow-templates/actions/cache-3.0.5 [16:15:29] [02.github] 07Universal-Omega deleted branch 03dependabot/github_actions/workflow-templates/actions/cache-3.0.5 - 13https://github.com/miraheze/.github [16:15:31] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.41, 1.56, 1.61 [16:15:41] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/e80a736f9672...62f7e590ab14 [16:15:42] [url] Comparing e80a736f9672...62f7e590ab14 · miraheze/ImportDump · GitHub | github.com [16:15:43] [02miraheze/ImportDump] 07Universal-Omega 0362f7e59 - Update workflow [16:15:44] [02miraheze/MatomoAnalytics] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/MatomoAnalytics/compare/5faa403495e0...3d550db96fc5 [16:15:45] [url] Comparing 5faa403495e0...3d550db96fc5 · miraheze/MatomoAnalytics · GitHub | github.com [16:15:46] [02miraheze/MatomoAnalytics] 07Universal-Omega 033d550db - Update workflow [16:15:48] [02miraheze/GlobalNewFiles] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/GlobalNewFiles/compare/53a897b52603...8a793af2c581 [16:15:48] [url] Comparing 53a897b52603...8a793af2c581 · miraheze/GlobalNewFiles · GitHub | github.com [16:15:49] [02miraheze/GlobalNewFiles] 07Universal-Omega 038a793af - Update workflow [16:15:50] [02miraheze/DataDump] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/DataDump/compare/58eedfa37b2f...32a3651646ad [16:15:51] [url] Comparing 58eedfa37b2f...32a3651646ad · miraheze/DataDump · GitHub | github.com [16:15:52] [02miraheze/DataDump] 07Universal-Omega 0332a3651 - Update workflow [16:15:54] [02miraheze/ManageWiki] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ManageWiki/compare/202eaec2d240...aad10605e817 [16:15:54] [url] Comparing 202eaec2d240...aad10605e817 · miraheze/ManageWiki · GitHub | github.com [16:15:55] [02miraheze/ManageWiki] 07Universal-Omega 03aad1060 - Update workflow [16:15:57] [02miraheze/CreateWiki] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/CreateWiki/compare/33899c0228fd...81917ce5e6be [16:15:57] [url] Comparing 33899c0228fd...81917ce5e6be · miraheze/CreateWiki · GitHub | github.com [16:15:58] [02miraheze/CreateWiki] 07Universal-Omega 0381917ce - Update workflow [16:15:59]