[00:00:08] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.10, 1.70, 1.67 [00:02:07] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.80, 1.83, 1.73 [00:02:54] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.57, 10.93, 10.37 [00:03:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.14, 11.38, 11.76 [00:04:06] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.43, 1.94, 1.77 [00:04:34] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.20, 1.76, 1.32 [00:04:47] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 13.18, 11.25, 10.47 [00:04:52] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.48, 10.08, 10.14 [00:05:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.74, 11.84, 11.87 [00:06:05] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.84, 1.58, 1.67 [00:06:32] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.90, 1.43, 1.25 [00:06:47] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.05, 10.63, 10.33 [00:07:24] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:07:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.82, 11.01, 11.57 [00:08:47] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 6.43, 9.37, 9.92 [00:09:34] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.08, 9.34, 10.06 [00:09:38] PROBLEM - cp20 Stunnel HTTP for mw122 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:10:40] PROBLEM - cp20 Stunnel HTTP for mw111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:10:48] PROBLEM - cp31 Stunnel HTTP for mw111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:10:52] PROBLEM - cp30 Stunnel HTTP for mw111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:10:52] PROBLEM - cp21 Stunnel HTTP for mw111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:11:38] RECOVERY - cp20 Stunnel HTTP for mw122 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 6.471 second response time [00:12:58] RECOVERY - cp30 Stunnel HTTP for mw111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 9.158 second response time [00:13:45] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 28479 bytes in 8.711 second response time [00:14:20] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.88, 1.59, 1.32 [00:15:34] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.63, 10.21, 10.28 [00:16:58] RECOVERY - cp20 Stunnel HTTP for mw111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 7.859 second response time [00:17:09] RECOVERY - cp31 Stunnel HTTP for mw111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 6.074 second response time [00:17:10] RECOVERY - cp21 Stunnel HTTP for mw111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 6.114 second response time [00:18:16] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.68, 1.23, 1.23 [00:19:57] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.19, 1.60, 1.50 [00:21:56] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.94, 1.67, 1.53 [00:23:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.34, 11.69, 11.34 [00:25:54] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.95, 1.49, 1.52 [00:27:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.87, 11.04, 11.25 [00:29:34] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 9.29, 9.98, 10.19 [00:31:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.76, 11.77, 11.48 [00:32:13] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 13.35, 11.30, 10.09 [00:33:34] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.46, 10.52, 10.34 [00:34:10] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.81, 10.85, 10.09 [00:35:41] PROBLEM - cp20 Stunnel HTTP for mw111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:35:53] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:35:57] PROBLEM - cp30 Stunnel HTTP for mw111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:36:03] PROBLEM - cp21 Stunnel HTTP for mw111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:36:05] PROBLEM - cp31 Stunnel HTTP for mw111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:36:08] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.15, 10.20, 9.94 [00:37:34] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 9.38, 9.99, 10.17 [00:39:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.75, 11.33, 11.51 [00:44:06] RECOVERY - cp20 Stunnel HTTP for mw111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 6.289 second response time [00:44:11] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 28480 bytes in 8.186 second response time [00:44:21] RECOVERY - cp30 Stunnel HTTP for mw111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 5.024 second response time [00:44:24] RECOVERY - cp31 Stunnel HTTP for mw111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 5.858 second response time [00:44:27] RECOVERY - cp21 Stunnel HTTP for mw111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 4.112 second response time [00:49:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.67, 11.93, 11.44 [00:50:04] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.96, 10.22, 9.62 [00:50:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.25, 9.95, 9.81 [00:51:03] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.41, 10.58, 9.57 [00:52:04] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.27, 10.77, 9.89 [00:53:01] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.09, 10.46, 9.64 [00:54:04] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.63, 10.47, 9.87 [00:54:21] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.47, 11.37, 10.43 [00:55:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.95, 11.98, 11.67 [00:56:37] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.45, 10.65, 9.81 [00:56:56] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 9.29, 9.71, 9.52 [00:57:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.51, 12.03, 11.71 [01:00:04] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 13.46, 11.48, 10.43 [01:00:35] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 13.32, 11.21, 10.15 [01:00:41] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.72, 11.44, 9.87 [01:02:45] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.34, 10.68, 9.97 [01:04:04] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.44, 11.99, 10.93 [01:04:35] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.84, 11.28, 10.14 [01:04:43] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 8.94, 9.88, 9.75 [01:06:13] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.50, 1.97, 1.57 [01:08:10] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.24, 1.62, 1.49 [01:08:32] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.83, 11.64, 10.94 [01:10:04] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 13.25, 12.19, 11.33 [01:10:26] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.34, 11.20, 10.40 [01:11:59] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.69, 11.27, 11.48 [01:12:04] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.44, 11.83, 11.30 [01:12:24] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.72, 11.25, 10.51 [01:12:31] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 14.52, 12.64, 11.46 [01:14:04] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.62, 11.98, 11.40 [01:14:21] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.42, 11.44, 10.65 [01:15:54] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.36, 12.38, 11.86 [01:20:12] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.95, 10.88, 10.72 [01:22:04] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.81, 11.69, 11.67 [01:23:44] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.41, 11.56, 11.76 [01:24:07] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.11, 9.42, 10.16 [01:24:26] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.43, 11.81, 11.91 [01:25:42] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.88, 12.09, 11.93 [01:26:25] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 13.16, 12.05, 11.96 [01:27:19] [02mw-config] 07LuciferianThomas synchronize pull request 03#4822: Update $wgTranslateBlacklist for hkrailwiki - 13https://github.com/miraheze/mw-config/pull/4822 [01:27:20] [url] Page not found · GitHub · GitHub | github.com [01:27:39] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.49, 11.66, 11.78 [01:27:54] [02mw-config] 07LuciferianThomas edited pull request 03#4822: Update $wgTranslateDisabledTargetLanguages for hkrailwiki - 13https://github.com/miraheze/mw-config/pull/4822 [01:27:54] [url] Page not found · GitHub · GitHub | github.com [01:28:24] miraheze/mw-config - LuciferianThomas the build passed. [01:28:25] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.09, 11.28, 11.69 [01:30:04] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 13.13, 11.20, 11.28 [01:30:38] [02mw-config] 07Universal-Omega closed pull request 03#4823: Rename `$wgTranslateBlacklist` - 13https://github.com/miraheze/mw-config/pull/4823 [01:30:38] [url] Page not found · GitHub · GitHub | github.com [01:30:39] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/c5e339b7486c...8f8ffa41704f [01:30:40] [url] Comparing c5e339b7486c...8f8ffa41704f · miraheze/mw-config · GitHub | github.com [01:30:41] [02miraheze/mw-config] 07Universal-Omega 038f8ffa4 - Rename `$wgTranslateBlacklist` (#4823) [01:30:42] [02miraheze/mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-4 [01:30:44] [02mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-4 - 13https://github.com/miraheze/mw-config [01:30:44] ... [01:31:34] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.59, 11.84, 11.81 [01:31:45] [02mw-config] 07Universal-Omega synchronize pull request 03#4822: Update $wgTranslateDisabledTargetLanguages for hkrailwiki - 13https://github.com/miraheze/mw-config/pull/4822 [01:31:46] ... [01:31:48] miraheze/mw-config - Universal-Omega the build passed. [01:32:04] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.81, 11.16, 11.27 [01:32:53] miraheze/mw-config - Universal-Omega the build passed. [01:35:14] !log [@test101] starting deploy of {'config': True} to all [01:35:16] !log [@test101] finished deploy of {'config': True} to all - SUCCESS in 3s [01:35:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:35:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:40:38] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.88, 10.40, 10.19 [01:41:34] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.72, 11.63, 11.98 [01:43:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.65, 10.89, 11.88 [01:46:30] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 9.21, 9.83, 10.09 [01:49:14] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 1.47, 2.15, 1.76 [01:50:04] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.07, 9.31, 9.99 [01:51:13] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.13, 1.81, 1.68 [01:51:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.68, 10.95, 11.34 [01:53:12] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.58, 2.04, 1.77 [01:53:59] !log [@mwtask111] starting deploy of {'config': True} to all [01:54:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:54:16] !log [@mwtask111] finished deploy of {'config': True} to all - SUCCESS in 17s [01:54:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:55:11] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.82, 1.56, 1.62 [02:00:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.67, 3.86, 4.00 [02:01:04] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.24, 9.75, 9.43 [02:01:34] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.31, 10.95, 10.90 [02:02:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 3.71, 4.02, 4.06 [02:02:52] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.49, 10.81, 10.30 [02:03:01] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 9.16, 9.67, 9.45 [02:03:40] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.97, 3.47, 3.90 [02:03:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.72, 11.93, 11.91 [02:05:34] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.27, 11.58, 11.19 [02:05:40] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.15, 3.82, 3.96 [02:05:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.18, 11.94, 11.91 [02:06:47] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.91, 9.49, 9.88 [02:09:34] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.18, 11.49, 11.22 [02:09:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.32, 11.75, 11.88 [02:11:34] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.49, 11.18, 11.15 [02:12:08] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 12.06, 11.26, 10.94 [02:12:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.08, 3.74, 3.96 [02:14:07] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.61, 10.55, 10.72 [02:18:06] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.28, 9.11, 10.08 [02:18:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.25, 3.35, 4.00 [02:18:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.20, 3.70, 3.85 [02:20:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.14, 3.58, 4.00 [02:22:12] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.42, 3.50, 3.92 [02:22:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.52, 3.51, 3.73 [02:23:41] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.02, 3.59, 3.94 [02:24:32] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk [02:26:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.06, 3.49, 3.80 [02:26:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.28, 3.32, 3.55 [02:27:34] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 9.21, 9.27, 10.12 [02:27:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.52, 10.89, 10.66 [02:28:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.94, 3.85, 3.91 [02:29:52] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.05, 9.41, 10.14 [02:30:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.41, 3.66, 3.68 [02:32:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.39, 3.89, 3.89 [02:33:34] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.54, 10.28, 10.23 [02:33:41] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.16, 4.50, 4.08 [02:34:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.05, 3.62, 3.63 [02:34:48] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 1.92, 2.13, 1.71 [02:35:34] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.88, 9.95, 10.13 [02:38:47] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.98, 1.56, 1.58 [02:40:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.82, 3.98, 3.83 [02:41:41] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.88, 3.85, 3.97 [02:42:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.03, 3.99, 3.85 [02:44:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.98, 3.72, 3.78 [02:46:40] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.34, 9.76, 9.57 [02:48:04] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.19, 10.04, 9.56 [02:48:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.86, 3.70, 4.00 [02:48:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.78, 10.72, 10.27 [02:49:40] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.25, 2.75, 3.40 [02:50:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.16, 3.85, 3.99 [02:50:24] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 9.44, 9.96, 10.03 [02:50:26] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.88, 1.72, 1.47 [02:50:36] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 10.10, 10.13, 9.78 [02:50:47] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.60, 2.00, 1.72 [02:52:04] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.46, 10.17, 9.75 [02:52:24] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.39, 1.63, 1.47 [02:52:47] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.69, 1.87, 1.71 [02:54:30] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.10, 10.49, 10.04 [02:54:47] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.13, 1.87, 1.72 [02:56:04] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.31, 10.24, 9.87 [02:56:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.64, 3.57, 3.83 [02:56:18] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.25, 3.18, 3.40 [02:56:47] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.43, 1.68, 1.67 [02:57:48] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.08, 10.19, 9.61 [02:58:04] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.08, 9.37, 9.59 [02:58:34] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.20, 3.34, 3.42 [02:59:47] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 12.99, 10.85, 9.89 [03:00:33] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.23, 3.50, 3.46 [03:01:46] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.08, 10.88, 10.02 [03:02:47] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.59, 2.05, 1.80 [03:03:46] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.48, 9.89, 9.76 [03:04:04] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 13.03, 11.17, 10.24 [03:04:30] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.83, 3.53, 3.50 [03:05:59] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.83, 10.61, 10.15 [03:06:18] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.11, 11.75, 10.79 [03:06:47] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.50, 1.92, 1.83 [03:08:04] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.08, 11.02, 10.42 [03:08:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.63, 3.57, 3.58 [03:08:26] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.56, 3.75, 3.55 [03:08:45] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.20, 3.50, 3.43 [03:10:15] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.54, 11.21, 10.79 [03:10:25] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.36, 3.59, 3.52 [03:10:47] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.16, 1.50, 1.68 [03:11:41] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.38, 10.95, 10.27 [03:12:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.35, 3.64, 3.62 [03:12:23] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.10, 3.88, 3.62 [03:12:32] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.06, 3.84, 3.64 [03:13:49] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.30, 11.07, 10.54 [03:16:13] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.25, 3.04, 3.35 [03:17:44] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.81, 10.66, 10.51 [03:18:04] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.26, 9.63, 10.04 [03:18:19] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.80, 3.93, 3.77 [03:20:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.10, 3.65, 3.55 [03:20:17] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.53, 4.04, 3.81 [03:20:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.69, 3.73, 3.58 [03:21:37] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.82, 9.35, 9.96 [03:22:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.52, 3.44, 3.50 [03:24:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 6.15, 4.35, 3.81 [03:25:34] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.56, 9.05, 9.89 [03:26:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.92, 3.72, 3.68 [03:28:47] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 4.20, 3.43, 2.36 [03:30:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.89, 3.91, 3.84 [03:34:06] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.27, 3.85, 3.97 [03:36:18] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.22, 3.14, 3.36 [03:37:40] [02mw-config] 07AgentIsai opened pull request 03#4824: Change web badge for anniversary - 13https://github.com/miraheze/mw-config/pull/4824 [03:37:40] [url] Page not found · GitHub · GitHub | github.com [03:38:13] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.10, 2.61, 3.25 [03:38:21] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.85, 10.41, 10.13 [03:38:41] miraheze/mw-config - AgentIsai the build passed. [03:38:47] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.01, 1.79, 2.00 [03:40:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.79, 3.78, 3.54 [03:40:19] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.54, 9.90, 10.00 [03:42:00] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.23, 3.85, 3.88 [03:42:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.51, 3.28, 3.41 [03:43:59] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.60, 3.70, 3.82 [03:44:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.99, 3.77, 3.56 [03:44:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.33, 3.92, 3.68 [03:44:47] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.72, 1.11, 1.62 [03:45:57] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.46, 4.07, 3.94 [03:46:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.91, 3.59, 3.53 [03:46:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.14, 4.24, 3.84 [03:47:52] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.18, 9.74, 10.16 [03:47:55] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.67, 3.69, 3.81 [03:48:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.64, 3.84, 3.75 [03:48:47] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.09, 1.80, 1.79 [03:49:54] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.52, 4.09, 3.91 [03:50:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 6.28, 4.44, 3.84 [03:50:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.71, 4.17, 3.87 [03:50:47] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.80, 1.92, 1.85 [03:51:52] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.24, 3.97, 3.91 [03:52:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.91, 3.84, 3.69 [03:52:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.78, 3.85, 3.78 [03:54:47] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.22, 1.48, 1.67 [04:01:45] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.05, 3.57, 3.65 [04:01:46] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.29, 9.79, 9.65 [04:02:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.17, 3.54, 3.54 [04:03:43] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.06, 3.01, 3.43 [04:04:13] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.56, 3.14, 3.36 [04:04:18] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.43, 3.11, 3.38 [04:04:53] [02mw-config] 07paladox closed pull request 03#4824: Change web badge for anniversary - 13https://github.com/miraheze/mw-config/pull/4824 [04:04:54] [url] Page not found · GitHub · GitHub | github.com [04:04:55] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/8f8ffa41704f...c174173b5777 [04:04:55] [url] Comparing 8f8ffa41704f...c174173b5777 · miraheze/mw-config · GitHub | github.com [04:04:56] [02miraheze/mw-config] 07AgentIsai 03c174173 - Change web badge for anniversary (#4824) [04:05:41] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.98, 2.77, 3.30 [04:06:04] miraheze/mw-config - paladox the build passed. [04:09:37] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 10.18, 10.04, 9.90 [04:17:42] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.45, 3.11, 3.13 [04:20:02] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk [04:20:34] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.98, 3.54, 3.24 [04:21:29] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.40, 3.64, 3.32 [04:23:10] !log [@mwtask111] starting deploy of {'config': True} to all [04:23:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:23:23] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.46, 3.20, 3.20 [04:23:23] !log [@mwtask111] finished deploy of {'config': True} to all - SUCCESS in 13s [04:23:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:24:14] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.14, 10.29, 9.59 [04:25:02] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk [04:26:12] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.68, 9.86, 9.53 [04:29:03] !log [@test101] starting deploy of {'config': True} to all [04:29:05] !log [@test101] finished deploy of {'config': True} to all - SUCCESS in 2s [04:29:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:29:12] PROBLEM - swests.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'swests.org' expires in 15 day(s) (Sun 07 Aug 2022 04:20:05 GMT +0000). [04:29:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:30:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.80, 3.78, 3.59 [04:32:05] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.06, 10.66, 9.93 [04:32:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.46, 4.00, 3.69 [04:33:46] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/291d02814beb...54a4e2dd6dcd [04:33:47] [url] Comparing 291d02814beb...54a4e2dd6dcd · miraheze/ssl · GitHub | github.com [04:33:48] [02miraheze/ssl] 07MirahezeSSLBot 0354a4e2d - Bot: Update SSL cert for swests.org [04:34:03] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.86, 10.44, 9.92 [04:34:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.33, 3.65, 3.59 [04:34:22] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.22, 3.88, 3.51 [04:36:20] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.68, 3.44, 3.39 [04:38:13] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.03, 3.07, 3.36 [04:38:36] PROBLEM - wiki.ameristraliagov.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.ameristraliagov.com' expires in 15 day(s) (Sun 07 Aug 2022 04:25:08 GMT +0000). [04:39:57] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.74, 10.07, 9.94 [04:40:24] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.24, 3.74, 3.49 [04:42:16] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.11, 3.22, 3.34 [04:44:18] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.90, 3.27, 3.34 [04:45:50] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/54a4e2dd6dcd...9f62b492559a [04:45:50] [url] Comparing 54a4e2dd6dcd...9f62b492559a · miraheze/ssl · GitHub | github.com [04:45:51] [02miraheze/ssl] 07MirahezeSSLBot 039f62b49 - Bot: Update SSL cert for wiki.ameristraliagov.com [04:51:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.06, 10.30, 9.86 [04:52:05] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.68, 3.55, 3.44 [04:52:40] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 6.09, 4.35, 3.59 [04:54:03] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 7.36, 5.18, 4.06 [04:56:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.32, 3.48, 3.33 [04:56:29] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.22, 3.86, 3.58 [04:58:00] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.46, 3.97, 3.79 [04:58:18] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.35, 3.20, 3.25 [04:58:53] RECOVERY - swests.org - LetsEncrypt on sslhost is OK: OK - Certificate 'swests.org' will expire on Thu 20 Oct 2022 03:33:41 GMT +0000. [05:02:13] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.51, 3.14, 3.38 [05:02:18] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.60, 10.77, 9.44 [05:03:55] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.22, 2.70, 3.33 [05:04:15] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.71, 9.43, 9.10 [05:07:43] RECOVERY - wiki.ameristraliagov.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.ameristraliagov.com' will expire on Thu 20 Oct 2022 03:45:43 GMT +0000. [05:07:52] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.88, 9.93, 10.18 [05:09:49] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.50, 3.19, 3.30 [05:11:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.14, 11.03, 10.60 [05:13:46] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.71, 2.96, 3.18 [05:16:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.34, 3.79, 3.46 [05:20:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.93, 3.82, 3.55 [05:24:13] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.84, 3.05, 3.31 [05:28:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.21, 3.85, 3.57 [05:29:52] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.25, 8.60, 9.83 [05:30:13] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.22, 3.26, 3.39 [05:31:28] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.16, 3.47, 3.32 [05:33:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.22, 3.95, 3.52 [05:35:25] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.08, 3.51, 3.40 [05:37:24] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.65, 3.26, 3.33 [05:44:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.22, 3.44, 3.32 [05:45:36] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.62, 3.37, 3.15 [05:46:13] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.71, 3.14, 3.22 [05:47:30] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.57, 3.48, 3.20 [05:49:23] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.49, 3.39, 3.20 [05:51:17] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.36, 3.03, 3.09 [05:52:09] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.26, 3.54, 3.38 [05:54:08] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.53, 3.17, 3.26 [05:55:04] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.62, 3.34, 3.20 [05:56:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.87, 3.27, 3.19 [05:58:51] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.89, 4.49, 3.66 [06:00:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.65, 3.61, 3.31 [06:02:13] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.06, 3.38, 3.26 [06:04:32] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.65, 3.92, 3.67 [06:06:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.14, 3.83, 3.49 [06:06:44] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.43, 9.61, 8.91 [06:08:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.22, 3.52, 3.42 [06:08:42] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.33, 9.36, 8.90 [06:10:18] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.17, 2.89, 3.27 [06:11:57] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.46, 9.83, 8.97 [06:13:57] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.53, 8.99, 8.77 [06:14:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.76, 4.31, 3.76 [06:14:34] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.42, 10.30, 9.38 [06:14:46] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.52, 4.17, 3.57 [06:16:32] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.02, 9.66, 9.28 [06:18:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.28, 3.77, 3.67 [06:18:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.63, 3.60, 3.44 [06:18:43] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.88, 3.82, 3.59 [06:20:18] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.94, 3.39, 3.38 [06:22:24] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.95, 10.92, 9.98 [06:27:05] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.74, 3.97, 3.62 [06:28:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.01, 4.10, 3.82 [06:28:58] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.53, 3.95, 3.67 [06:30:33] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.99, 2.83, 3.22 [06:32:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.73, 3.62, 3.71 [06:34:12] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.25, 11.21, 10.62 [06:36:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.02, 3.85, 3.55 [06:36:33] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.00, 3.03, 3.38 [06:38:09] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.50, 11.78, 11.04 [06:38:25] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.83, 3.59, 3.50 [06:40:13] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.34, 2.99, 3.40 [06:40:24] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.11, 3.25, 3.39 [06:48:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.64, 4.15, 3.62 [06:48:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.77, 3.86, 3.56 [06:49:57] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 10.03, 9.46, 10.15 [06:50:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.83, 3.65, 3.50 [06:52:13] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.72, 3.32, 3.39 [06:52:13] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.70, 3.20, 3.35 [06:57:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.58, 10.68, 10.42 [06:58:07] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.37, 4.30, 3.76 [06:58:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 6.84, 5.21, 4.11 [06:58:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.34, 4.88, 3.86 [07:04:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.42, 3.91, 3.93 [07:04:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.29, 3.48, 3.60 [07:06:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 6.02, 4.90, 4.29 [07:07:52] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.85, 9.75, 10.17 [07:07:59] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.15, 3.81, 3.78 [07:10:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.29, 3.66, 3.90 [07:10:18] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.64, 2.99, 3.36 [07:11:56] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.48, 2.85, 3.40 [07:15:51] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.59, 3.38, 3.50 [07:16:12] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.08, 2.72, 3.40 [07:17:49] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.34, 3.06, 3.38 [07:26:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.71, 3.21, 3.25 [07:26:15] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.30, 9.30, 8.67 [07:28:13] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.42, 8.88, 8.59 [07:30:13] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.71, 3.33, 3.31 [07:30:37] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.24, 3.44, 3.37 [07:34:33] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.23, 3.01, 3.23 [07:36:00] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.68, 10.19, 9.31 [07:36:44] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.67, 10.63, 10.14 [07:39:55] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.77, 9.81, 9.37 [07:40:40] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.34, 10.04, 10.02 [07:48:25] [02mw-config] 07Universal-Omega closed pull request 03#4822: Update $wgTranslateDisabledTargetLanguages for hkrailwiki - 13https://github.com/miraheze/mw-config/pull/4822 [07:48:26] [url] Page not found · GitHub · GitHub | github.com [07:48:27] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/c174173b5777...523aefbd1880 [07:48:27] [url] Comparing c174173b5777...523aefbd1880 · miraheze/mw-config · GitHub | github.com [07:48:28] [02miraheze/mw-config] 07LuciferianThomas 03523aefb - Update $wgTranslateDisabledTargetLanguages for hkrailwiki (#4822) [07:49:42] miraheze/mw-config - Universal-Omega the build passed. [07:51:15] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.11, 1.67, 1.18 [07:52:55] !log [@mwtask111] starting deploy of {'config': True} to all [07:53:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [07:53:03] !log [@mwtask111] finished deploy of {'config': True} to all - SUCCESS in 8s [07:53:09] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.84, 1.63, 1.22 [07:53:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [07:54:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.77, 3.34, 3.09 [07:54:15] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.05, 3.67, 3.36 [07:55:04] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 1.42, 1.55, 1.24 [07:56:13] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.99, 3.40, 3.29 [07:58:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.99, 4.64, 3.66 [07:58:20] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.04, 10.21, 9.60 [08:00:18] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.47, 10.28, 9.71 [08:02:07] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.68, 4.08, 3.55 [08:03:52] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.60, 3.68, 3.33 [08:03:54] !log [@test101] starting deploy of {'config': True} to all [08:03:55] !log [@test101] finished deploy of {'config': True} to all - SUCCESS in 1s [08:04:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [08:04:05] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.77, 3.58, 3.43 [08:04:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [08:05:46] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.21, 3.80, 3.40 [08:06:04] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.25, 3.24, 3.33 [08:06:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 1.73, 3.31, 3.49 [08:07:39] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.97, 3.49, 3.33 [08:08:10] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.26, 9.88, 9.81 [08:08:13] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.68, 3.07, 3.37 [08:09:33] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.31, 3.58, 3.37 [08:11:27] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.44, 3.40, 3.32 [08:13:20] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.92, 3.98, 3.54 [08:13:56] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.07, 4.31, 3.65 [08:15:14] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.24, 3.98, 3.61 [08:15:54] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.52, 3.65, 3.49 [08:16:24] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.59, 10.37, 9.84 [08:19:01] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.49, 2.91, 3.27 [08:20:19] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.66, 9.82, 9.74 [08:22:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.32, 10.39, 10.09 [08:23:48] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.70, 3.25, 3.38 [08:26:48] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.31, 9.98, 10.02 [08:46:28] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.26, 3.43, 3.14 [08:48:26] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.82, 3.26, 3.12 [08:56:52] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.02, 3.67, 3.07 [08:58:45] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.75, 3.32, 3.02 [08:59:28] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.53, 4.04, 3.47 [09:01:23] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.84, 3.24, 3.24 [09:03:12] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.81, 3.46, 3.17 [09:05:10] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.32, 3.07, 3.07 [09:05:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.55, 3.42, 3.36 [09:06:59] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.42, 10.23, 9.77 [09:08:57] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.85, 10.01, 9.74 [09:09:01] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.72, 2.84, 3.17 [09:26:42] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.53, 1.66, 1.24 [09:28:42] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 2.00, 1.60, 1.26 [09:30:42] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 1.36, 1.41, 1.22 [09:54:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.90, 3.99, 3.05 [09:56:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.25, 3.99, 3.47 [10:00:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.02, 3.78, 3.53 [10:00:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.19, 3.75, 3.28 [10:02:13] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.29, 3.26, 3.37 [10:02:18] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.15, 3.33, 3.19 [10:04:56] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.55, 11.17, 9.95 [10:06:54] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.13, 11.09, 10.08 [10:08:52] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.00, 10.20, 9.89 [10:12:15] PROBLEM - famedata.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:14:42] PROBLEM - cp21 Varnish Backends on cp21 is CRITICAL: 5 backends are down. mw101 mw111 mw112 mw121 mw122 [10:15:12] PROBLEM - cp21 Stunnel HTTP for mw102 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:15:45] PROBLEM - cp31 Stunnel HTTP for mw102 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:16:08] PROBLEM - cp30 Stunnel HTTP for mw102 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:17:08] RECOVERY - cp21 Stunnel HTTP for mw102 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 0.019 second response time [10:17:43] RECOVERY - cp31 Stunnel HTTP for mw102 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.342 second response time [10:17:47] PROBLEM - cp31 Varnish Backends on cp31 is CRITICAL: 1 backends are down. mw102 [10:18:08] RECOVERY - cp30 Stunnel HTTP for mw102 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.329 second response time [10:18:42] RECOVERY - cp21 Varnish Backends on cp21 is OK: All 14 backends are healthy [10:19:47] RECOVERY - cp31 Varnish Backends on cp31 is OK: All 14 backends are healthy [10:32:45] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.04, 3.45, 3.20 [10:34:39] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.99, 2.92, 3.04 [10:42:10] RECOVERY - famedata.org - LetsEncrypt on sslhost is OK: OK - Certificate 'famedata.org' will expire on Thu 29 Sep 2022 07:42:43 GMT +0000. [10:46:00] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.27, 3.39, 2.83 [10:47:58] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.97, 3.20, 2.82 [10:50:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.31, 3.50, 3.07 [10:52:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.43, 3.55, 3.12 [10:54:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.16, 3.41, 3.12 [10:54:13] [02mw-config] 07ugochimobi opened pull request 03#4825: T9561: set wgMinervaApplyKnownTemplateHacks to false and configure wgMFRemovableClasses - 13https://github.com/miraheze/mw-config/pull/4825 [10:54:14] [url] Page not found · GitHub · GitHub | github.com [10:55:18] miraheze/mw-config - ugochimobi the build passed. [10:56:13] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.13, 3.01, 3.01 [11:26:42] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.44, 1.93, 1.55 [11:26:47] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.40, 2.78, 1.56 [11:28:42] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 1.12, 1.58, 1.46 [11:30:47] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.06, 1.81, 1.43 [11:32:47] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.53, 1.35, 1.31 [11:35:40] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 6.03, 4.37, 3.49 [11:36:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.52, 3.78, 3.21 [11:37:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.62, 3.66, 3.34 [11:38:18] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.69, 3.35, 3.12 [11:43:18] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.95, 3.39, 3.31 [11:47:30] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.44, 9.68, 8.99 [11:48:26] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.27, 3.79, 3.31 [11:49:28] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.92, 9.91, 9.17 [11:50:25] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.25, 3.67, 3.33 [11:52:23] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.74, 3.11, 3.14 [11:58:03] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 1.45, 2.54, 1.56 [11:58:16] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.34, 10.53, 9.79 [11:58:43] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.86, 1.78, 1.41 [12:00:14] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.51, 9.84, 9.63 [12:02:03] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.01, 1.77, 1.47 [12:02:42] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 3.34, 2.19, 1.64 [12:04:01] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.52, 10.75, 9.72 [12:04:42] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.89, 1.95, 1.61 [12:05:05] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 12.27, 10.09, 8.84 [12:05:59] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.89, 9.70, 9.50 [12:06:03] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.30, 1.53, 1.43 [12:06:06] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.90, 11.34, 10.20 [12:07:04] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.51, 9.29, 8.69 [12:08:42] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.01, 1.91, 1.66 [12:10:02] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.92, 12.00, 10.76 [12:10:42] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 0.93, 1.67, 1.61 [12:12:00] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.58, 12.29, 11.02 [12:13:01] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.28, 10.31, 9.31 [12:15:00] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.40, 9.97, 9.31 [12:15:57] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.43, 11.74, 11.13 [12:17:02] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.16, 3.81, 3.32 [12:20:51] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.65, 3.10, 3.17 [12:21:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.91, 11.84, 11.29 [12:23:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.68, 11.97, 11.42 [12:25:48] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.48, 3.40, 3.08 [12:26:47] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.41, 1.84, 1.29 [12:28:47] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.64, 1.39, 1.18 [12:29:36] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.53, 3.64, 3.26 [12:29:55] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.69, 1.71, 1.53 [12:31:21] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.25, 3.44, 3.32 [12:31:29] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.27, 3.40, 3.24 [12:31:49] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 4.30, 2.48, 1.82 [12:33:16] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.08, 3.38, 3.32 [12:35:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.34, 11.26, 11.04 [12:37:32] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 1.20, 1.68, 1.66 [12:37:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.11, 11.17, 11.04 [12:38:24] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.28, 10.21, 9.73 [12:39:32] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk [12:41:40] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.92, 1.66, 1.18 [12:41:41] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.52, 3.32, 3.09 [12:42:19] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 9.45, 9.87, 9.69 [12:43:40] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.99, 1.51, 1.19 [12:43:40] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.18, 3.58, 3.21 [12:45:40] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.92, 3.48, 3.20 [12:47:40] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.28, 3.05, 3.08 [12:53:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.98, 11.52, 11.07 [12:57:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.78, 11.73, 11.30 [12:59:32] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk [13:03:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.83, 11.66, 11.32 [13:06:28] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.69, 3.75, 3.33 [13:08:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.06, 3.70, 3.23 [13:08:26] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.25, 3.55, 3.31 [13:09:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.03, 11.89, 11.63 [13:10:12] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.38, 3.20, 3.10 [13:10:37] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.15, 10.45, 9.90 [13:11:57] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.57, 10.91, 9.35 [13:12:23] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.20, 3.76, 3.44 [13:12:34] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.72, 11.30, 10.28 [13:13:57] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.62, 10.41, 9.36 [13:14:32] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.86, 10.92, 10.28 [13:15:57] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.22, 9.61, 9.20 [13:16:20] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.03, 3.59, 3.46 [13:16:29] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.37, 10.19, 10.09 [13:17:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.90, 11.50, 11.41 [13:19:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.81, 11.33, 11.38 [13:20:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.43, 3.47, 3.25 [13:20:17] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.17, 3.91, 3.62 [13:20:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.59, 3.43, 3.02 [13:20:22] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.15, 11.04, 10.45 [13:22:13] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.04, 3.40, 3.26 [13:22:15] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.73, 3.53, 3.51 [13:22:19] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.98, 10.29, 10.25 [13:24:14] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.37, 4.51, 3.89 [13:24:17] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.24, 9.47, 9.95 [13:26:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 6.57, 4.88, 3.87 [13:26:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.10, 4.58, 3.61 [13:34:02] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.74, 10.15, 9.87 [13:35:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.29, 11.25, 10.92 [13:37:49] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.88, 1.78, 1.51 [13:37:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.95, 11.30, 10.97 [13:39:44] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 1.63, 1.69, 1.51 [13:39:55] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.26, 11.48, 10.58 [13:42:23] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.20, 10.49, 9.53 [13:42:23] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.53, 10.85, 9.90 [13:43:50] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.52, 11.43, 10.75 [13:44:08] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.77, 10.21, 9.70 [13:46:18] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.91, 9.73, 9.45 [13:46:22] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.83, 10.08, 9.81 [13:47:45] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.47, 11.51, 10.90 [13:48:04] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.09, 10.01, 9.77 [13:49:42] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.68, 11.01, 10.78 [13:51:10] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 4.22, 2.45, 1.81 [13:53:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.11, 11.53, 11.02 [13:54:59] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.42, 1.97, 1.75 [13:55:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.19, 11.00, 10.87 [13:56:53] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.31, 2.22, 1.88 [13:58:48] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 0.56, 1.59, 1.68 [14:03:34] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.19, 11.12, 10.71 [14:05:34] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.53, 10.74, 10.64 [14:13:34] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.41, 9.41, 10.08 [14:15:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.44, 11.24, 11.01 [14:17:34] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.87, 10.00, 10.13 [14:17:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.48, 11.52, 11.16 [14:21:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.02, 11.20, 11.08 [14:23:06] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.04, 10.40, 9.88 [14:25:34] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 9.83, 10.06, 10.14 [14:27:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.79, 11.73, 11.42 [14:28:13] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.35, 10.72, 9.85 [14:32:08] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.78, 11.53, 10.42 [14:32:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.95, 10.33, 10.14 [14:33:02] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.75, 9.94, 10.00 [14:33:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.33, 11.40, 11.35 [14:36:21] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.96, 9.54, 9.91 [14:37:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.77, 11.42, 11.42 [14:38:02] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk [14:39:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.28, 11.76, 11.55 [14:40:04] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.08, 9.79, 10.03 [14:41:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.91, 11.40, 11.47 [14:42:57] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.78, 10.69, 10.13 [14:44:49] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.23, 9.87, 9.34 [14:44:56] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 12.46, 11.15, 10.36 [14:45:06] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.17, 10.41, 10.12 [14:45:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.28, 11.89, 11.60 [14:46:04] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.19, 10.26, 10.12 [14:46:46] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.82, 9.56, 9.30 [14:46:55] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 8.77, 10.44, 10.22 [14:48:04] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.68, 10.16, 10.11 [14:49:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.93, 11.93, 11.72 [14:50:53] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 12.08, 10.42, 10.18 [14:51:05] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 10.47, 9.70, 9.02 [14:52:04] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.02, 10.54, 10.32 [14:52:53] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.78, 10.23, 10.15 [14:53:02] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.93, 9.19, 8.92 [14:53:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.99, 12.18, 11.82 [14:54:52] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.47, 9.82, 10.00 [14:58:04] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 10.14, 9.95, 10.14 [14:58:49] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.42, 9.80, 10.15 [15:02:04] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 13.04, 11.15, 10.56 [15:02:20] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.38, 9.57, 9.30 [15:02:42] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.02, 10.91, 10.58 [15:02:43] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.86, 1.72, 1.36 [15:04:04] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.40, 10.12, 10.24 [15:04:18] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 9.07, 9.59, 9.35 [15:04:42] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 1.01, 1.49, 1.32 [15:08:04] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.94, 9.80, 10.10 [15:11:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.02, 11.65, 11.97 [15:13:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.07, 11.66, 11.93 [15:17:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.00, 10.70, 11.52 [15:18:22] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.30, 11.18, 10.84 [15:20:19] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.90, 10.74, 10.72 [15:21:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.05, 11.36, 11.61 [15:23:05] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:25:05] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 28496 bytes in 2.195 second response time [15:26:03] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.74, 1.76, 1.29 [15:27:45] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 10.63, 9.38, 8.54 [15:28:03] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.52, 1.30, 1.18 [15:28:54] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.06, 10.52, 10.09 [15:29:45] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.41, 8.58, 8.35 [15:29:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.01, 11.93, 11.91 [15:30:52] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.26, 11.10, 10.35 [15:32:49] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 8.64, 10.25, 10.13 [15:34:47] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 6.36, 9.13, 9.75 [15:35:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.16, 11.51, 11.67 [15:37:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.24, 10.71, 11.36 [15:40:12] PROBLEM - db121 Current Load on db121 is CRITICAL: CRITICAL - load average: 9.65, 7.80, 4.76 [15:42:07] PROBLEM - db121 Current Load on db121 is WARNING: WARNING - load average: 7.95, 7.73, 5.09 [15:42:39] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:43:50] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.30, 10.41, 10.47 [15:44:39] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 28496 bytes in 0.738 second response time [15:45:48] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.58, 10.72, 10.57 [15:45:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.13, 10.84, 11.02 [15:45:54] RECOVERY - db121 Current Load on db121 is OK: OK - load average: 2.87, 5.47, 4.79 [15:46:30] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.62, 9.08, 9.03 [15:47:30] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.36, 9.77, 9.08 [15:47:45] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.52, 11.42, 10.85 [15:47:51] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.19, 2.58, 1.80 [15:47:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.54, 11.18, 11.14 [15:49:43] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.26, 10.75, 10.69 [15:49:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 14.40, 12.34, 11.57 [15:50:25] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.52, 10.86, 9.79 [15:51:29] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.21, 9.33, 9.11 [15:51:39] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 0.49, 1.49, 1.53 [15:51:50] [02mediawiki] 07Universal-Omega assigned pull request 03#8292: Bump extensions/UserFunctions from `b29316c` to `f10a2d8` - 13https://github.com/miraheze/mediawiki/pull/8292 [15:51:50] [url] Page not found · GitHub · GitHub | github.com [15:52:01] [02mediawiki] 07Universal-Omega assigned pull request 03#8293: Bump extensions/MobileDetect from `ee9a00f` to `7efd1f4` - 13https://github.com/miraheze/mediawiki/pull/8293 [15:52:01] [url] Page not found · GitHub · GitHub | github.com [15:52:22] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.09, 10.46, 9.77 [15:52:28] [02mediawiki] 07Universal-Omega assigned pull request 03#8253: Bump skins/Cosmos from `48e1b8a` to `36df7a3` - 13https://github.com/miraheze/mediawiki/pull/8253 [15:52:29] [url] Page not found · GitHub · GitHub | github.com [15:52:46] [02mediawiki] 07Universal-Omega assigned pull request 03#8215: Bump extensions/GroupsSidebar from `cbea2bd` to `22d3804` - 13https://github.com/miraheze/mediawiki/pull/8215 [15:52:46] ... [15:53:04] [02mediawiki] 07Universal-Omega labeled pull request 03#8215: Bump extensions/GroupsSidebar from `cbea2bd` to `22d3804` - 13https://github.com/miraheze/mediawiki/pull/8215 [15:53:04] ... [15:53:19] [02mediawiki] 07Universal-Omega labeled pull request 03#8293: Bump extensions/MobileDetect from `ee9a00f` to `7efd1f4` - 13https://github.com/miraheze/mediawiki/pull/8293 [15:53:20] ... [15:53:34] [02mediawiki] 07Universal-Omega labeled pull request 03#8292: Bump extensions/UserFunctions from `b29316c` to `f10a2d8` - 13https://github.com/miraheze/mediawiki/pull/8292 [15:53:35] [url] Page not found · GitHub · GitHub | github.com [15:54:20] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.46, 10.16, 9.75 [15:56:12] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.68, 3.31, 3.87 [15:56:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.20, 3.58, 3.96 [15:58:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.01, 3.73, 3.96 [15:59:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.28, 11.98, 11.92 [16:02:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.14, 4.42, 4.18 [16:05:53] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.03, 11.81, 11.78 [16:06:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.06, 3.65, 3.93 [16:07:48] PROBLEM - cp21 Stunnel HTTP for mw112 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:07:51] PROBLEM - cp30 Stunnel HTTP for mw111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:07:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.96, 11.10, 11.52 [16:08:02] PROBLEM - cp31 Stunnel HTTP for mw111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:08:03] PROBLEM - cp20 Stunnel HTTP for mw111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:08:05] PROBLEM - cp21 Stunnel HTTP for mw111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:08:08] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:08:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.31, 3.28, 3.81 [16:09:34] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.94, 8.67, 9.83 [16:09:43] RECOVERY - cp21 Stunnel HTTP for mw112 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 0.456 second response time [16:09:56] RECOVERY - cp30 Stunnel HTTP for mw111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 7.647 second response time [16:10:06] RECOVERY - cp20 Stunnel HTTP for mw111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 5.948 second response time [16:10:06] RECOVERY - cp31 Stunnel HTTP for mw111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 6.181 second response time [16:10:08] RECOVERY - cp21 Stunnel HTTP for mw111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 5.419 second response time [16:10:11] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 28496 bytes in 5.371 second response time [16:10:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.37, 3.58, 3.84 [16:10:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.27, 3.73, 3.87 [16:11:40] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.31, 3.65, 3.97 [16:11:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.20, 11.55, 11.56 [16:12:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.79, 3.74, 3.86 [16:13:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.47, 10.52, 11.19 [16:14:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.32, 3.35, 3.71 [16:15:40] PROBLEM - mw122 MediaWiki Rendering on mw122 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:15:40] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.74, 3.74, 3.90 [16:15:46] PROBLEM - cp21 Stunnel HTTP for mw122 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:16:43] PROBLEM - cp31 Stunnel HTTP for mw122 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:17:02] PROBLEM - cp30 Stunnel HTTP for mw122 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:17:09] PROBLEM - cp20 Stunnel HTTP for mw122 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:18:50] RECOVERY - cp31 Stunnel HTTP for mw122 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 9.829 second response time [16:19:24] PROBLEM - db121 Current Load on db121 is CRITICAL: CRITICAL - load average: 8.47, 5.99, 3.95 [16:19:40] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.00, 3.60, 3.82 [16:19:52] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.73, 8.52, 10.16 [16:20:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.02, 3.09, 3.43 [16:20:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.90, 3.50, 3.62 [16:22:04] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.81, 1.63, 1.41 [16:22:12] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.71, 2.72, 3.27 [16:22:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.34, 3.12, 3.47 [16:22:56] PROBLEM - cp31 Stunnel HTTP for mw102 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:22:57] RhinosF1: Do we have elastic search installed already? [16:23:01] PROBLEM - cp31 Stunnel HTTP for mw122 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:23:16] RECOVERY - cp30 Stunnel HTTP for mw122 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 9.630 second response time [16:23:20] RECOVERY - cp20 Stunnel HTTP for mw122 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 8.410 second response time [16:23:24] RECOVERY - db121 Current Load on db121 is OK: OK - load average: 5.80, 6.71, 4.77 [16:23:47] RECOVERY - cp21 Stunnel HTTP for mw122 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 0.059 second response time [16:23:47] RECOVERY - mw122 MediaWiki Rendering on mw122 is OK: HTTP OK: HTTP/1.1 200 OK - 28496 bytes in 0.477 second response time [16:24:03] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.67, 1.58, 1.41 [16:24:18] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.05, 3.00, 3.37 [16:24:53] RECOVERY - cp31 Stunnel HTTP for mw102 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 0.332 second response time [16:24:59] RECOVERY - cp31 Stunnel HTTP for mw122 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 0.314 second response time [16:25:41] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.38, 3.80, 3.72 [16:25:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.32, 9.41, 9.92 [16:27:41] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.43, 3.51, 3.63 [16:27:52] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.75, 9.41, 9.86 [16:28:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.98, 3.85, 3.50 [16:28:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.39, 3.60, 3.51 [16:30:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.75, 3.62, 3.54 [16:32:04] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.68, 9.76, 8.70 [16:32:13] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.66, 3.27, 3.33 [16:34:18] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.65, 3.22, 3.40 [16:34:19] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.06, 10.72, 9.81 [16:36:04] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.93, 9.29, 8.79 [16:36:16] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.39, 10.74, 9.91 [16:36:31] ugochimobi: not for mediawiki [16:36:41] Moan at the search team [16:37:05] eehn? [16:37:25] for graylog? [16:37:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.07, 10.94, 10.31 [16:37:54] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.08, 1.75, 1.48 [16:38:02] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk [16:38:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.86, 3.75, 3.52 [16:38:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.00, 3.37, 3.43 [16:39:40] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.38, 3.01, 3.31 [16:39:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.92, 10.95, 10.37 [16:39:53] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.85, 1.86, 1.56 [16:40:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.77, 3.86, 3.57 [16:40:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.01, 3.39, 3.42 [16:42:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.44, 4.00, 3.67 [16:43:08] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.57, 10.15, 9.20 [16:44:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.19, 3.78, 3.61 [16:45:07] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.69, 10.15, 9.33 [16:45:50] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.05, 1.60, 1.55 [16:46:36] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.05, 4.01, 3.59 [16:48:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.57, 4.25, 3.81 [16:48:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.35, 4.37, 3.85 [16:50:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.35, 3.98, 3.77 [16:50:19] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.53, 2.29, 1.48 [16:51:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.46, 11.23, 10.58 [16:52:17] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.30, 1.91, 1.43 [16:52:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.44, 4.38, 3.95 [16:52:30] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.22, 9.99, 9.33 [16:53:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.79, 11.33, 10.69 [16:54:28] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.78, 9.54, 9.25 [16:55:45] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.24, 10.19, 8.88 [16:55:52] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.59, 11.41, 10.74 [16:55:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.87, 11.90, 10.98 [16:56:02] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 12.18, 11.37, 10.21 [16:56:13] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.61, 1.45, 1.37 [16:56:54] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.23, 9.66, 9.14 [16:57:49] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.62, 11.29, 10.79 [16:57:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.97, 11.67, 11.00 [16:58:01] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.17, 11.18, 10.27 [16:58:52] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.40, 9.20, 9.03 [16:59:45] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 8.23, 9.42, 8.89 [17:02:44] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.72, 10.29, 9.54 [17:03:42] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.41, 11.76, 11.07 [17:03:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.05, 11.43, 11.06 [17:04:42] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 10.01, 9.93, 9.48 [17:05:39] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.28, 11.77, 11.17 [17:05:58] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.38, 9.96, 10.06 [17:07:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.51, 11.14, 11.08 [17:08:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.56, 3.32, 3.85 [17:09:54] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.48, 1.72, 1.43 [17:10:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.78, 4.44, 4.17 [17:11:52] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.10, 1.45, 1.36 [17:12:32] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk [17:15:34] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.51, 11.51, 11.23 [17:17:32] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk [17:19:21] PROBLEM - www.puzzles.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.puzzles.wiki' expires in 15 day(s) (Sun 07 Aug 2022 17:02:49 GMT +0000). [17:19:34] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.06, 11.33, 11.21 [17:19:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.90, 11.02, 10.80 [17:20:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.10, 2.97, 3.77 [17:21:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.77, 11.16, 10.88 [17:22:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.44, 3.52, 3.88 [17:22:31] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/9f62b492559a...94de04b99a67 [17:22:31] [url] Comparing 9f62b492559a...94de04b99a67 · miraheze/ssl · GitHub | github.com [17:22:32] [02miraheze/ssl] 07MirahezeSSLBot 0394de04b - Bot: Update SSL cert for www.puzzles.wiki [17:24:44] PROBLEM - puzzles.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.puzzles.wiki' expires in 15 day(s) (Sun 07 Aug 2022 17:02:49 GMT +0000). [17:25:34] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.01, 10.77, 10.88 [17:25:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.13, 11.89, 11.26 [17:26:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.82, 3.71, 3.87 [17:26:45] [02mw-config] 07ugochimobi opened pull request 03#4826: Configure Vector for gratispaideia wikis - 13https://github.com/miraheze/mw-config/pull/4826 [17:26:46] [url] Page not found · GitHub · GitHub | github.com [17:27:34] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.84, 10.37, 10.70 [17:27:37] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/94de04b99a67...97962aa82b5d [17:27:37] [url] Comparing 94de04b99a67...97962aa82b5d · miraheze/ssl · GitHub | github.com [17:27:38] [02miraheze/ssl] 07MirahezeSSLBot 0397962aa - Bot: Update SSL cert for puzzles.wiki [17:27:53] miraheze/mw-config - ugochimobi the build passed. [17:28:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 6.93, 4.87, 4.27 [17:29:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.64, 11.33, 11.23 [17:32:06] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.17, 10.93, 10.06 [17:33:34] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.05, 11.35, 10.95 [17:34:04] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.62, 10.13, 9.88 [17:35:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.88, 12.25, 11.61 [17:37:34] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.19, 11.43, 11.10 [17:37:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.00, 11.81, 11.53 [17:43:34] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.06, 11.06, 10.93 [17:45:34] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.83, 11.12, 10.99 [17:47:02] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk [17:48:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.53, 3.25, 3.88 [17:49:20] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.65, 2.99, 2.45 [17:49:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.02, 10.96, 10.95 [17:50:03] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 10.74, 9.71, 8.95 [17:50:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.33, 3.67, 3.94 [17:51:20] RECOVERY - matomo101 Current Load on matomo101 is OK: OK - load average: 2.89, 2.96, 2.51 [17:52:01] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 8.17, 9.21, 8.86 [17:52:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.62, 3.49, 3.83 [17:52:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.53, 3.68, 4.00 [17:53:09] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.49, 2.00, 1.66 [17:53:40] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.29, 1.63, 1.29 [17:53:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.09, 10.50, 10.81 [17:54:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.94, 4.44, 4.15 [17:54:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 7.19, 4.80, 4.36 [17:55:40] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.96, 1.32, 1.21 [17:57:07] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.55, 1.71, 1.62 [17:58:52] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.88, 10.54, 10.05 [17:59:05] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.68, 1.63, 1.59 [18:03:34] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.59, 11.22, 10.82 [18:04:44] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.77, 9.77, 9.91 [18:05:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.09, 11.26, 10.97 [18:07:34] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.52, 11.42, 11.01 [18:07:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.29, 11.18, 10.98 [18:11:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.56, 12.02, 11.35 [18:12:02] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk [18:13:41] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.25, 10.16, 9.45 [18:15:40] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.97, 10.13, 9.53 [18:18:03] RECOVERY - www.puzzles.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'www.puzzles.wiki' will expire on Thu 20 Oct 2022 16:22:25 GMT +0000. [18:19:38] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.60, 10.65, 9.89 [18:21:34] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.65, 9.40, 10.16 [18:22:45] RECOVERY - puzzles.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'www.puzzles.wiki' will expire on Thu 20 Oct 2022 16:22:25 GMT +0000. [18:22:50] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.97, 2.01, 1.67 [18:23:37] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.42, 9.89, 9.78 [18:23:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.05, 11.72, 11.66 [18:25:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.13, 12.43, 11.94 [18:26:48] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.80, 1.61, 1.61 [18:29:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.25, 11.58, 11.73 [18:31:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.40, 12.06, 11.90 [18:33:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.30, 11.37, 11.67 [18:35:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.71, 12.17, 11.93 [18:37:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.31, 11.77, 11.82 [18:39:32] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk [18:41:09] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.61, 10.60, 10.19 [18:43:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.23, 12.04, 11.85 [18:44:49] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.89, 9.71, 9.39 [18:45:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.85, 11.91, 11.82 [18:48:44] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.16, 9.60, 9.45 [18:49:42] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 1.62, 2.14, 1.70 [18:50:56] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 9.83, 10.18, 10.16 [18:51:40] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.58, 2.00, 1.71 [18:51:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.80, 11.73, 11.70 [18:53:40] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.08, 1.70, 1.63 [18:53:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.12, 11.58, 11.65 [19:02:25] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.91, 10.61, 9.80 [19:02:37] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.22, 11.17, 10.45 [19:03:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 14.06, 11.70, 11.42 [19:06:20] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.89, 10.00, 9.74 [19:06:32] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.65, 11.67, 10.84 [19:09:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.75, 11.93, 11.73 [19:10:16] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.12, 10.33, 9.62 [19:10:40] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.79, 10.12, 9.34 [19:14:14] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.98, 10.02, 9.71 [19:14:22] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.08, 11.57, 11.04 [19:15:57] PROBLEM - commonwealth.0um.io - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'commonwealth.0um.io' expires in 15 day(s) (Sun 07 Aug 2022 19:11:26 GMT +0000). [19:16:19] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.19, 11.13, 10.93 [19:16:32] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 9.19, 10.02, 9.62 [19:17:04] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.55, 10.31, 9.98 [19:19:01] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.40, 10.14, 9.97 [19:19:32] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk [19:19:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.47, 11.43, 11.45 [19:21:15] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/97962aa82b5d...41b07db27f53 [19:21:15] [url] Comparing 97962aa82b5d...41b07db27f53 · miraheze/ssl · GitHub | github.com [19:21:16] [02miraheze/ssl] 07MirahezeSSLBot 0341b07db - Bot: Update SSL cert for commonwealth.0um.io [19:21:40] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.23, 1.67, 1.24 [19:21:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.57, 11.57, 11.51 [19:23:40] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.18, 1.38, 1.18 [19:25:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.07, 11.45, 11.42 [19:26:07] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.46, 9.36, 10.12 [19:26:47] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.55, 10.57, 10.26 [19:28:45] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.62, 10.10, 10.12 [19:31:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.63, 11.47, 11.57 [19:32:37] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 14.79, 11.92, 10.79 [19:32:55] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.01, 10.45, 10.42 [19:34:35] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.65, 11.65, 10.83 [19:35:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.20, 11.28, 11.43 [19:37:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.02, 10.49, 11.12 [19:41:52] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.51, 8.62, 10.19 [19:44:40] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.64, 9.52, 10.04 [19:46:20] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.71, 9.67, 10.18 [19:57:42] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.98, 10.90, 10.46 [19:58:21] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.68, 10.37, 9.85 [20:00:00] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.64, 2.74, 1.90 [20:00:18] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 9.71, 10.05, 9.80 [20:02:04] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.99, 10.16, 9.84 [20:03:58] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.20, 1.91, 1.75 [20:04:04] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.79, 9.23, 9.54 [20:04:11] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.51, 10.50, 10.06 [20:05:34] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.13, 11.43, 10.76 [20:05:57] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.98, 1.65, 1.68 [20:07:32] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.94, 11.71, 10.95 [20:09:32] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk [20:13:57] RECOVERY - commonwealth.0um.io - LetsEncrypt on sslhost is OK: OK - Certificate 'commonwealth.0um.io' will expire on Thu 20 Oct 2022 18:21:09 GMT +0000. [20:16:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.29, 3.56, 3.99 [20:18:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.83, 3.86, 4.04 [20:21:49] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.06, 10.80, 10.45 [20:25:44] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.75, 11.47, 10.82 [20:26:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.90, 3.71, 3.99 [20:27:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.75, 11.75, 10.99 [20:28:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.83, 4.21, 4.14 [20:29:39] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.89, 11.02, 10.82 [20:29:40] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.61, 10.46, 9.83 [20:31:40] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.49, 10.08, 9.77 [20:32:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.37, 3.47, 3.88 [20:32:47] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.81, 10.26, 9.72 [20:34:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.30, 3.81, 3.95 [20:34:44] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.50, 9.70, 9.59 [20:36:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.09, 3.52, 3.83 [20:38:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.19, 3.72, 3.97 [20:40:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.43, 4.16, 4.11 [20:41:15] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.14, 10.08, 9.51 [20:42:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.67, 3.61, 3.91 [20:42:57] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.10, 11.26, 10.93 [20:43:34] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.16, 11.37, 11.12 [20:44:13] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.59, 2.90, 3.39 [20:44:27] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.21, 10.62, 9.99 [20:44:55] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.50, 11.40, 11.02 [20:45:10] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.55, 9.78, 9.57 [20:45:34] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.64, 10.77, 10.94 [20:46:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.89, 4.19, 4.02 [20:46:25] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.30, 10.10, 9.88 [20:48:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.22, 4.35, 3.88 [20:48:51] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.99, 12.00, 11.28 [20:49:34] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.00, 11.63, 11.22 [20:50:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.51, 3.77, 3.73 [20:50:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.74, 3.88, 3.93 [20:50:18] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.96, 10.77, 10.20 [20:51:34] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.15, 11.07, 11.06 [20:52:12] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 3.85, 4.03, 3.84 [20:53:25] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.42, 2.03, 1.68 [20:54:13] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.29, 9.89, 9.97 [20:54:45] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.91, 11.73, 11.45 [20:55:24] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.39, 1.72, 1.61 [20:56:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.77, 3.72, 3.79 [20:56:43] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.10, 11.61, 11.42 [20:56:45] PROBLEM - mwtask111 Puppet on mwtask111 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/etc/apt/trusted.gpg.d/gluster.gpg] [20:57:23] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.29, 1.64, 1.59 [20:58:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.41, 3.61, 3.75 [20:58:41] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.53, 10.51, 11.05 [20:59:32] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk [21:00:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.42, 3.69, 3.82 [21:02:04] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.63, 10.76, 10.14 [21:02:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.46, 4.21, 4.00 [21:04:04] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.52, 9.95, 9.90 [21:04:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.53, 3.97, 3.94 [21:04:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.56, 4.10, 3.92 [21:06:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 3.61, 4.09, 4.00 [21:06:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.20, 3.89, 3.88 [21:08:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.52, 3.87, 3.92 [21:12:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.20, 3.95, 3.88 [21:13:34] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 9.77, 9.77, 10.16 [21:18:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.33, 3.73, 3.86 [21:20:19] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.46, 9.29, 10.12 [21:24:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.82, 3.71, 3.63 [21:24:13] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.56, 10.15, 10.29 [21:24:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.41, 4.20, 3.92 [21:24:39] RECOVERY - mwtask111 Puppet on mwtask111 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [21:26:11] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.29, 9.69, 10.10 [21:26:12] PROBLEM - puppet111 APT on puppet111 is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [21:26:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.90, 3.79, 3.79 [21:28:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.07, 4.19, 3.94 [21:29:40] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.92, 3.65, 4.00 [21:32:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.44, 3.89, 3.81 [21:32:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.09, 3.83, 3.87 [21:33:41] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.56, 3.84, 3.94 [21:34:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.22, 3.88, 3.80 [21:34:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.30, 4.32, 4.04 [21:35:41] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.77, 3.80, 3.91 [21:36:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.91, 3.83, 3.78 [21:36:21] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.03, 10.26, 9.98 [21:36:57] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.11, 10.73, 10.35 [21:38:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.02, 3.73, 3.74 [21:40:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.54, 3.34, 3.60 [21:40:53] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.52, 9.77, 10.06 [21:41:41] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.03, 3.54, 3.73 [21:42:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 7.28, 4.51, 3.97 [21:42:14] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.48, 9.74, 9.92 [21:43:10] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.00, 9.97, 9.49 [21:43:41] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.19, 3.44, 3.68 [21:44:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.91, 3.97, 3.83 [21:45:09] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.30, 9.52, 9.39 [21:45:40] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.56, 4.14, 3.92 [21:46:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 3.99, 4.16, 3.93 [21:52:37] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.76, 11.36, 10.61 [21:53:49] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.07, 1.64, 1.38 [21:54:35] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.42, 10.69, 10.47 [21:55:48] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.10, 1.44, 1.34 [21:58:30] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.80, 1.68, 1.35 [22:00:28] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.33, 1.86, 1.45 [22:00:47] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.93, 10.77, 10.08 [22:02:26] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.22, 1.54, 1.38 [22:02:44] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.06, 10.78, 10.14 [22:04:25] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.47, 11.47, 10.94 [22:04:42] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.68, 11.01, 10.30 [22:06:39] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.88, 11.59, 10.59 [22:08:21] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.07, 10.77, 10.85 [22:08:37] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.39, 11.33, 10.64 [22:12:12] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.79, 1.51, 1.35 [22:14:10] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.40, 1.45, 1.35 [22:14:29] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.95, 9.23, 9.95 [22:19:40] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.25, 3.38, 3.96 [22:20:09] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.64, 8.76, 9.83 [22:21:40] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.27, 3.73, 4.01 [22:23:56] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.79, 1.96, 1.55 [22:25:40] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.65, 3.32, 3.82 [22:26:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.12, 3.49, 3.92 [22:27:52] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.95, 1.52, 1.46 [22:28:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.19, 3.67, 3.92 [22:30:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.81, 3.30, 3.75 [22:31:25] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.26, 9.31, 9.02 [22:33:22] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.10, 9.07, 8.96 [22:34:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.76, 3.37, 3.91 [22:34:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.23, 3.46, 3.69 [22:35:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.11, 11.34, 10.37 [22:36:00] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.40, 10.66, 9.83 [22:37:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.99, 10.79, 10.29 [22:38:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.19, 3.46, 3.65 [22:39:55] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.88, 9.73, 9.64 [22:41:40] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.61, 3.44, 3.57 [22:41:52] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.26, 9.72, 9.99 [22:43:40] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.54, 3.20, 3.47 [22:47:28] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 1595 bytes in 0.023 second response time [22:47:40] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.62, 2.91, 3.30 [22:47:47] PROBLEM - cp30 Stunnel HTTP for mw121 on cp30 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 328 bytes in 0.261 second response time [22:47:57] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.74, 11.13, 9.45 [22:48:18] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.00, 2.71, 3.27 [22:48:20] PROBLEM - cp31 Stunnel HTTP for mw121 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:49:28] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 28496 bytes in 0.303 second response time [22:49:43] RECOVERY - cp30 Stunnel HTTP for mw121 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.334 second response time [22:50:18] RECOVERY - cp31 Stunnel HTTP for mw121 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.331 second response time [22:50:42] PROBLEM - cp21 Varnish Backends on cp21 is CRITICAL: 1 backends are down. mw121 [22:51:57] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.91, 9.12, 9.07 [22:52:13] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.49, 2.83, 3.24 [22:52:42] RECOVERY - cp21 Varnish Backends on cp21 is OK: All 14 backends are healthy [22:53:15] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 3.03, 1.77, 1.36 [22:57:13] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.14, 1.57, 1.38 [23:01:40] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.83, 3.69, 3.29 [23:01:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.92, 10.89, 10.07 [23:02:56] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.09, 3.74, 3.43 [23:02:58] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.08, 3.52, 3.37 [23:06:45] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.12, 4.04, 3.63 [23:08:39] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.60, 4.00, 3.67 [23:09:52] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.98, 9.61, 9.92 [23:10:33] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.25, 4.15, 3.77 [23:10:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.51, 3.97, 3.72 [23:11:40] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.71, 3.75, 3.72 [23:12:26] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.31, 3.42, 3.54 [23:12:29] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.37, 4.06, 3.77 [23:13:40] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.58, 3.93, 3.78 [23:14:20] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.33, 3.80, 3.67 [23:16:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.41, 3.91, 3.74 [23:18:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.14, 3.93, 3.83 [23:21:41] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.08, 3.99, 3.94 [23:22:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.57, 4.39, 4.00 [23:23:40] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.71, 2.00, 1.41 [23:25:40] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.74, 1.57, 1.32 [23:25:41] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.15, 4.11, 3.99 [23:26:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.51, 4.12, 3.82 [23:27:41] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.90, 3.62, 3.83 [23:28:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.41, 3.90, 3.78 [23:29:41] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.54, 3.93, 3.91 [23:30:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 3.82, 4.02, 3.84 [23:31:41] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.36, 3.53, 3.76 [23:32:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.48, 3.52, 3.68 [23:34:12] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.08, 3.55, 3.91 [23:35:41] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.03, 3.38, 3.60 [23:36:12] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 3.65, 3.96, 4.04 [23:37:40] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.82, 3.80, 3.74 [23:38:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.05, 3.84, 3.74 [23:40:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.47, 3.75, 3.99 [23:40:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.64, 3.43, 3.60 [23:42:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.37, 3.60, 3.63 [23:44:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.31, 3.43, 3.56 [23:45:41] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.86, 2.74, 3.26 [23:46:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.43, 3.58, 3.59 [23:47:40] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.20, 2.12, 1.66 [23:48:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.70, 3.84, 3.83 [23:49:40] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.63, 1.97, 1.66 [23:49:41] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.58, 3.72, 3.49 [23:52:47] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 4.32, 3.30, 2.12 [23:53:40] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.51, 1.42, 1.53