[00:01:00] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.74, 4.05, 3.81 [00:01:46] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.66, 4.18, 3.44 [00:03:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [00:03:31] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.63, 1.73, 1.66 [00:03:40] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.49, 4.65, 3.75 [00:04:05] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.40, 1.82, 1.97 [00:05:26] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.05, 1.52, 1.60 [00:06:26] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.92, 11.55, 11.41 [00:07:36] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.57, 3.57, 3.52 [00:08:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [00:08:26] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.24, 11.12, 11.26 [00:12:05] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.15, 1.32, 1.65 [00:12:42] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.21, 10.33, 9.69 [00:13:46] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.03, 3.90, 3.84 [00:16:33] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.31, 10.25, 9.80 [00:16:42] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 6.73, 9.12, 9.41 [00:17:27] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.25, 2.92, 3.33 [00:18:33] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.00, 9.32, 9.51 [00:20:05] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.98, 1.72, 1.67 [00:21:46] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.91, 2.95, 3.34 [00:23:56] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.01, 2.27, 1.75 [00:24:06] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.12, 1.52, 1.61 [00:25:00] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.11, 3.52, 3.95 [00:25:37] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.47, 10.33, 9.69 [00:25:56] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.15, 1.81, 1.64 [00:27:00] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.48, 3.83, 4.01 [00:27:56] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.09, 1.50, 1.54 [00:29:00] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 1.98, 3.15, 3.74 [00:29:37] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.91, 9.26, 9.42 [00:35:00] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.91, 2.81, 3.35 [00:36:26] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.25, 9.72, 10.18 [00:46:00] [02ManageWiki] 07Universal-Omega closed pull request 03#359: Fix retrieving default for autopromote groups multiselect field - 13https://github.com/miraheze/ManageWiki/pull/359 [00:46:00] [url] Page not found · GitHub · GitHub | github.com [00:46:01] [02miraheze/ManageWiki] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ManageWiki/compare/a4c71658043d...89e465430fd7 [00:46:02] [url] Comparing a4c71658043d...89e465430fd7 · miraheze/ManageWiki · GitHub | github.com [00:46:03] [02miraheze/ManageWiki] 07Universal-Omega 0389e4654 - Fix retrieving default for autopromote groups multiselect field (#359) [00:46:04] [02ManageWiki] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/ManageWiki [00:46:05] [url] Page not found · GitHub · GitHub | github.com [00:46:06] [02miraheze/ManageWiki] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 [00:48:26] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.28, 10.67, 10.25 [00:48:57] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 3.19, 2.10, 1.73 [00:49:37] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.94, 10.28, 9.50 [00:49:44] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/e28e79fd9621...e30f886ad930 [00:49:45] [url] Comparing e28e79fd9621...e30f886ad930 · miraheze/mw-config · GitHub | github.com [00:49:46] [02miraheze/mw-config] 07Universal-Omega 03e30f886 - Lower `innodb_lock_wait_timeout` to 15 [00:50:17] !log [@test101] starting deploy of {'config': True} to all [00:50:19] !log [@test101] finished deploy of {'config': True} to all - SUCCESS in 1s [00:50:26] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.43, 10.97, 10.42 [00:50:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:50:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:50:49] miraheze/mw-config - Universal-Omega the build passed. [00:50:53] miraheze/ManageWiki - Universal-Omega the build passed. [00:51:37] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 9.42, 9.98, 9.50 [00:53:10] !log [@mwtask111] starting deploy of {'config': True} to all [00:53:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:53:19] !log [@mwtask111] finished deploy of {'config': True} to all - SUCCESS in 9s [00:53:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:54:05] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.82, 3.41, 3.13 [00:54:53] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.42, 1.80, 1.74 [00:55:22] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.59, 3.49, 3.33 [00:55:37] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.52, 10.91, 10.01 [00:56:52] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.39, 1.66, 1.70 [00:57:16] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.92, 3.35, 3.30 [00:58:26] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.78, 11.41, 10.79 [00:58:31] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.21, 3.47, 3.05 [00:59:37] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 8.82, 10.03, 9.89 [01:00:26] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.53, 10.61, 10.57 [01:02:28] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.94, 3.42, 3.14 [01:02:59] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/240e49b4e271 [01:03:00] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.90, 3.74, 3.48 [01:03:00] [02miraheze/mw-config] 07Universal-Omega 03240e49b - Fix $wgLBFactoryConf class [01:03:02] [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/mw-config [01:03:02] [url] Page not found · GitHub · GitHub | github.com [01:03:03] [02mw-config] 07Universal-Omega opened pull request 03#4686: Fix $wgLBFactoryConf class - 13https://github.com/miraheze/mw-config/pull/4686 [01:03:04] [url] Page not found · GitHub · GitHub | github.com [01:03:19] [02mw-config] 07Universal-Omega closed pull request 03#4686: Fix $wgLBFactoryConf class - 13https://github.com/miraheze/mw-config/pull/4686 [01:03:19] [url] Page not found · GitHub · GitHub | github.com [01:03:20] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/e30f886ad930...f3743947fdb4 [01:03:21] [url] Comparing e30f886ad930...f3743947fdb4 · miraheze/mw-config · GitHub | github.com [01:03:22] [02miraheze/mw-config] 07Universal-Omega 03f374394 - Fix $wgLBFactoryConf class (#4686) [01:03:23] [02mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/mw-config [01:03:24] ... [01:03:25] [02miraheze/mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 [01:03:55] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.39, 3.24, 3.23 [01:04:13] miraheze/mw-config - Universal-Omega the build passed. [01:04:26] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.62, 3.17, 3.09 [01:04:26] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.53, 9.57, 10.15 [01:04:27] miraheze/mw-config - Universal-Omega the build passed. [01:07:00] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.07, 3.22, 3.32 [01:07:35] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/8d9eb4ec2063 [01:07:37] [02miraheze/mw-config] 07Universal-Omega 038d9eb4e - Improve performance of getDatabaseClusters() [01:07:38] [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/mw-config [01:07:39] [url] Page not found · GitHub · GitHub | github.com [01:07:40] [02mw-config] 07Universal-Omega opened pull request 03#4687: Improve performance of getDatabaseClusters() - 13https://github.com/miraheze/mw-config/pull/4687 [01:07:40] ... [01:08:40] [02mw-config] 07Universal-Omega closed pull request 03#4687: Improve performance of getDatabaseClusters() - 13https://github.com/miraheze/mw-config/pull/4687 [01:08:41] [url] Page not found · GitHub · GitHub | github.com [01:08:42] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/f3743947fdb4...788110989f62 [01:08:42] [url] Comparing f3743947fdb4...788110989f62 · miraheze/mw-config · GitHub | github.com [01:08:43] [02miraheze/mw-config] 07Universal-Omega 037881109 - Improve performance of getDatabaseClusters() (#4687) [01:08:45] [02mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/mw-config [01:08:45] [url] Page not found · GitHub · GitHub | github.com [01:08:46] [02miraheze/mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 [01:08:48] miraheze/mw-config - Universal-Omega the build passed. [01:09:49] miraheze/mw-config - Universal-Omega the build passed. [01:10:44] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.86, 1.74, 1.67 [01:14:26] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.56, 10.05, 10.02 [01:14:41] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.66, 1.70, 1.67 [01:16:26] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 10.05, 10.01, 10.00 [01:18:39] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.66, 2.05, 1.81 [01:20:34] !log [@test101] starting deploy of {'config': True} to all [01:20:35] !log [@test101] finished deploy of {'config': True} to all - SUCCESS in 0s [01:20:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:21:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:21:37] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 10.32, 9.93, 9.35 [01:22:26] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.46, 11.33, 10.51 [01:23:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:23:20] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp20.miraheze.org - CNAME OK [01:23:32] !log [@mwtask111] starting deploy of {'config': True} to all [01:23:37] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.64, 9.15, 9.14 [01:23:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:23:42] !log [@mwtask111] finished deploy of {'config': True} to all - SUCCESS in 10s [01:23:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:24:26] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.93, 11.00, 10.51 [01:26:34] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.77, 1.97, 1.90 [01:28:26] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.42, 11.62, 10.87 [01:30:26] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.69, 11.09, 10.75 [01:35:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:40:27] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.40, 1.51, 1.69 [01:40:48] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.82, 2.36, 1.49 [01:42:43] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.24, 1.89, 1.42 [01:44:38] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.03, 1.56, 1.35 [01:45:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:46:26] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.27, 9.57, 10.18 [01:50:21] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.38, 1.90, 1.73 [01:50:26] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.69, 10.97, 10.57 [01:51:51] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.49, 10.87, 9.95 [01:52:20] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.57, 1.84, 1.74 [01:53:46] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.38, 11.33, 10.22 [01:54:18] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.14, 1.61, 1.66 [01:54:26] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.30, 11.15, 10.78 [01:54:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:56:26] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.19, 12.02, 11.15 [01:57:03] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.47, 10.60, 9.56 [01:57:37] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.63, 11.39, 10.47 [01:58:16] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.83, 3.18, 2.82 [01:58:26] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.53, 11.84, 11.20 [01:58:28] PROBLEM - cp31 Stunnel HTTP for mw111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:58:38] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:59:02] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.22, 10.00, 9.47 [02:00:26] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.93, 12.27, 11.43 [02:00:28] RECOVERY - cp31 Stunnel HTTP for mw111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 14626 bytes in 4.935 second response time [02:00:35] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 22334 bytes in 3.029 second response time [02:01:37] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 9.78, 9.92, 10.05 [02:02:26] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.44, 11.92, 11.41 [02:03:00] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.91, 3.92, 3.31 [02:03:01] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.56, 10.48, 9.84 [02:04:26] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.46, 12.27, 11.58 [02:05:00] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.28, 3.30, 3.15 [02:05:00] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.68, 10.04, 9.74 [02:06:16] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.43, 3.08, 2.97 [02:08:26] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.84, 11.91, 11.60 [02:09:00] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.16, 4.86, 3.82 [02:09:37] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 10.06, 10.23, 10.20 [02:10:16] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.73, 3.54, 3.19 [02:10:26] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.94, 12.22, 11.74 [02:11:37] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 8.38, 9.56, 9.96 [02:12:16] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.24, 3.03, 3.04 [02:13:00] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.37, 3.70, 3.61 [02:13:23] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.12, 10.61, 9.90 [02:17:10] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 13.02, 11.67, 10.46 [02:19:00] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.46, 2.80, 3.24 [02:19:04] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.64, 11.40, 10.52 [02:19:37] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.13, 11.25, 10.55 [02:20:05] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.59, 2.09, 1.68 [02:21:59] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.81, 1.60, 1.36 [02:23:37] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 10.23, 11.55, 10.90 [02:24:05] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.52, 1.81, 1.67 [02:25:48] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:25:49] PROBLEM - cp31 Stunnel HTTP for mw111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:26:03] PROBLEM - cp21 Stunnel HTTP for mw111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:26:05] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.40, 1.69, 1.64 [02:26:12] PROBLEM - cp30 Stunnel HTTP for mw111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:26:42] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.34, 9.96, 10.17 [02:27:49] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 22334 bytes in 6.595 second response time [02:27:50] RECOVERY - cp31 Stunnel HTTP for mw111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 14634 bytes in 5.270 second response time [02:27:56] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.80, 1.44, 1.41 [02:28:02] RECOVERY - cp21 Stunnel HTTP for mw111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 14640 bytes in 2.489 second response time [02:28:14] RECOVERY - cp30 Stunnel HTTP for mw111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 14626 bytes in 2.707 second response time [02:33:51] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.93, 11.06, 10.35 [02:34:55] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.86, 10.69, 9.67 [02:38:26] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.72, 11.51, 11.87 [02:38:53] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.92, 9.68, 9.52 [02:39:50] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.64, 9.88, 10.06 [02:40:26] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.20, 11.44, 11.78 [02:41:29] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 12.55, 11.23, 10.62 [02:41:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.36, 9.93, 9.22 [02:43:23] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.44, 10.75, 10.53 [02:43:40] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.98, 9.59, 9.21 [02:43:48] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.86, 11.34, 10.62 [02:45:09] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.81, 3.40, 3.20 [02:45:16] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 12.39, 11.28, 10.74 [02:47:03] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.52, 3.71, 3.33 [02:47:10] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.12, 11.05, 10.74 [02:47:37] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 8.79, 9.25, 9.99 [02:49:00] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.60, 3.31, 3.23 [02:49:55] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.63, 1.96, 1.72 [02:51:43] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - dnd.bellinrattin.it All nameservers failed to answer the query. [02:51:46] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.17, 9.28, 10.05 [02:52:51] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.51, 9.50, 10.14 [02:53:27] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:53:33] PROBLEM - cp31 Stunnel HTTP for mw111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:54:42] PROBLEM - cp20 Stunnel HTTP for mw111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:54:50] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.02, 3.80, 3.07 [02:54:59] PROBLEM - cp30 Stunnel HTTP for mw111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:55:37] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 10.85, 9.88, 9.92 [02:55:47] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.90, 4.64, 3.76 [02:55:52] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.74, 1.96, 1.83 [02:55:54] PROBLEM - cp21 Stunnel HTTP for mw111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:55:56] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 1.62, 2.25, 1.82 [02:56:06] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 7.59, 4.68, 3.40 [02:57:56] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.02, 1.85, 1.73 [02:59:56] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.62, 1.44, 1.59 [03:00:06] RECOVERY - cp21 Stunnel HTTP for mw111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 14626 bytes in 9.903 second response time [03:01:43] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.59, 10.87, 10.33 [03:03:37] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 8.83, 10.19, 10.17 [03:03:42] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.49, 10.27, 10.18 [03:04:30] PROBLEM - cp21 Stunnel HTTP for mw111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:07:37] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.86, 10.48, 10.32 [03:09:41] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.04, 9.97, 10.14 [03:13:37] RECOVERY - cp20 Stunnel HTTP for mw111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 14626 bytes in 5.463 second response time [03:13:39] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 13.70, 11.42, 10.64 [03:14:14] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 22334 bytes in 9.160 second response time [03:14:17] RECOVERY - cp30 Stunnel HTTP for mw111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 14634 bytes in 8.770 second response time [03:14:27] RECOVERY - cp31 Stunnel HTTP for mw111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 14634 bytes in 9.398 second response time [03:15:00] RECOVERY - cp21 Stunnel HTTP for mw111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 14640 bytes in 9.449 second response time [03:15:37] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 8.31, 9.59, 10.07 [03:17:38] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 7.06, 9.94, 10.29 [03:17:43] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.13, 1.90, 1.86 [03:18:26] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.90, 11.05, 11.92 [03:19:38] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.42, 8.91, 9.85 [03:20:25] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.01, 3.39, 3.94 [03:22:23] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.98, 4.11, 4.15 [03:23:40] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.50, 1.95, 1.94 [03:29:37] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.11, 1.42, 1.70 [03:38:26] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.17, 9.24, 10.20 [03:49:18] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/l [03:49:18] n3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [03:50:26] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.71, 10.69, 10.30 [03:51:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.79, 1.63, 1.50 [03:51:46] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.93, 3.62, 3.93 [03:53:23] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.32, 1.49, 1.46 [03:54:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:56:26] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.53, 10.08, 10.17 [04:00:26] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.68, 10.08, 10.21 [04:01:29] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 14.05, 10.87, 9.46 [04:02:16] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.19, 3.58, 3.97 [04:03:28] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.91, 10.45, 9.49 [04:03:46] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.59, 2.92, 3.39 [04:05:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:07:27] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.63, 9.64, 9.39 [04:08:16] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.79, 3.72, 3.83 [04:08:26] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.23, 8.89, 9.75 [04:10:16] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.22, 3.10, 3.59 [04:15:00] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.13, 2.86, 3.81 [04:18:06] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp21.miraheze.org - CNAME OK [04:18:08] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.19, 1.76, 1.57 [04:18:16] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.43, 2.82, 3.27 [04:20:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:20:08] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.60, 10.09, 9.21 [04:21:00] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.87, 3.27, 3.65 [04:22:03] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 9.82, 10.02, 9.29 [04:24:05] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.61, 1.94, 1.76 [04:24:26] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.20, 10.57, 9.80 [04:25:00] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.12, 3.27, 3.58 [04:26:26] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.89, 10.67, 9.94 [04:27:00] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.65, 3.83, 3.75 [04:27:11] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.99, 3.74, 3.44 [04:27:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:27:50] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.39, 10.29, 9.54 [04:28:05] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.46, 1.68, 1.69 [04:28:33] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.11, 3.57, 3.18 [04:29:00] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.88, 3.48, 3.63 [04:29:09] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.78, 3.75, 3.48 [04:29:45] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 6.79, 8.97, 9.15 [04:30:31] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.87, 3.48, 3.18 [04:32:26] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.24, 9.60, 9.75 [04:32:30] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.46, 3.22, 3.14 [04:36:05] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.91, 1.78, 1.71 [04:37:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:39:00] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.69, 3.11, 3.40 [04:40:57] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.96, 2.91, 3.25 [04:43:00] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.30, 3.46, 3.48 [04:44:05] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.07, 1.89, 1.79 [04:45:00] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.04, 3.68, 3.59 [04:46:05] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.79, 1.83, 1.78 [04:47:00] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.71, 3.11, 3.40 [04:48:05] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.41, 2.01, 1.85 [04:51:00] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.70, 3.38, 3.48 [04:53:00] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.74, 2.94, 3.31 [04:54:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:56:05] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.71, 1.97, 1.95 [04:58:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.46, 3.18, 3.12 [04:59:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:04:15] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.15, 9.81, 8.68 [05:06:15] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.81, 9.81, 8.82 [05:06:27] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.09, 2.95, 3.09 [05:08:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:13:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:14:05] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.34, 1.50, 1.69 [05:16:16] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.45, 3.40, 3.13 [05:17:00] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.83, 4.60, 3.59 [05:17:28] [02ssl] 07ajhalili2006 opened pull request 03#514: Add redirects for wiki.rtapp.tk and wiki.recaptime.tk as per T9210 - 13https://github.com/miraheze/ssl/pull/514 [05:17:28] [url] Page not found · GitHub · GitHub | github.com [05:18:47] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 10.43, 5.45, 3.66 [05:19:16] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.60, 10.85, 10.32 [05:20:00] [02ssl] 07Universal-Omega reviewed pull request 03#514 commit - 13https://github.com/miraheze/ssl/pull/514#discussion_r869879623 [05:20:01] [url] Add redirects for wiki.rtapp.tk and wiki.recaptime.tk as per T9210 by ajhalili2006 · Pull Request #514 · miraheze/ssl · GitHub | github.com [05:20:05] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.62, 2.06, 1.84 [05:24:05] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.09, 1.68, 1.75 [05:24:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:25:05] [02miraheze/mediawiki] 07Reception123 pushed 031 commit to 03REL1_37 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/79097b1bbfaa...75e0d3b93289 [05:25:06] [url] Comparing 79097b1bbfaa...75e0d3b93289 · miraheze/mediawiki · GitHub | github.com [05:25:07] [02miraheze/mediawiki] 07Reception123 0375e0d3b - Update ManageWiki [05:25:35] !log [reception@mwtask111] starting deploy of {'world': True, 'l10n': True} to all [05:25:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:26:05] [02miraheze/mediawiki] 07Reception123 pushed 031 commit to 03REL1_38 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/e6f5c96f69a1...3f16766ba4a8 [05:26:05] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.25, 1.50, 1.67 [05:26:05] [url] Comparing e6f5c96f69a1...3f16766ba4a8 · miraheze/mediawiki · GitHub | github.com [05:26:06] [02miraheze/mediawiki] 07Reception123 033f16766 - Update ManageWiki [05:28:47] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:28:57] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.53, 9.69, 10.14 [05:29:16] PROBLEM - cp20 Stunnel HTTP for mw111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:29:36] PROBLEM - cp31 Stunnel HTTP for mw111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:30:45] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 22334 bytes in 4.004 second response time [05:31:14] RECOVERY - cp20 Stunnel HTTP for mw111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 14634 bytes in 1.001 second response time [05:31:33] RECOVERY - cp31 Stunnel HTTP for mw111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 14634 bytes in 1.404 second response time [05:32:54] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.83, 10.48, 10.39 [05:39:28] !log [reception@mwtask111] finished deploy of {'world': True, 'l10n': True} to all - SUCCESS in 832s [05:39:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:42:34] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.70, 9.68, 10.12 [05:50:05] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.10, 1.91, 1.64 [05:52:05] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.68, 1.83, 1.64 [05:53:56] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.62, 1.74, 1.25 [05:55:56] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.90, 1.35, 1.16 [05:56:05] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.41, 1.62, 1.60 [05:59:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [06:04:08] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.54, 3.63, 4.00 [06:06:05] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.44, 1.90, 1.71 [06:08:05] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.62, 3.83, 3.97 [06:09:00] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.59, 3.32, 3.98 [06:10:03] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.12, 3.49, 3.83 [06:11:46] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.50, 9.68, 8.82 [06:12:02] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.71, 3.70, 3.85 [06:13:00] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.85, 3.84, 4.01 [06:13:41] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.12, 8.77, 8.59 [06:15:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [06:15:58] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.72, 3.88, 3.95 [06:17:00] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.26, 3.63, 3.95 [06:20:16] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.15, 3.35, 3.84 [06:20:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [06:21:53] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.06, 2.57, 3.33 [06:25:00] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.69, 2.49, 3.29 [06:28:16] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.13, 2.77, 3.36 [06:31:37] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.19, 10.04, 9.10 [06:32:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [06:33:37] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 9.38, 9.84, 9.15 [06:37:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [06:38:02] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.80, 10.30, 9.88 [06:39:59] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 10.09, 10.06, 9.83 [06:44:25] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.53, 3.89, 3.39 [06:46:19] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.75, 3.49, 3.30 [06:46:23] PROBLEM - thenintendowiki.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'thenintendowiki.org' expires in 15 day(s) (Fri 27 May 2022 06:23:28 GMT +0000). [06:48:13] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.24, 3.06, 3.16 [06:48:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [06:50:26] PROBLEM - sunrin.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'sunrin.net' expires in 15 day(s) (Fri 27 May 2022 06:45:30 GMT +0000). [06:51:04] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/fdde20740e48...e972f565ceee [06:51:04] [url] Comparing fdde20740e48...e972f565ceee · miraheze/ssl · GitHub | github.com [06:51:05] [02miraheze/ssl] 07MirahezeSSLBot 03e972f56 - Bot: Update SSL cert for thenintendowiki.org [06:52:00] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.15, 3.38, 3.25 [06:53:10] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/e972f565ceee...c33bd15b2367 [06:53:11] [url] Comparing e972f565ceee...c33bd15b2367 · miraheze/ssl · GitHub | github.com [06:53:12] [02miraheze/ssl] 07MirahezeSSLBot 03c33bd15 - Bot: Update SSL cert for sunrin.net [06:53:36] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.32, 10.36, 9.92 [06:57:42] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.36, 3.53, 3.47 [06:59:25] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.84, 10.06, 10.00 [07:01:29] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.16, 3.59, 3.50 [07:03:19] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.31, 10.58, 10.22 [07:03:23] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.90, 2.87, 3.25 [07:03:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [07:08:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [07:09:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.71, 10.08, 10.16 [07:13:38] PROBLEM - www.scum-wiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'scum-wiki.com' expires in 15 day(s) (Fri 27 May 2022 07:05:51 GMT +0000). [07:15:53] PROBLEM - scum-wiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'scum-wiki.com' expires in 15 day(s) (Fri 27 May 2022 07:05:51 GMT +0000). [07:16:23] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/c33bd15b2367...a1a57929d419 [07:16:24] [url] Comparing c33bd15b2367...a1a57929d419 · miraheze/ssl · GitHub | github.com [07:16:25] [02miraheze/ssl] 07MirahezeSSLBot 03a1a5792 - Bot: Update SSL cert for scum-wiki.com [07:17:44] PROBLEM - smashbroswiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'smashbroswiki.com' expires in 15 day(s) (Fri 27 May 2022 06:54:46 GMT +0000). [07:18:00] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/a1a57929d419...2497cecaa5c1 [07:18:00]