[00:00:08] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.91, 6.62, 6.15 [00:02:05] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 6.11, 6.45, 6.15 [00:02:19] PROBLEM - graylog2 Puppet on graylog2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [00:03:06] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 25.49, 21.23, 18.37 [00:05:06] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 17.64, 19.69, 18.15 [00:05:44] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 1 datacenter is down: 198.244.148.90/cpweb [00:05:49] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 2607:5300:201:3100::5ebc/cpweb [00:05:56] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.10, 6.92, 6.42 [00:07:44] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [00:07:45] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [00:07:53] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.56, 6.56, 6.36 [00:11:16] PROBLEM - mw12 MediaWiki Rendering on mw12 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:11:34] PROBLEM - cp21 Stunnel Http for mw12 on cp21 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [00:12:14] PROBLEM - cp31 Stunnel Http for mw12 on cp31 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [00:12:20] PROBLEM - cp30 Stunnel Http for mw12 on cp30 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [00:13:36] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 8 datacenters are down: 51.195.220.68/cpweb, 198.244.148.90/cpweb, 2001:41d0:801:2000::4c25/cpweb, 2001:41d0:801:2000::1b80/cpweb, 149.56.140.43/cpweb, 149.56.141.75/cpweb, 2607:5300:201:3100::929a/cpweb, 2607:5300:201:3100::5ebc/cpweb [00:13:42] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 5 datacenters are down: 51.195.220.68/cpweb, 2001:41d0:801:2000::4c25/cpweb, 2001:41d0:801:2000::1b80/cpweb, 149.56.140.43/cpweb, 149.56.141.75/cpweb [00:15:32] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [00:15:47] RECOVERY - cp21 Stunnel Http for mw12 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 14565 bytes in 5.317 second response time [00:16:15] RECOVERY - cp31 Stunnel Http for mw12 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 14565 bytes in 1.704 second response time [00:16:33] RECOVERY - cp30 Stunnel Http for mw12 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 14557 bytes in 2.160 second response time [00:17:26] RECOVERY - mw12 MediaWiki Rendering on mw12 is OK: HTTP OK: HTTP/1.1 200 OK - 20526 bytes in 4.439 second response time [00:17:41] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [00:19:45] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.51, 3.68, 3.32 [00:20:28] PROBLEM - cp30 Stunnel Http for mw9 on cp30 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [00:21:03] PROBLEM - cp30 Stunnel Http for mw12 on cp30 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [00:21:06] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 23.91, 21.19, 19.40 [00:21:19] PROBLEM - cp20 Stunnel Http for mw9 on cp20 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [00:21:24] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 7 datacenters are down: 51.195.220.68/cpweb, 198.244.148.90/cpweb, 2001:41d0:801:2000::4c25/cpweb, 149.56.140.43/cpweb, 149.56.141.75/cpweb, 2607:5300:201:3100::929a/cpweb, 2607:5300:201:3100::5ebc/cpweb [00:21:38] PROBLEM - mw9 MediaWiki Rendering on mw9 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:21:39] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 7 datacenters are down: 51.195.220.68/cpweb, 2001:41d0:801:2000::4c25/cpweb, 2001:41d0:801:2000::1b80/cpweb, 149.56.140.43/cpweb, 149.56.141.75/cpweb, 2607:5300:201:3100::929a/cpweb, 2607:5300:201:3100::5ebc/cpweb [00:21:42] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.20, 3.42, 3.26 [00:21:47] PROBLEM - mw12 MediaWiki Rendering on mw12 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:22:11] PROBLEM - cp20 Stunnel Http for mw12 on cp20 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [00:22:23] PROBLEM - cp21 Stunnel Http for mw12 on cp21 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [00:22:34] PROBLEM - cp31 Stunnel Http for mw12 on cp31 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [00:23:05] PROBLEM - cp21 Stunnel Http for mw9 on cp21 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [00:23:05] PROBLEM - cp31 Stunnel Http for mw9 on cp31 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [00:23:19] RECOVERY - cp20 Stunnel Http for mw9 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 14564 bytes in 6.200 second response time [00:23:47] RECOVERY - mw9 MediaWiki Rendering on mw9 is OK: HTTP OK: HTTP/1.1 200 OK - 20524 bytes in 9.979 second response time [00:25:36] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.82, 4.07, 3.55 [00:27:06] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 27.99, 23.40, 20.92 [00:27:37] PROBLEM - cp20 Stunnel Http for mw9 on cp20 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [00:28:15] PROBLEM - mw9 MediaWiki Rendering on mw9 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:29:06] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 23.45, 23.08, 21.09 [00:29:29] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.81, 3.93, 3.61 [00:30:38] RECOVERY - cp20 Stunnel Http for mw12 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 14557 bytes in 6.967 second response time [00:30:44] RECOVERY - cp31 Stunnel Http for mw12 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 14557 bytes in 1.609 second response time [00:30:49] RECOVERY - cp21 Stunnel Http for mw12 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 14565 bytes in 0.013 second response time [00:31:06] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 26.04, 24.37, 21.83 [00:31:21] RECOVERY - cp21 Stunnel Http for mw9 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 14556 bytes in 9.302 second response time [00:31:25] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.22, 4.36, 3.71 [00:31:37] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [00:31:43] RECOVERY - cp30 Stunnel Http for mw12 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 14557 bytes in 0.323 second response time [00:31:45] RECOVERY - cp20 Stunnel Http for mw9 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 14556 bytes in 9.848 second response time [00:32:07] RECOVERY - mw12 MediaWiki Rendering on mw12 is OK: HTTP OK: HTTP/1.1 200 OK - 20526 bytes in 3.587 second response time [00:32:14] RECOVERY - graylog2 Puppet on graylog2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:32:27] RECOVERY - mw9 MediaWiki Rendering on mw9 is OK: HTTP OK: HTTP/1.1 200 OK - 20524 bytes in 3.254 second response time [00:32:53] RECOVERY - cp30 Stunnel Http for mw9 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 14564 bytes in 0.852 second response time [00:33:03] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [00:33:23] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.85, 3.96, 3.66 [00:33:25] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 3.97, 4.55, 3.89 [00:33:35] RECOVERY - cp31 Stunnel Http for mw9 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 14556 bytes in 0.355 second response time [00:33:39] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [00:35:20] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.04, 3.53, 3.54 [00:37:06] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 15.93, 21.90, 21.95 [00:39:15] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.34, 3.91, 3.67 [00:40:39] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [00:41:11] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.84, 3.68, 3.63 [00:45:06] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.48, 4.14, 3.82 [00:45:06] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 16.35, 18.69, 20.31 [00:51:32] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 3 datacenters are down: 2001:41d0:801:2000::4c25/cpweb, 2001:41d0:801:2000::1b80/cpweb, 149.56.140.43/cpweb [00:51:54] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 22.25, 21.41, 20.90 [00:52:00] PROBLEM - mw12 MediaWiki Rendering on mw12 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:53:03] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 3 datacenters are down: 2001:41d0:801:2000::1b80/cpweb, 149.56.141.75/cpweb, 2607:5300:201:3100::5ebc/cpweb [00:54:00] RECOVERY - mw12 MediaWiki Rendering on mw12 is OK: HTTP OK: HTTP/1.1 200 OK - 20526 bytes in 4.592 second response time [00:55:03] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [00:55:31] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [00:56:48] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.79, 3.85, 3.94 [00:57:25] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.85, 5.23, 4.53 [00:59:25] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.64, 5.64, 4.75 [01:01:25] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.72, 5.38, 4.74 [01:03:18] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 18.83, 19.84, 20.32 [01:05:25] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 3.42, 4.63, 4.60 [01:10:27] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 5.39, 4.10, 3.92 [01:10:39] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:13:41] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.88, 6.36, 5.65 [01:14:06] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.83, 6.52, 6.20 [01:14:39] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:15:40] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 6.16, 6.29, 5.72 [01:17:54] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.24, 6.21, 6.17 [01:20:51] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.82, 6.78, 6.56 [01:22:15] PROBLEM - cp31 Stunnel Http for mw12 on cp31 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [01:22:51] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.98, 6.50, 6.48 [01:24:14] RECOVERY - cp31 Stunnel Http for mw12 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 14557 bytes in 2.901 second response time [01:24:31] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.83, 6.99, 6.54 [01:24:54] [02mw-config] 07AgentIsai opened pull request 03#4313: Add sitenotice about Comments being unavailable - 13https://git.io/JyDgB [01:25:51] miraheze/mw-config - AgentIsai the build passed. [01:27:02] [02mw-config] 07AgentIsai synchronize pull request 03#4313: Add sitenotice about Comments being unavailable - 13https://git.io/JyDgB [01:28:01] miraheze/mw-config - AgentIsai the build passed. [01:28:19] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 4.82, 6.03, 6.25 [01:28:25] [02mw-config] 07AgentIsai commented on pull request 03#4313: Add sitenotice about Comments being unavailable - 13https://git.io/JyDgz [01:28:43] [02mw-config] 07AgentIsai edited a comment on pull request 03#4313: Add sitenotice about Comments being unavailable - 13https://git.io/JyDgz [01:29:12] [02mw-config] 07Universal-Omega closed pull request 03#4313: Add sitenotice about Comments being unavailable - 13https://git.io/JyDgB [01:29:14] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JyDgg [01:29:15] [02miraheze/mw-config] 07AgentIsai 030504761 - Add sitenotice about Comments being unavailable (#4313) [01:29:38] [02miraheze/mw-config] 07github-actions[bot] pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JyDg2 [01:29:39] [02miraheze/mw-config] 07github-actions 031822880 - CI: lint code to MediaWiki standards [01:30:15] miraheze/mw-config - Universal-Omega the build passed. [01:33:22] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 8.03, 6.73, 6.06 [01:34:13] [02mw-config] 07Universal-Omega edited pull request 03#4312: Update CreateWiki subdomain blacklist - 13https://git.io/JyD0K [01:34:52] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.88, 6.45, 6.30 [01:35:21] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 4.67, 6.11, 5.93 [01:36:51] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.58, 7.17, 6.58 [01:37:18] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 51.195.220.68/cpweb [01:38:51] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.19, 6.46, 6.40 [01:39:14] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [01:41:25] !log [@mw11] starting deploy of {'config': True} to ovlon [01:41:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:41:39] !log [@mw11] finished deploy of {'config': True} to ovlon - SUCCESS in 14s [01:41:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:42:27] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.94, 3.48, 3.91 [01:44:27] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.19, 3.68, 3.92 [01:46:52] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.08, 7.43, 6.92 [01:48:51] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.29, 7.57, 7.02 [01:50:27] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.89, 3.54, 3.84 [01:51:01] PROBLEM - cp21 Stunnel Http for mw9 on cp21 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [01:51:16] PROBLEM - mw9 MediaWiki Rendering on mw9 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:51:30] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.87, 6.69, 5.95 [01:52:27] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.85, 4.01, 3.97 [01:52:51] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.42, 7.84, 7.29 [01:53:01] RECOVERY - cp21 Stunnel Http for mw9 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 14556 bytes in 5.691 second response time [01:53:22] RECOVERY - mw9 MediaWiki Rendering on mw9 is OK: HTTP OK: HTTP/1.1 200 OK - 20524 bytes in 6.854 second response time [01:54:27] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.59, 3.52, 3.80 [01:54:37] !log [@test3] starting deploy of {'config': True} to skip [01:54:38] !log [@test3] finished deploy of {'config': True} to skip - SUCCESS in 0s [01:54:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:55:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:55:29] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 6.08, 6.43, 6.02 [02:04:27] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.23, 2.93, 3.39 [02:04:51] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.60, 6.32, 6.73 [02:13:19] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.71, 3.50, 3.43 [02:15:16] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.19, 3.87, 3.59 [02:17:13] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.37, 3.53, 3.49 [02:21:06] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 7.62, 7.26, 6.37 [02:21:26] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.68, 6.63, 5.77 [02:23:25] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 6.31, 6.60, 5.88 [02:25:01] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.37, 3.17, 3.38 [02:25:06] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.07, 6.31, 6.21 [02:26:58] [02miraheze/MirahezeDebug] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JyDr4 [02:27:00] [02miraheze/MirahezeDebug] 07Universal-Omega 039ae244d - Update popup.html [02:27:01] [02MirahezeDebug] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://git.io/fpk7C [02:27:10] [02MirahezeDebug] 07Universal-Omega opened pull request 03#6: Switch servers - 13https://git.io/JyDrR [02:27:48] [02miraheze/MirahezeDebug] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JyDrw [02:27:50] [02miraheze/MirahezeDebug] 07Universal-Omega 032c6399e - Update background.js [02:27:51] [02MirahezeDebug] 07Universal-Omega synchronize pull request 03#6: Switch servers - 13https://git.io/JyDrR [02:34:43] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.82, 3.45, 3.42 [02:38:37] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.19, 3.15, 3.30 [02:46:27] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.63, 3.77, 3.54 [02:50:27] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.21, 3.81, 3.59 [02:51:06] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.94, 6.51, 6.32 [02:52:16] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 9.08, 7.12, 6.29 [02:52:27] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.08, 3.50, 3.50 [02:53:06] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 8.28, 7.31, 6.64 [02:54:16] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.62, 6.82, 6.27 [02:54:27] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.59, 3.80, 3.60 [02:54:43] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 7.06, 6.77, 6.28 [02:55:06] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 7.67, 7.16, 6.65 [02:56:15] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.84, 6.12, 6.09 [02:57:06] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 8.21, 7.25, 6.73 [02:58:43] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 5.42, 6.65, 6.39 [02:59:06] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 4.92, 6.40, 6.48 [03:06:28] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.69, 3.66, 3.81 [03:09:40] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:10:27] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.61, 4.01, 3.91 [03:12:01] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.97, 6.64, 6.22 [03:12:27] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.37, 3.90, 3.89 [03:12:39] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:13:57] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 6.42, 6.49, 6.21 [03:16:27] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.11, 3.94, 3.89 [03:18:27] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 4.00, 3.79, 3.83 [03:20:27] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.15, 3.91, 3.87 [03:20:43] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 7.34, 6.31, 5.61 [03:22:43] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 5.09, 5.95, 5.58 [03:23:39] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.72, 7.16, 6.47 [03:24:27] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.69, 3.83, 3.85 [03:25:36] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.32, 6.94, 6.49 [03:26:27] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.55, 4.05, 3.92 [03:27:33] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.90, 6.64, 6.43 [03:28:27] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.31, 3.92, 3.90 [03:32:39] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:34:28] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.93, 3.67, 3.72 [03:36:27] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.10, 3.50, 3.66 [03:36:39] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:41:39] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:42:27] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 6.18, 4.17, 3.83 [03:43:33] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 7.65, 6.99, 6.01 [03:44:27] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.29, 3.85, 3.76 [03:46:27] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.32, 4.06, 3.85 [03:46:39] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:47:23] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 3.80, 5.76, 5.76 [03:47:42] [02mw-config] 07Universal-Omega synchronize pull request 03#4312: Update CreateWiki subdomain blacklist - 13https://git.io/JyD0K [03:47:43] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/JyDA2 [03:47:44] [02miraheze/mw-config] 07Universal-Omega 03b57c786 - Update LocalSettings.php [03:48:27] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.60, 3.47, 3.66 [03:48:41] miraheze/mw-config - Universal-Omega the build passed. [03:49:26] [02mw-config] 07Universal-Omega edited pull request 03#4312: Update CreateWiki subdomain blacklist - 13https://git.io/JyD0K [03:52:51] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.02, 7.14, 6.34 [03:54:27] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.09, 3.51, 3.57 [03:54:51] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.51, 6.88, 6.35 [03:56:51] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 6.06, 6.51, 6.27 [03:58:27] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.86, 3.83, 3.68 [04:14:21] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/JyyTF [04:14:22] [02miraheze/mw-config] 07Universal-Omega 03985d4f7 - Update LocalSettings.php [04:14:24] [02mw-config] 07Universal-Omega synchronize pull request 03#4312: Update CreateWiki subdomain blacklist - 13https://git.io/JyD0K [04:14:40] [02mw-config] 07Universal-Omega edited pull request 03#4312: Update CreateWiki subdomain blacklist - 13https://git.io/JyD0K [04:15:30] miraheze/mw-config - Universal-Omega the build passed. [04:15:54] [02mw-config] 07Universal-Omega edited pull request 03#4312: Update CreateWiki subdomain blacklist - 13https://git.io/JyD0K [04:16:27] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.86, 3.91, 3.68 [04:16:33] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/Jyykp [04:16:35] [02miraheze/mw-config] 07Universal-Omega 03a30b63e - Update LocalSettings.php [04:16:36] [02mw-config] 07Universal-Omega synchronize pull request 03#4312: Update CreateWiki subdomain blacklist - 13https://git.io/JyD0K [04:16:53] [02mw-config] 07Universal-Omega edited pull request 03#4312: Update CreateWiki subdomain blacklist - 13https://git.io/JyD0K [04:17:35] miraheze/mw-config - Universal-Omega the build passed. [04:20:28] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.56, 3.66, 3.62 [04:26:27] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 5.17, 4.10, 3.81 [04:30:27] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.21, 3.93, 3.83 [04:36:39] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:42:27] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 5.04, 4.02, 3.84 [04:47:39] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:48:27] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.42, 3.78, 3.80 [04:52:39] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:54:28] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.78, 3.91, 3.82 [04:55:39] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:58:27] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.88, 3.89, 3.84 [05:02:27] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.37, 3.82, 3.79 [05:08:27] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.20, 3.97, 3.91 [05:12:14] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.14, 5.88, 5.40 [05:12:27] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.16, 3.63, 3.75 [05:14:12] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 3.76, 5.02, 5.14 [05:14:51] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.98, 6.58, 5.77 [05:15:39] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:16:27] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.22, 3.65, 3.74 [05:16:51] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.57, 6.31, 5.78 [05:23:39] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:26:27] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.76, 3.79, 3.66 [05:30:27] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.83, 3.46, 3.57 [05:33:39] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:34:27] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.12, 2.97, 3.36 [05:40:39] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:44:06] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 7.58, 6.09, 5.56 [05:46:00] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 6.12, 6.10, 5.63 [06:03:48] RECOVERY - db13 Disk Space on db13 is OK: DISK OK - free space: / 48974 MB (11% inode=98%); [06:03:52] PROBLEM - db11 Current Load on db11 is CRITICAL: CRITICAL - load average: 10.54, 7.02, 4.29 [06:07:48] PROBLEM - db13 Disk Space on db13 is WARNING: DISK WARNING - free space: / 48329 MB (10% inode=98%); [06:10:18] PROBLEM - db13 Current Load on db13 is WARNING: WARNING - load average: 7.65, 6.69, 4.51 [06:11:34] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.15, 2.21, 1.43 [06:11:48] RECOVERY - db13 Disk Space on db13 is OK: DISK OK - free space: / 56102 MB (12% inode=98%); [06:12:12] RECOVERY - db13 Current Load on db13 is OK: OK - load average: 5.92, 6.46, 4.69 [06:13:34] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.92, 1.67, 1.32 [06:27:48] PROBLEM - db13 Disk Space on db13 is WARNING: DISK WARNING - free space: / 48303 MB (10% inode=98%); [06:29:52] PROBLEM - db11 Current Load on db11 is WARNING: WARNING - load average: 7.39, 7.84, 7.93 [06:41:52] PROBLEM - db11 Current Load on db11 is CRITICAL: CRITICAL - load average: 9.93, 8.08, 7.75 [06:43:52] PROBLEM - db11 Current Load on db11 is WARNING: WARNING - load average: 7.23, 7.94, 7.76 [06:47:52] PROBLEM - db11 Current Load on db11 is CRITICAL: CRITICAL - load average: 8.14, 7.70, 7.68 [06:49:52] PROBLEM - db11 Current Load on db11 is WARNING: WARNING - load average: 7.88, 7.54, 7.61 [06:51:20] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/Jyyxl [06:51:21] [02miraheze/mw-config] 07Universal-Omega 03e37e2ed - Update LocalSettings.php [06:51:23] [02mw-config] 07Universal-Omega synchronize pull request 03#4312: Update CreateWiki subdomain blacklist - 13https://git.io/JyD0K [06:51:52] PROBLEM - db11 Current Load on db11 is CRITICAL: CRITICAL - load average: 11.53, 8.76, 8.02 [06:52:11] [02mw-config] 07Universal-Omega edited pull request 03#4312: Update CreateWiki subdomain blacklist - 13https://git.io/JyD0K [06:52:22] miraheze/mw-config - Universal-Omega the build passed. [06:55:30] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/Jyypn [06:55:32] [02miraheze/mw-config] 07Universal-Omega 03ed97d4e - Update LocalSettings.php [06:55:33] [02mw-config] 07Universal-Omega synchronize pull request 03#4312: Update CreateWiki subdomain blacklist - 13https://git.io/JyD0K [06:55:59] [02mw-config] 07Universal-Omega edited pull request 03#4312: Update CreateWiki subdomain blacklist - 13https://git.io/JyD0K [06:56:35] miraheze/mw-config - Universal-Omega the build passed. [06:57:36] [02mw-config] 07Universal-Omega edited pull request 03#4312: Redo CreateWiki subdomain blacklist - 13https://git.io/JyD0K [07:01:38] [02mw-config] 07Universal-Omega edited pull request 03#4312: Redo CreateWiki subdomain blacklist - 13https://git.io/JyD0K [07:05:52] PROBLEM - db11 Current Load on db11 is WARNING: WARNING - load average: 7.15, 7.71, 8.00 [07:11:52] PROBLEM - db11 Current Load on db11 is CRITICAL: CRITICAL - load average: 8.96, 7.74, 7.83 [07:15:52] PROBLEM - db11 Current Load on db11 is WARNING: WARNING - load average: 7.04, 7.57, 7.77 [07:21:52] [02mw-config] 07Universal-Omega edited pull request 03#4312: Redo CreateWiki subdomain blacklist - 13https://git.io/JyD0K [07:31:52] PROBLEM - db11 Current Load on db11 is CRITICAL: CRITICAL - load average: 8.24, 7.41, 7.45 [07:35:52] PROBLEM - db11 Current Load on db11 is WARNING: WARNING - load average: 7.00, 7.71, 7.60 [07:42:12] PROBLEM - db13 Current Load on db13 is WARNING: WARNING - load average: 7.93, 7.03, 6.44 [07:44:06] PROBLEM - db13 Current Load on db13 is CRITICAL: CRITICAL - load average: 8.06, 7.36, 6.63 [07:45:52] PROBLEM - db11 Current Load on db11 is CRITICAL: CRITICAL - load average: 10.81, 8.12, 7.60 [07:46:00] PROBLEM - db13 Current Load on db13 is WARNING: WARNING - load average: 6.93, 7.15, 6.64 [07:49:48] RECOVERY - db13 Current Load on db13 is OK: OK - load average: 5.12, 6.34, 6.43 [07:53:52] PROBLEM - db11 Current Load on db11 is WARNING: WARNING - load average: 6.34, 7.62, 7.69 [08:09:52] PROBLEM - db11 Current Load on db11 is CRITICAL: CRITICAL - load average: 8.27, 7.73, 7.60 [08:11:52] PROBLEM - db11 Current Load on db11 is WARNING: WARNING - load average: 7.87, 7.87, 7.67 [08:13:55] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.19, 2.00, 1.24 [08:15:52] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.41, 1.44, 1.12 [08:17:52] PROBLEM - db11 Current Load on db11 is CRITICAL: CRITICAL - load average: 8.39, 7.85, 7.69 [08:19:48] RECOVERY - db13 Disk Space on db13 is OK: DISK OK - free space: / 50709 MB (11% inode=98%); [08:19:53] PROBLEM - db11 Current Load on db11 is WARNING: WARNING - load average: 6.37, 7.34, 7.53 [08:20:39] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [08:23:39] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [08:23:52] PROBLEM - db11 Current Load on db11 is CRITICAL: CRITICAL - load average: 8.84, 8.04, 7.75 [08:25:48] PROBLEM - db13 Disk Space on db13 is WARNING: DISK WARNING - free space: / 47715 MB (10% inode=98%); [08:25:52] PROBLEM - db11 Current Load on db11 is WARNING: WARNING - load average: 7.04, 7.54, 7.59 [08:27:52] PROBLEM - db11 Current Load on db11 is CRITICAL: CRITICAL - load average: 8.24, 7.60, 7.59 [08:29:52] PROBLEM - db11 Current Load on db11 is WARNING: WARNING - load average: 5.30, 6.90, 7.35 [08:35:48] RECOVERY - db13 Disk Space on db13 is OK: DISK OK - free space: / 49195 MB (11% inode=98%); [08:37:52] RECOVERY - db11 Current Load on db11 is OK: OK - load average: 5.26, 6.06, 6.78 [08:39:48] PROBLEM - db13 Disk Space on db13 is WARNING: DISK WARNING - free space: / 47677 MB (10% inode=98%); [08:48:39] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [08:49:52] PROBLEM - db11 Current Load on db11 is WARNING: WARNING - load average: 7.21, 7.04, 6.85 [08:51:39] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [08:55:52] RECOVERY - db11 Current Load on db11 is OK: OK - load average: 6.62, 6.73, 6.79 [09:01:39] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [09:02:44] PROBLEM - db11 Current Load on db11 is WARNING: WARNING - load average: 7.57, 6.91, 6.79 [09:06:40] RECOVERY - db11 Current Load on db11 is OK: OK - load average: 4.71, 6.21, 6.56 [09:13:39] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [09:31:06] PROBLEM - db11 Current Load on db11 is WARNING: WARNING - load average: 7.76, 6.99, 6.62 [09:33:39] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [09:36:39] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [09:38:56] PROBLEM - db11 Current Load on db11 is CRITICAL: CRITICAL - load average: 8.07, 7.44, 6.99 [09:42:51] PROBLEM - db11 Current Load on db11 is WARNING: WARNING - load average: 6.65, 7.41, 7.11 [10:01:39] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [10:06:22] RECOVERY - db11 Current Load on db11 is OK: OK - load average: 5.37, 6.23, 6.70 [10:08:39] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [10:13:39] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [10:19:39] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [10:29:39] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [10:32:53] [02mw-config] 07RhinosF1 edited pull request 03#4310: SCSVG: setup - 13https://git.io/JyipJ [10:32:55] [02mw-config] 07RhinosF1 edited pull request 03#4310: SCSVG: setup - 13https://git.io/JyipJ [10:33:01] [02mw-config] 07RhinosF1 edited pull request 03#4310: SCSVG: setup - 13https://git.io/JyipJ [10:33:41] [02miraheze/mw-config] 07RhinosF1 pushed 031 commit to 03RhinosF1-patch-2 [+0/-0/±1] 13https://git.io/JySmn [10:33:42] [02miraheze/mw-config] 07RhinosF1 0350057a8 - Cache: add variable config [10:33:44] [02mw-config] 07RhinosF1 synchronize pull request 03#4310: SCSVG: setup - 13https://git.io/JyipJ [10:34:35] [02miraheze/mw-config] 07RhinosF1 pushed 031 commit to 03RhinosF1-patch-2 [+0/-0/±1] 13https://git.io/JySmB [10:34:36] [02miraheze/mw-config] 07RhinosF1 03f86b8e7 - LS: Cache settings in GlobalCache [10:34:38] [02mw-config] 07RhinosF1 synchronize pull request 03#4310: SCSVG: setup - 13https://git.io/JyipJ [10:34:40] miraheze/mw-config - RhinosF1 the build has errored. [10:35:32] miraheze/mw-config - RhinosF1 the build has errored. [10:35:34] [02miraheze/mw-config] 07RhinosF1 pushed 031 commit to 03RhinosF1-patch-2 [+0/-0/±1] 13https://git.io/JySmz [10:35:36] [02miraheze/mw-config] 07RhinosF1 03db43bb2 - Update GlobalCache.php [10:35:37] [02mw-config] 07RhinosF1 synchronize pull request 03#4310: SCSVG: setup - 13https://git.io/JyipJ [10:36:42] miraheze/mw-config - RhinosF1 the build has errored. [10:37:50] [02miraheze/mw-config] 07RhinosF1 pushed 031 commit to 03RhinosF1-patch-2 [+0/-0/±1] 13https://git.io/JySmo [10:37:51] [02miraheze/mw-config] 07RhinosF1 032ac2028 - Database: Flip hostname based on DC [10:37:53] [02mw-config] 07RhinosF1 synchronize pull request 03#4310: SCSVG: setup - 13https://git.io/JyipJ [10:38:40] [02miraheze/mw-config] 07RhinosF1 pushed 031 commit to 03RhinosF1-patch-2 [+0/-0/±1] 13https://git.io/JySm6 [10:38:42] [02miraheze/mw-config] 07RhinosF1 03fb0d538 - Update Database.php [10:38:43] [02mw-config] 07RhinosF1 synchronize pull request 03#4310: SCSVG: setup - 13https://git.io/JyipJ [10:38:53] miraheze/mw-config - RhinosF1 the build has errored. [10:39:37] [02mw-config] 07RhinosF1 edited pull request 03#4310: SCSVG: setup - 13https://git.io/JyipJ [10:39:39] miraheze/mw-config - RhinosF1 the build has errored. [10:39:49] [02miraheze/mw-config] 07RhinosF1 pushed 031 commit to 03RhinosF1-patch-2 [+0/-0/±1] 13https://git.io/JySmS [10:39:51] [02miraheze/mw-config] 07RhinosF1 039664239 - Update GlobalCache.php [10:39:52] [02mw-config] 07RhinosF1 synchronize pull request 03#4310: SCSVG: setup - 13https://git.io/JyipJ [10:40:20] CosmicAlpha: ^ should auto-flip config to the local DC [10:40:27] will make setup faster [10:40:44] miraheze/mw-config - RhinosF1 the build has errored. [10:41:36] [02miraheze/mw-config] 07RhinosF1 pushed 031 commit to 03RhinosF1-patch-2 [+0/-0/±1] 13https://git.io/JySm7 [10:41:38] [02miraheze/mw-config] 07RhinosF1 031d79cb0 - Update GlobalCache.php [10:41:39] [02mw-config] 07RhinosF1 synchronize pull request 03#4310: SCSVG: setup - 13https://git.io/JyipJ [10:42:01] [02miraheze/mw-config] 07github-actions[bot] pushed 031 commit to 03RhinosF1-patch-2 [+0/-0/±2] 13https://git.io/JySmF [10:42:03] [02miraheze/mw-config] 07github-actions 03d39bde0 - CI: lint code to MediaWiki standards [10:42:04] [02mw-config] 07github-actions[bot] synchronize pull request 03#4310: SCSVG: setup - 13https://git.io/JyipJ [10:42:38] miraheze/mw-config - RhinosF1 the build passed. [10:42:39] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [10:49:19] [02puppet] 07RhinosF1 opened pull request 03#2183: setup new mw* with stunnel & prometheus - 13https://git.io/JySYc [10:50:49] [02puppet] 07RhinosF1 synchronize pull request 03#2183: setup new mw* with stunnel & prometheus - 13https://git.io/JySYc [10:53:38] [02puppet] 07RhinosF1 synchronize pull request 03#2183: setup new mw* with stunnel & prometheus - 13https://git.io/JySYc [10:54:29] [02puppet] 07RhinosF1 edited pull request 03#2183: setup new mw* with stunnel & prometheus - 13https://git.io/JySYc [10:57:39] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [11:01:00] [02puppet] 07RhinosF1 opened pull request 03#2184: varnish: add new mw servers - 13https://git.io/JySOU [11:04:39] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [11:09:39] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [11:18:35] [02mediawiki] 07RhinosF1 closed pull request 03#4786: Bump extensions/Math from `8f80b75` to `9ec9134` - 13https://git.io/JyWsE [11:18:37] [02miraheze/mediawiki] 07RhinosF1 deleted branch 03dependabot/submodules/REL1_37/extensions/Math-9ec9134 [11:18:39] [02mediawiki] 07RhinosF1 deleted branch 03dependabot/submodules/REL1_37/extensions/Math-9ec9134 - 13https://git.io/vbL5b [11:18:40] [02miraheze/mediawiki] 07RhinosF1 pushed 031 commit to 03REL1_37 [+0/-0/±1] 13https://git.io/JyS38 [11:18:42] [02miraheze/mediawiki] 07dependabot[bot] 03984c665 - Bump extensions/Math from `8f80b75` to `9ec9134` (#4786) [11:18:43] [02mediawiki] 07RhinosF1 closed pull request 03#4785: Bump extensions/MassMessage from `1ab58e8` to `3a0d106` - 13https://git.io/JyWsR [11:18:45] [02mediawiki] 07RhinosF1 deleted branch 03dependabot/submodules/REL1_37/extensions/MassMessage-3a0d106 - 13https://git.io/vbL5b [11:18:46] [02miraheze/mediawiki] 07RhinosF1 pushed 031 commit to 03REL1_37 [+0/-0/±1] 13https://git.io/JyS3B [11:18:48] [02miraheze/mediawiki] 07dependabot[bot] 03dab9364 - Bump extensions/MassMessage from `1ab58e8` to `3a0d106` (#4785) [11:18:49] [02miraheze/mediawiki] 07RhinosF1 deleted branch 03dependabot/submodules/REL1_37/extensions/MassMessage-3a0d106 [11:18:51] [02mediawiki] 07RhinosF1 closed pull request 03#4784: Bump extensions/MobileFrontend from `016576e` to `6c4bee5` - 13https://git.io/JyWs4 [11:18:52] [02miraheze/mediawiki] 07RhinosF1 pushed 031 commit to 03REL1_37 [+0/-0/±1] 13https://git.io/JyS30 [11:18:54] [02miraheze/mediawiki] 07dependabot[bot] 036babba2 - Bump extensions/MobileFrontend from `016576e` to `6c4bee5` (#4784) [11:18:55] [02miraheze/mediawiki] 07RhinosF1 deleted branch 03dependabot/submodules/REL1_37/extensions/MobileFrontend-6c4bee5 [11:18:57] [02mediawiki] 07RhinosF1 deleted branch 03dependabot/submodules/REL1_37/extensions/MobileFrontend-6c4bee5 - 13https://git.io/vbL5b [11:18:58] [02mediawiki] 07RhinosF1 closed pull request 03#4783: Bump extensions/MsCatSelect from `35df952` to `6eaf689` - 13https://git.io/JyWsl [11:19:00] [02miraheze/mediawiki] 07RhinosF1 pushed 031 commit to 03REL1_37 [+0/-0/±1] 13https://git.io/JyS3E [11:19:01] [02miraheze/mediawiki] 07dependabot[bot] 03b4b905d - Bump extensions/MsCatSelect from `35df952` to `6eaf689` (#4783) [11:19:03] [02mediawiki] 07RhinosF1 deleted branch 03dependabot/submodules/REL1_37/extensions/MsCatSelect-6eaf689 - 13https://git.io/vbL5b [11:19:04] [02miraheze/mediawiki] 07RhinosF1 deleted branch 03dependabot/submodules/REL1_37/extensions/MsCatSelect-6eaf689 [11:19:06] [02mediawiki] 07RhinosF1 closed pull request 03#4782: Bump extensions/Nuke from `0ce1cef` to `c193f88` - 13https://git.io/JyWsC [11:19:07] [02miraheze/mediawiki] 07RhinosF1 pushed 031 commit to 03REL1_37 [+0/-0/±1] 13https://git.io/JyS3z [11:19:09] [02miraheze/mediawiki] 07dependabot[bot] 038e82370 - Bump extensions/Nuke from `0ce1cef` to `c193f88` (#4782) [11:19:10] [02mediawiki] 07RhinosF1 deleted branch 03dependabot/submodules/REL1_37/extensions/Nuke-c193f88 - 13https://git.io/vbL5b [11:19:12] [02miraheze/mediawiki] 07RhinosF1 deleted branch 03dependabot/submodules/REL1_37/extensions/Nuke-c193f88 [11:19:13] [02mediawiki] 07RhinosF1 closed pull request 03#4781: Bump extensions/GlobalUsage from `08db81e` to `1ed72d4` - 13https://git.io/JyWsn [11:19:15] [02miraheze/mediawiki] 07RhinosF1 pushed 031 commit to 03REL1_37 [+0/-0/±1] 13https://git.io/JyS3g [11:19:16]