[00:03:22] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 10.10, 7.35, 5.95 [00:03:32] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [00:08:25] PROBLEM - graylog2 Current Load on graylog2 is CRITICAL: CRITICAL - load average: 4.24, 3.25, 2.37 [00:08:32] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [00:09:54] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.42, 3.53, 3.33 [00:10:23] PROBLEM - graylog2 Current Load on graylog2 is WARNING: WARNING - load average: 3.87, 3.37, 2.52 [00:12:02] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [00:12:21] RECOVERY - graylog2 Current Load on graylog2 is OK: OK - load average: 3.25, 3.33, 2.61 [00:15:46] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.02, 3.34, 3.31 [00:17:02] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [00:19:28] PROBLEM - cp20 Stunnel Http for mw8 on cp20 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [00:19:30] PROBLEM - cp30 Stunnel Http for mw8 on cp30 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [00:20:32] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [00:20:55] PROBLEM - cp31 Stunnel Http for mw8 on cp31 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [00:21:43] PROBLEM - mw8 MediaWiki Rendering on mw8 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:22:17] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 7 datacenters are down: 51.195.220.68/cpweb, 198.244.148.90/cpweb, 2001:41d0:801:2000::4c25/cpweb, 149.56.140.43/cpweb, 149.56.141.75/cpweb, 2607:5300:201:3100::929a/cpweb, 2607:5300:201:3100::5ebc/cpweb [00:22:59] RECOVERY - cp31 Stunnel Http for mw8 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15860 bytes in 7.767 second response time [00:23:10] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 1 datacenter is down: 2001:41d0:801:2000::4c25/cpweb [00:23:33] RECOVERY - cp30 Stunnel Http for mw8 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15860 bytes in 0.664 second response time [00:23:34] RECOVERY - cp20 Stunnel Http for mw8 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15868 bytes in 0.178 second response time [00:25:09] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [00:25:49] RECOVERY - mw8 MediaWiki Rendering on mw8 is OK: HTTP OK: HTTP/1.1 200 OK - 20515 bytes in 1.622 second response time [00:26:08] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [00:30:32] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [00:31:37] PROBLEM - madzebrascience.wiki - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:34:14] [02mediawiki] 07Universal-Omega commented on pull request 03#4673: Bump skins/Femiwiki from `be075c6` to `9b3f30b` - 13https://git.io/JDr3n [00:34:17] [02mediawiki] 07dependabot[bot] edited pull request 03#4673: Bump skins/Femiwiki from `be075c6` to `9b3f30b` - 13https://git.io/JDafT [00:34:36] [02mediawiki] 07Universal-Omega commented on pull request 03#4655: Bump extensions/PortableInfobox from `b9f59b5` to `1e8dad4` - 13https://git.io/JDr3C [00:34:39] [02mediawiki] 07dependabot[bot] edited pull request 03#4655: Bump extensions/PortableInfobox from `b9f59b5` to `1e8dad4` - 13https://git.io/JD25y [00:35:12] [02mediawiki] 07Universal-Omega commented on pull request 03#4654: Bump skins/MinervaNeue from `10e4fdc` to `b7767bd` - 13https://git.io/JDr34 [00:35:14] [02mediawiki] 07dependabot[bot] edited pull request 03#4654: Bump skins/MinervaNeue from `10e4fdc` to `b7767bd` - 13https://git.io/JD251 [00:35:16] [02mediawiki] 07dependabot[bot] edited pull request 03#4673: Bump skins/Femiwiki from `be075c6` to `9b3f30b` - 13https://git.io/JDafT [00:35:20] [02mediawiki] 07dependabot[bot] opened pull request 03#4675: Bump skins/Femiwiki from `be075c6` to `8a99aef` - 13https://git.io/JDr30 [00:35:22] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_37/skins/Femiwiki-8a99aef [+0/-0/±1] 13https://git.io/JDr3E [00:35:23] [02miraheze/mediawiki] 07dependabot[bot] 03cc6c9d0 - Bump skins/Femiwiki from `be075c6` to `8a99aef` [00:35:25] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_37/skins/Femiwiki-8a99aef - 13https://git.io/vbL5b [00:35:26] [02mediawiki] 07dependabot[bot] commented on pull request 03#4673: Bump skins/Femiwiki from `be075c6` to `9b3f30b` - 13https://git.io/JDr3u [00:35:28] [02mediawiki] 07dependabot[bot] closed pull request 03#4673: Bump skins/Femiwiki from `be075c6` to `9b3f30b` - 13https://git.io/JDafT [00:35:29] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_37/skins/Femiwiki-9b3f30b - 13https://git.io/vbL5b [00:35:31] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_37/skins/Femiwiki-9b3f30b [00:35:44] [02mediawiki] 07dependabot[bot] edited pull request 03#4655: Bump extensions/PortableInfobox from `b9f59b5` to `1e8dad4` - 13https://git.io/JD25y [00:35:45] [02mediawiki] 07github-actions[bot] labeled pull request 03#4675: Bump skins/Femiwiki from `be075c6` to `8a99aef` - 13https://git.io/JDr30 [00:35:47] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_37/extensions/PortableInfobox-4a0bac9 [+0/-0/±1] 13https://git.io/JDr3g [00:35:49] [02miraheze/mediawiki] 07dependabot[bot] 03a7fed1c - Bump extensions/PortableInfobox from `b9f59b5` to `4a0bac9` [00:35:50] [02mediawiki] 07dependabot[bot] opened pull request 03#4676: Bump extensions/PortableInfobox from `b9f59b5` to `4a0bac9` - 13https://git.io/JDr3a [00:35:52] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_37/extensions/PortableInfobox-4a0bac9 - 13https://git.io/vbL5b [00:35:53] [02mediawiki] 07dependabot[bot] commented on pull request 03#4655: Bump extensions/PortableInfobox from `b9f59b5` to `1e8dad4` - 13https://git.io/JDr3V [00:35:55] [02mediawiki] 07dependabot[bot] closed pull request 03#4655: Bump extensions/PortableInfobox from `b9f59b5` to `1e8dad4` - 13https://git.io/JD25y [00:35:56] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_37/extensions/PortableInfobox-1e8dad4 - 13https://git.io/vbL5b [00:35:58] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_37/extensions/PortableInfobox-1e8dad4 [00:35:59] [02mediawiki] 07Universal-Omega commented on pull request 03#4653: Bump extensions/CiteThisPage from `aa11f2b` to `83517a1` - 13https://git.io/JDr3w [00:36:01] [02mediawiki] 07dependabot[bot] edited pull request 03#4653: Bump extensions/CiteThisPage from `aa11f2b` to `83517a1` - 13https://git.io/JD25i [00:36:02] [02mediawiki] 07Universal-Omega commented on pull request 03#4652: Bump extensions/WikiEditor from `d6b2db9` to `45e312b` - 13https://git.io/JDr3r [00:36:05] [02mediawiki] 07dependabot[bot] edited pull request 03#4652: Bump extensions/WikiEditor from `d6b2db9` to `45e312b` - 13https://git.io/JD25M [00:36:10] [02mediawiki] 07Universal-Omega commented on pull request 03#4651: Bump extensions/BetaFeatures from `21d3add` to `8ce691e` - 13https://git.io/JDr3o [00:36:12] [02mediawiki] 07github-actions[bot] labeled pull request 03#4676: Bump extensions/PortableInfobox from `b9f59b5` to `4a0bac9` - 13https://git.io/JDr3a [00:36:13] [02mediawiki] 07dependabot[bot] edited pull request 03#4651: Bump extensions/BetaFeatures from `21d3add` to `8ce691e` - 13https://git.io/JD25K [00:36:15] [02mediawiki] 07github-actions[bot] labeled pull request 03#4676: Bump extensions/PortableInfobox from `b9f59b5` to `4a0bac9` - 13https://git.io/JDr3a [00:36:19] [02mediawiki] 07Universal-Omega commented on pull request 03#4650: Bump extensions/PageImages from `890e06a` to `c4f13bd` - 13https://git.io/JDr36 [00:36:21] [02mediawiki] 07dependabot[bot] edited pull request 03#4654: Bump skins/MinervaNeue from `10e4fdc` to `b7767bd` - 13https://git.io/JD251 [00:36:22] [02mediawiki] 07dependabot[bot] edited pull request 03#4650: Bump extensions/PageImages from `890e06a` to `c4f13bd` - 13https://git.io/JD25r [00:36:24] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_37/skins/MinervaNeue-b7767bd [+0/-0/±1] 13https://git.io/JDr3P [00:36:25] [02miraheze/mediawiki] 07dependabot[bot] 032d6b46d - Bump skins/MinervaNeue from `10e4fdc` to `b7767bd` [00:36:27] [02mediawiki] 07dependabot[bot] synchronize pull request 03#4654: Bump skins/MinervaNeue from `10e4fdc` to `b7767bd` - 13https://git.io/JD251 [00:36:28] [02mediawiki] 07Universal-Omega commented on pull request 03#4649: Bump extensions/MassMessage from `534a726` to `1ab58e8` - 13https://git.io/JDr3X [00:36:30] [02mediawiki] 07dependabot[bot] edited pull request 03#4649: Bump extensions/MassMessage from `534a726` to `1ab58e8` - 13https://git.io/JD25V [00:36:35] [02mediawiki] 07Universal-Omega commented on pull request 03#4648: Bump extensions/RevisionSlider from `67a524c` to `2e38ae3` - 13https://git.io/JDr31 [00:36:36] [02mediawiki] 07dependabot[bot] edited pull request 03#4648: Bump extensions/RevisionSlider from `67a524c` to `2e38ae3` - 13https://git.io/JD25g [00:36:42] [02mediawiki] 07Universal-Omega commented on pull request 03#4647: Bump skins/MonoBook from `8759b85` to `b349ede` - 13https://git.io/JDr3M [00:36:43] [02mediawiki] 07dependabot[bot] edited pull request 03#4647: Bump skins/MonoBook from `8759b85` to `b349ede` - 13https://git.io/JD25u [00:36:48] [02mediawiki] 07github-actions[bot] labeled pull request 03#4654: Bump skins/MinervaNeue from `10e4fdc` to `b7767bd` - 13https://git.io/JD251 [00:36:51] [02mediawiki] 07Universal-Omega commented on pull request 03#4646: Bump extensions/Popups from `01ab6ad` to `db5cc44` - 13https://git.io/JDr3y [00:36:53] [02mediawiki] 07dependabot[bot] edited pull request 03#4653: Bump extensions/CiteThisPage from `aa11f2b` to `83517a1` - 13https://git.io/JD25i [00:36:54] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_37/extensions/CiteThisPage-83517a1 [+0/-0/±1] 13https://git.io/JDr3S [00:36:56] [02miraheze/mediawiki] 07dependabot[bot] 03ba4377c - Bump extensions/CiteThisPage from `aa11f2b` to `83517a1` [00:36:58] [02mediawiki] 07dependabot[bot] edited pull request 03#4646: Bump extensions/Popups from `01ab6ad` to `db5cc44` - 13https://git.io/JD25R [00:36:59] [02mediawiki] 07dependabot[bot] synchronize pull request 03#4653: Bump extensions/CiteThisPage from `aa11f2b` to `83517a1` - 13https://git.io/JD25i [00:37:13] [02mediawiki] 07dependabot[bot] edited pull request 03#4652: Bump extensions/WikiEditor from `d6b2db9` to `45e312b` - 13https://git.io/JD25M [00:37:15] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_37/extensions/WikiEditor-45e312b [+0/-0/±1] 13https://git.io/JDr3Q [00:37:17] [02miraheze/mediawiki] 07dependabot[bot] 034e114d0 - Bump extensions/WikiEditor from `d6b2db9` to `45e312b` [00:37:18] [02mediawiki] 07dependabot[bot] synchronize pull request 03#4652: Bump extensions/WikiEditor from `d6b2db9` to `45e312b` - 13https://git.io/JD25M [00:37:20] [02mediawiki] 07dependabot[bot] edited pull request 03#4649: Bump extensions/MassMessage from `534a726` to `1ab58e8` - 13https://git.io/JD25V [00:37:21] [02mediawiki] 07github-actions[bot] labeled pull request 03#4653: Bump extensions/CiteThisPage from `aa11f2b` to `83517a1` - 13https://git.io/JD25i [00:37:23] [02mediawiki] 07dependabot[bot] edited pull request 03#4650: Bump extensions/PageImages from `890e06a` to `c4f13bd` - 13https://git.io/JD25r [00:37:24] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_37/extensions/MassMessage-1ab58e8 [+0/-0/±1] 13https://git.io/JDr37 [00:37:26] [02miraheze/mediawiki] 07dependabot[bot] 0307a2850 - Bump extensions/MassMessage from `534a726` to `1ab58e8` [00:37:27] [02mediawiki] 07dependabot[bot] edited pull request 03#4651: Bump extensions/BetaFeatures from `21d3add` to `8ce691e` - 13https://git.io/JD25K [00:37:29] [02mediawiki] 07dependabot[bot] synchronize pull request 03#4649: Bump extensions/MassMessage from `534a726` to `1ab58e8` - 13https://git.io/JD25V [00:37:30] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_37/extensions/PageImages-c4f13bd [+0/-0/±1] 13https://git.io/JDr35 [00:37:32] [02miraheze/mediawiki] 07dependabot[bot] 03cf4cba5 - Bump extensions/PageImages from `890e06a` to `c4f13bd` [00:37:33] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_37/extensions/BetaFeatures-8ce691e [+0/-0/±1] 13https://git.io/JDr3d [00:37:35] [02miraheze/mediawiki] 07dependabot[bot] 030a62c09 - Bump extensions/BetaFeatures from `21d3add` to `8ce691e` [00:37:36] [02mediawiki] 07dependabot[bot] synchronize pull request 03#4650: Bump extensions/PageImages from `890e06a` to `c4f13bd` - 13https://git.io/JD25r [00:37:38] [02mediawiki] 07dependabot[bot] synchronize pull request 03#4651: Bump extensions/BetaFeatures from `21d3add` to `8ce691e` - 13https://git.io/JD25K [00:37:39] [02mediawiki] 07dependabot[bot] edited pull request 03#4648: Bump extensions/RevisionSlider from `67a524c` to `2e38ae3` - 13https://git.io/JD25g [00:37:41] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_37/extensions/RevisionSlider-2e38ae3 [+0/-0/±1] 13https://git.io/JDr3N [00:37:42] [02miraheze/mediawiki] 07dependabot[bot] 03fcb2f99 - Bump extensions/RevisionSlider from `67a524c` to `2e38ae3` [00:37:44] [02mediawiki] 07dependabot[bot] synchronize pull request 03#4648: Bump extensions/RevisionSlider from `67a524c` to `2e38ae3` - 13https://git.io/JD25g [00:37:45] [02mediawiki] 07dependabot[bot] edited pull request 03#4647: Bump skins/MonoBook from `8759b85` to `b349ede` - 13https://git.io/JD25u [00:37:47] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_37/skins/MonoBook-b349ede [+0/-0/±1] 13https://git.io/JDr3x [00:37:48] [02miraheze/mediawiki] 07dependabot[bot] 033edac5a - Bump skins/MonoBook from `8759b85` to `b349ede` [00:37:50] [02mediawiki] 07dependabot[bot] synchronize pull request 03#4647: Bump skins/MonoBook from `8759b85` to `b349ede` - 13https://git.io/JD25u [00:37:51] [02mediawiki] 07github-actions[bot] labeled pull request 03#4649: Bump extensions/MassMessage from `534a726` to `1ab58e8` - 13https://git.io/JD25V [00:37:53] [02mediawiki] 07github-actions[bot] labeled pull request 03#4652: Bump extensions/WikiEditor from `d6b2db9` to `45e312b` - 13https://git.io/JD25M [00:37:54] [02mediawiki] 07github-actions[bot] labeled pull request 03#4650: Bump extensions/PageImages from `890e06a` to `c4f13bd` - 13https://git.io/JD25r [00:37:56] [02mediawiki] 07github-actions[bot] labeled pull request 03#4651: Bump extensions/BetaFeatures from `21d3add` to `8ce691e` - 13https://git.io/JD25K [00:37:57] [02mediawiki] 07dependabot[bot] edited pull request 03#4646: Bump extensions/Popups from `01ab6ad` to `db5cc44` - 13https://git.io/JD25R [00:37:59] [02mediawiki] 07github-actions[bot] labeled pull request 03#4648: Bump extensions/RevisionSlider from `67a524c` to `2e38ae3` - 13https://git.io/JD25g [00:38:00] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_37/extensions/Popups-db5cc44 [+0/-0/±1] 13https://git.io/JDr3p [00:38:02] [02miraheze/mediawiki] 07dependabot[bot] 03b5c572a - Bump extensions/Popups from `01ab6ad` to `db5cc44` [00:38:03] [02mediawiki] 07dependabot[bot] synchronize pull request 03#4646: Bump extensions/Popups from `01ab6ad` to `db5cc44` - 13https://git.io/JD25R [00:38:05] [02mediawiki] 07github-actions[bot] labeled pull request 03#4647: Bump skins/MonoBook from `8759b85` to `b349ede` - 13https://git.io/JD25u [00:38:19] [02mediawiki] 07github-actions[bot] labeled pull request 03#4646: Bump extensions/Popups from `01ab6ad` to `db5cc44` - 13https://git.io/JD25R [00:48:38] PROBLEM - madzebrascience.wiki - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['dns101.registrar-servers.com.', 'dns102.registrar-servers.com.', 'dns1.registrar-servers.com.', 'dns2.registrar-servers.com.'], 'CNAME': None} [00:50:23] [02mediawiki] 07Universal-Omega closed pull request 03#4620: Bump skins/chameleon from `a923c46` to `821f6b0` - 13https://git.io/JD272 [00:50:25] [02mediawiki] 07dependabot[bot] commented on pull request 03#4620: Bump skins/chameleon from `a923c46` to `821f6b0` - 13https://git.io/JDrGU [00:50:28] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_37/skins/chameleon-821f6b0 - 13https://git.io/vbL5b [00:50:29] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_37/skins/chameleon-821f6b0 [00:52:40] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.70, 3.82, 3.35 [00:54:04] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 6.93, 6.38, 5.45 [00:54:53] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 2.50, 4.19, 5.82 [00:55:21] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 9.69, 6.99, 5.94 [00:55:26] PROBLEM - madzebrascience.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - madzebrascience.wiki reverse DNS resolves to a51062ecadbb5a26e.awsglobalaccelerator.com [00:56:02] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 5.05, 5.87, 5.38 [00:56:35] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.42, 3.91, 3.48 [00:58:32] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.66, 3.74, 3.47 [01:00:53] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.26, 4.51, 5.39 [01:02:15] RECOVERY - madzebrascience.wiki - reverse DNS on sslhost is OK: SSL OK - madzebrascience.wiki reverse DNS resolves to cp20.miraheze.org - NS RECORDS OK [01:02:24] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.17, 7.17, 6.00 [01:02:53] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 4.37, 4.12, 5.12 [01:04:22] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 13.31, 9.55, 7.01 [01:04:36] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 19.62, 21.60, 23.80 [01:04:53] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 3.20, 3.97, 4.95 [01:06:21] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.64, 3.14, 3.31 [01:06:27] [02miraheze/ManageWiki] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+1/-0/±0] 13https://git.io/JDrZD [01:06:29] [02miraheze/ManageWiki] 07Universal-Omega 0310171aa - Create phpunit test for ApiQueryWikiConfig [01:06:30] [02ManageWiki] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://git.io/vpSns [01:06:32] [02ManageWiki] 07Universal-Omega opened pull request 03#320: Create phpunit test for ApiQueryWikiConfig - 13https://git.io/JDrZy [01:09:31] miraheze/ManageWiki - Universal-Omega the build has errored. [01:10:16] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 4.13, 6.83, 6.69 [01:12:14] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.56, 6.64, 6.65 [01:12:36] [02miraheze/ManageWiki] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JDrns [01:12:38] [02miraheze/ManageWiki] 07Universal-Omega 034b12048 - Update extension.json [01:12:39] [02ManageWiki] 07Universal-Omega synchronize pull request 03#320: Create phpunit test for ApiQueryWikiConfig - 13https://git.io/JDrZy [01:13:20] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.30, 7.58, 7.93 [01:15:39] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.27, 6.87, 6.23 [01:16:36] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 14.27, 16.73, 20.00 [01:17:24] miraheze/ManageWiki - Universal-Omega the build passed. [01:17:37] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 4.69, 5.94, 5.96 [01:17:51] PROBLEM - madzebrascience.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - madzebrascience.wiki reverse DNS resolves to a51062ecadbb5a26e.awsglobalaccelerator.com [01:19:20] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 8.17, 7.41, 7.66 [01:20:36] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 20.59, 20.25, 20.77 [01:21:04] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 11.28, 8.08, 7.07 [01:22:36] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 17.96, 19.31, 20.36 [01:22:55] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.95, 3.89, 3.59 [01:23:04] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.46, 7.95, 7.16 [01:24:52] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 5.27, 4.30, 3.77 [01:26:32] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:27:04] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.83, 6.41, 6.72 [01:27:24] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.90, 6.36, 6.14 [01:29:20] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 5.78, 7.30, 7.71 [01:29:22] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.17, 5.83, 5.97 [01:31:32] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:36:34] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.00, 3.62, 3.74 [01:37:02] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:38:31] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.33, 3.71, 3.74 [01:39:20] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.15, 5.64, 6.71 [01:40:28] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.90, 3.84, 3.79 [01:42:46] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 21.03, 19.25, 18.79 [01:44:40] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 17.66, 18.46, 18.56 [01:46:21] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.14, 4.02, 3.89 [01:47:02] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:47:52] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 4.40, 4.28, 2.33 [01:48:21] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.17, 3.84, 3.84 [01:48:36] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 25.16, 21.02, 19.47 [01:49:24] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 9.70, 6.05, 4.27 [01:49:32] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:50:14] PROBLEM - mw8 MediaWiki Rendering on mw8 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:50:19] PROBLEM - cp21 Stunnel Http for mw8 on cp21 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [01:50:21] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.94, 4.10, 3.92 [01:51:23] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.41, 5.85, 4.42 [01:51:52] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 10.69, 7.80, 6.61 [01:52:13] RECOVERY - mw8 MediaWiki Rendering on mw8 is OK: HTTP OK: HTTP/1.1 200 OK - 20515 bytes in 2.107 second response time [01:52:18] RECOVERY - cp21 Stunnel Http for mw8 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15860 bytes in 1.859 second response time [01:52:21] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.75, 3.83, 3.84 [01:52:53] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.70, 6.94, 6.26 [01:53:22] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.27, 5.78, 4.55 [01:54:09] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 10.05, 7.12, 6.09 [01:54:21] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 5.37, 4.25, 3.98 [01:54:36] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 21.04, 23.14, 21.32 [01:54:44] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 6.83, 6.81, 6.04 [01:55:00] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 7.74, 7.31, 6.70 [01:55:39] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.49, 1.88, 1.89 [01:55:49] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.00, 7.74, 6.89 [01:56:08] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.43, 7.15, 6.23 [01:56:21] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.14, 3.88, 3.88 [01:56:36] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 32.40, 25.62, 22.39 [01:56:56] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 9.81, 8.11, 7.06 [01:58:06] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 4.77, 6.18, 5.98 [01:59:17] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.10, 5.75, 4.96 [01:59:39] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.18, 1.70, 1.78 [02:00:44] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.31, 6.71, 6.52 [02:00:44] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 5.28, 6.47, 6.19 [02:01:39] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.69, 1.77, 1.80 [02:02:36] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 22.56, 23.89, 22.72 [02:04:37] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 9.53, 8.03, 7.07 [02:05:13] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 4.06, 4.99, 4.90 [02:05:39] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.64, 1.35, 1.64 [02:05:39] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.70, 6.29, 6.61 [02:06:35] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 5.09, 6.96, 6.80 [02:06:35] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 4.69, 7.58, 7.71 [02:08:32] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 4.89, 6.43, 6.63 [02:09:08] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 7.37, 6.30, 5.47 [02:09:32] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [02:10:21] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 5.02, 3.95, 3.84 [02:11:02] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [02:11:06] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 3.63, 5.45, 5.27 [02:12:21] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.60, 3.54, 3.71 [02:12:36] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 14.67, 17.65, 20.14 [02:13:05] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 3.43, 4.77, 5.04 [02:14:21] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 5.70, 4.26, 3.95 [02:15:38] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 8.00, 7.23, 6.52 [02:16:21] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.80, 3.62, 3.75 [02:20:06] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 9.52, 7.66, 7.39 [02:20:21] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 10.25, 7.58, 6.63 [02:20:21] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 5.02, 3.83, 3.78 [02:20:37] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 23.59, 20.97, 20.55 [02:20:45] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 9.61, 7.36, 6.45 [02:21:16] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.69, 6.88, 6.69 [02:22:01] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 7.07, 7.36, 7.31 [02:22:19] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.78, 7.11, 6.57 [02:22:21] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.79, 3.96, 3.85 [02:22:37] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 18.45, 20.20, 20.34 [02:22:43] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 6.15, 6.85, 6.37 [02:23:14] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.91, 6.61, 6.62 [02:24:21] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.06, 4.08, 3.91 [02:25:54] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 8.45, 8.02, 7.59 [02:26:02] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [02:26:15] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 6.25, 6.71, 6.52 [02:28:38] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 5.76, 6.78, 6.54 [02:29:05] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 8.00, 7.16, 6.81 [02:29:12] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 5.72, 6.51, 6.64 [02:29:32] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [02:30:09] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.90, 7.48, 6.95 [02:30:36] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 24.27, 21.89, 20.77 [02:31:48] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 8.19, 5.27, 4.75 [02:32:07] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.55, 6.75, 6.75 [02:32:36] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 19.57, 20.54, 20.38 [02:33:00] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 4.36, 6.10, 6.50 [02:33:46] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 5.07, 5.03, 4.72 [02:34:32] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [02:34:36] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 16.92, 19.09, 19.86 [02:35:33] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 7.42, 7.82, 7.81 [02:37:42] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 10.70, 6.91, 5.47 [02:38:21] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.28, 3.70, 3.92 [02:38:36] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 20.20, 20.88, 20.52 [02:40:36] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 13.68, 18.30, 19.62 [02:41:39] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 1.85, 4.61, 4.90 [02:43:20] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.22, 5.88, 6.78 [02:44:21] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 5.36, 4.26, 4.04 [02:51:39] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.11, 1.65, 1.30 [02:53:39] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.73, 1.28, 1.21 [03:02:21] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.82, 3.29, 3.81 [03:22:21] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.15, 3.59, 3.54 [03:24:21] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.82, 3.50, 3.50 [03:26:21] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.37, 3.79, 3.60 [03:28:21] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.84, 3.75, 3.61 [03:40:21] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.33, 2.99, 3.34 [04:25:44] [02mw-config] 07Universal-Omega closed pull request 03#4286: Remove emergency actions - 13https://git.io/JDwWZ [04:25:45] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JDrXE [04:25:47] [02miraheze/mw-config] 07Universal-Omega 033b1788c - Remove emergency actions (#4286) [04:25:48] [02mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 - 13https://git.io/vbvb3 [04:25:50] [02miraheze/mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 [04:26:51] miraheze/mw-config - Universal-Omega the build passed. [04:36:21] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.74, 3.23, 3.02 [04:38:21] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.25, 3.37, 3.11 [04:40:07] PROBLEM - storytime.jdstroy.cf - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for storytime.jdstroy.cf could not be found [04:41:08] PROBLEM - storytime.jdstroy.cf - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:41:39] !log [@mw11] starting deploy of {'config': True} to all [04:41:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:41:51] !log [@mw11] finished deploy of {'config': True} to all - SUCCESS in 11s [04:41:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:45:12] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.91, 3.78, 3.38 [04:49:06] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.09, 3.25, 3.24 [04:52:57] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.99, 3.43, 3.31 [04:54:33] !log [@test3] starting deploy of {'config': True} to skip [04:54:34] !log [@test3] finished deploy of {'config': True} to skip - SUCCESS in 0s [04:54:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:54:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:54:54] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.92, 3.22, 3.24 [06:34:36] PROBLEM - db12 Current Load on db12 is CRITICAL: CRITICAL - load average: 12.51, 10.99, 6.70 [06:38:32] RECOVERY - db12 Current Load on db12 is OK: OK - load average: 1.26, 5.73, 5.50 [09:37:30] !log [reception@mwtask1] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=clockupwikiwiki --username-prefix wikia:clockup --no-updates --report 1 /home/reception/clockupeuphoria_pages_full.xml (START) [09:37:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [09:37:51] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.64, 3.13, 2.61 [09:38:26] !log [reception@mwtask1] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=clockupwikiwiki --username-prefix wikia:clockup --no-updates --report 1 /home/reception/clockupeuphoria_pages_full.xml (END - exit=0) [09:38:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [09:39:48] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.43, 2.88, 2.59 [09:43:18] [02miraheze/MirahezeMagic] 07Reception123 pushed 031 commit to 03Reception123-patch-1 [+0/-0/±1] 13https://git.io/JDoz2 [09:43:20] [02miraheze/MirahezeMagic] 07Reception123 0387ea2dc - customise import message [09:43:21] [02MirahezeMagic] 07Reception123 created branch 03Reception123-patch-1 - 13https://git.io/fQRGX [09:43:23] [02MirahezeMagic] 07Reception123 opened pull request 03#308: customise import message - 13https://git.io/JDoza [09:43:50] [02miraheze/MirahezeMagic] 07Reception123 pushed 031 commit to 03Reception123-patch-1 [+0/-0/±1] 13https://git.io/JDozr [09:43:51] [02miraheze/MirahezeMagic] 07Reception123 03ca63e50 - Update MirahezeMagicHooks.php [09:43:53] [02MirahezeMagic] 07Reception123 synchronize pull request 03#308: customise import message - 13https://git.io/JDoza [09:46:36] miraheze/MirahezeMagic - Reception123 the build has errored. [09:47:36] [02miraheze/MirahezeMagic] 07Reception123 pushed 031 commit to 03Reception123-patch-1 [+0/-0/±1] 13https://git.io/JDogq [09:47:37] [02miraheze/MirahezeMagic] 07Reception123 03ccf9cff - syntax [09:47:39] [02MirahezeMagic] 07Reception123 synchronize pull request 03#308: customise import message - 13https://git.io/JDoza [09:52:15] [02miraheze/ssl] 07Reception123 pushed 031 commit to 03master [+0/-0/±4] 13https://git.io/JDogA [09:52:16] [02miraheze/ssl] 07Reception123 033e4b536 - renew SSL certs (hopefully last time) [09:54:10] miraheze/MirahezeMagic - Reception123 the build passed. [10:19:09] RECOVERY - wiki.minkyu.kim - reverse DNS on sslhost is OK: SSL OK - wiki.minkyu.kim reverse DNS resolves to cp20.miraheze.org - CNAME OK [10:25:46] RECOVERY - wiki.jeremyboyd.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.jeremyboyd.org' will expire on Thu 17 Mar 2022 08:47:56 GMT +0000. [10:28:36] RECOVERY - sarovia.graalmilitary.com - LetsEncrypt on sslhost is OK: OK - Certificate 'sarovia.graalmilitary.com' will expire on Thu 17 Mar 2022 08:49:22 GMT +0000. [10:29:56] RECOVERY - lakehub.ga - LetsEncrypt on sslhost is OK: OK - Certificate 'lakehub.ga' will expire on Thu 17 Mar 2022 08:50:47 GMT +0000. [10:30:48] RECOVERY - fanon.polandballwiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'fanon.polandballwiki.com' will expire on Thu 17 Mar 2022 08:48:39 GMT +0000. [10:35:09] PROBLEM - wiki.minkyu.kim - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['curitiba.porkbun.com.', 'maceio.porkbun.com.', 'fortaleza.porkbun.com.', 'salvador.porkbun.com.'], 'CNAME': 'mk.miraheze.org.'} [11:08:56] RECOVERY - wiki.minkyu.kim - reverse DNS on sslhost is OK: SSL OK - wiki.minkyu.kim reverse DNS resolves to cp21.miraheze.org - CNAME OK [12:15:00] [02MirahezeMagic] 07Universal-Omega reviewed pull request 03#308 commit - 13https://git.io/JDoN5 [12:18:08] [02MirahezeMagic] 07Reception123 reviewed pull request 03#308 commit - 13https://git.io/JDoAE [12:21:15] PROBLEM - cloud5 Current Load on cloud5 is WARNING: WARNING - load average: 21.13, 19.38, 16.85 [12:23:11] RECOVERY - cloud5 Current Load on cloud5 is OK: OK - load average: 19.10, 19.27, 17.11 [12:34:59] PROBLEM - gluster4 Current Load on gluster4 is CRITICAL: CRITICAL - load average: 9.11, 5.76, 3.89 [12:36:59] PROBLEM - gluster4 Current Load on gluster4 is WARNING: WARNING - load average: 3.55, 5.16, 3.92 [12:38:59] RECOVERY - gluster4 Current Load on gluster4 is OK: OK - load average: 2.82, 4.60, 3.88 [13:23:02] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [13:23:58] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.17, 3.58, 3.07 [13:24:26] PROBLEM - cloud5 Current Load on cloud5 is WARNING: WARNING - load average: 20.58, 18.61, 16.62 [13:25:55] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.97, 3.35, 3.04 [13:26:26] RECOVERY - cloud5 Current Load on cloud5 is OK: OK - load average: 16.95, 18.20, 16.74 [13:28:02] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [13:40:26] PROBLEM - cloud5 Current Load on cloud5 is WARNING: WARNING - load average: 21.41, 19.53, 17.77 [13:42:27] RECOVERY - cloud5 Current Load on cloud5 is OK: OK - load average: 17.23, 18.85, 17.75 [13:45:21] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.47, 3.91, 3.44 [13:50:51] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 6.87, 5.88, 5.04 [13:51:12] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.90, 3.88, 3.60 [13:52:27] PROBLEM - cloud5 Current Load on cloud5 is WARNING: WARNING - load average: 21.88, 20.49, 18.81 [13:52:47] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 4.40, 5.36, 4.95 [13:53:04] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.49, 6.64, 5.59 [13:53:09] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 5.05, 4.34, 3.80 [13:55:04] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.52, 5.81, 5.41 [13:55:48] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 6.98, 6.32, 5.41 [13:56:27] RECOVERY - cloud5 Current Load on cloud5 is OK: OK - load average: 18.58, 20.14, 19.11 [13:57:03] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.84, 3.79, 3.70 [13:57:46] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 5.80, 5.76, 5.29 [14:00:27] PROBLEM - cloud5 Current Load on cloud5 is WARNING: WARNING - load average: 21.86, 21.73, 20.03 [14:04:26] RECOVERY - cloud5 Current Load on cloud5 is OK: OK - load average: 17.07, 19.59, 19.60 [14:04:51] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 1.65, 2.67, 3.27 [14:18:14] PROBLEM - gluster4 Current Load on gluster4 is CRITICAL: CRITICAL - load average: 6.26, 4.98, 4.06 [14:19:24] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 8.28, 6.89, 5.64 [14:19:32] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [14:20:14] RECOVERY - gluster4 Current Load on gluster4 is OK: OK - load average: 2.93, 4.32, 3.94 [14:21:20] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.85, 6.40, 5.61 [14:24:32] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [14:52:45] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 5.03, 4.10, 3.39 [14:56:39] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.07, 3.86, 3.48 [14:58:36] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.31, 4.18, 3.65 [15:02:31] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.57, 3.90, 3.69 [15:10:21] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.48, 3.75, 3.63 [15:12:21] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.56, 3.71, 3.63 [15:14:21] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.56, 3.93, 3.71 [15:16:21] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.51, 3.61, 3.61 [15:20:21] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.54, 3.55, 3.54 [15:22:02] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [15:22:21] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.69, 3.73, 3.62 [15:26:21] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.67, 4.00, 3.73 [15:27:02] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [15:28:21] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.85, 3.79, 3.68 [15:28:54] !log [rhinos@mwtask1] sudo -u www-data /usr/local/bin/foreachwikiindblist /home/rhinos/pageforms.json /srv/mediawiki/w/maintenance/test.php (START) [15:28:56] !log [rhinos@mwtask1] sudo -u www-data /usr/local/bin/foreachwikiindblist /home/rhinos/pageforms.json /srv/mediawiki/w/maintenance/test.php (END - exit=2) [15:28:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:29:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:42:21] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 5.65, 4.12, 3.73 [15:43:47] PROBLEM - cloud5 Current Load on cloud5 is WARNING: WARNING - load average: 22.25, 20.02, 18.69 [15:45:06] PROBLEM - gluster4 Current Load on gluster4 is CRITICAL: CRITICAL - load average: 7.24, 5.50, 4.56 [15:46:21] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.63, 3.52, 3.59 [15:47:04] PROBLEM - gluster4 Current Load on gluster4 is WARNING: WARNING - load average: 4.64, 5.32, 4.62 [15:47:40] RECOVERY - cloud5 Current Load on cloud5 is OK: OK - load average: 17.28, 19.54, 18.90 [15:49:03] RECOVERY - gluster4 Current Load on gluster4 is OK: OK - load average: 3.54, 4.50, 4.39 [15:49:26] @Joseph: read the task. They say it failed. [15:51:45] how does it fail though? [15:51:50] RhinosF1: [15:56:04] ugochimobi: because something went wrong [15:56:24] Oh, was it a kinda backend issue? [15:56:39] RhinosF1: Should I reopen? [15:58:19] Well yes [15:58:21] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.35, 3.04, 3.38 [15:58:24] And definately [15:59:05] they reopened it already though [16:19:49] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.70, 3.52, 3.32 [16:21:02] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [16:21:46] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 5.25, 4.03, 3.52 [16:23:36] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.01, 6.33, 5.21 [16:23:43] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.75, 3.92, 3.55 [16:25:35] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 6.40, 6.30, 5.33 [16:25:40] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.75, 4.18, 3.68 [16:29:34] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.53, 3.98, 3.72 [16:31:02] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [16:33:22] PROBLEM - cloud5 Current Load on cloud5 is WARNING: WARNING - load average: 23.03, 20.98, 19.25 [16:33:26] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 9.19, 7.10, 5.94 [16:34:59] PROBLEM - gluster4 Current Load on gluster4 is WARNING: WARNING - load average: 5.39, 4.00, 3.34 [16:35:19] RECOVERY - cloud5 Current Load on cloud5 is OK: OK - load average: 18.65, 19.93, 19.07 [16:35:25] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.15, 3.70, 3.63 [16:36:59] RECOVERY - gluster4 Current Load on gluster4 is OK: OK - load average: 4.00, 3.94, 3.39 [16:37:22] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.69, 7.47, 6.39 [16:41:19] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 6.00, 6.36, 6.15 [16:42:32] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [16:42:40] RhinosF1, @Joseph/Ugochimobi: talking about thr FlaggedRevs task? That error from ManageWiki is an SQL or other install step failure. I don't know why it would happen but toggleExtension might give more information. [16:43:14] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.97, 3.45, 3.59 [16:45:09] [02dns] 07RhinosF1 opened pull request 03#232: mh.o: remove southparkfan - 13https://git.io/JDK7s [16:45:11] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.28, 3.66, 3.64 [16:47:32] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [16:48:25] CosmicAlpha: yep, haven't had access to try yet [16:48:35] JohnLewis: can you merge #232 [16:49:05] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.98, 3.83, 3.72 [16:49:33] [02dns] 07JohnFLewis closed pull request 03#232: mh.o: remove southparkfan - 13https://git.io/JDK7s [16:49:35] [02miraheze/dns] 07JohnFLewis pushed 032 commits to 03master [+0/-0/±2] 13https://git.io/JDK5W [16:49:36] [02miraheze/dns] 07RhinosF1 0391cc94b - mh.o: remove southparkfan [16:49:38] [02miraheze/dns] 07JohnFLewis 033cb366f - Merge pull request #232 from RhinosF1/patch-24 [16:50:36] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 24.76, 20.62, 16.97 [16:51:02] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 6.58, 4.54, 3.97 [16:51:10] Ty JohnLewis [16:51:32] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 7.18, 6.66, 6.18 [16:52:36] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 23.92, 21.21, 17.62 [16:53:28] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 4.40, 5.75, 5.90 [16:55:50] CosmicAlpha: Oh, I see [16:56:36] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 14.06, 17.92, 17.12 [16:58:51] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.94, 3.88, 3.91 [17:03:13] .task T8449 [17:03:16] https://phabricator.miraheze.org/T8449 - Extension: SVGEdit, authored by Ugochimobi, assigned to None, Priority: Normal, Status: Open [17:03:18] still pending [17:06:39] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.97, 3.70, 3.71 [17:14:27] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.63, 3.64, 3.74 [17:20:21] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.47, 3.62, 3.66 [17:22:18] !log deleted southparkfan from google search console [17:22:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:22:21] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.71, 3.81, 3.73 [17:24:21] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.29, 3.92, 3.78 [17:28:21] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.85, 3.80, 3.76 [17:34:21] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.21, 2.92, 3.38 [18:01:40] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.31, 3.71, 3.38 [18:03:37] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.22, 3.47, 3.34 [18:07:31] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.06, 3.65, 3.41 [18:09:28] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.66, 3.69, 3.46 [18:09:33] [02mediawiki] 07Universal-Omega commented on pull request 03#4676: Bump extensions/PortableInfobox from `b9f59b5` to `4a0bac9` - 13https://git.io/JD6UP [18:09:36] [02mediawiki] 07dependabot[bot] edited pull request 03#4676: Bump extensions/PortableInfobox from `b9f59b5` to `4a0bac9` - 13https://git.io/JDr3a [18:10:32] [02mediawiki] 07dependabot[bot] edited pull request 03#4676: Bump extensions/PortableInfobox from `b9f59b5` to `4a0bac9` - 13https://git.io/JDr3a [18:10:36] [02mediawiki] 07dependabot[bot] opened pull request 03#4677: Bump extensions/PortableInfobox from `b9f59b5` to `25c6c90` - 13https://git.io/JD6U7 [18:10:37] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_37/extensions/PortableInfobox-25c6c90 [+0/-0/±1] 13https://git.io/JD6U5 [18:10:39] [02miraheze/mediawiki] 07dependabot[bot] 03ab1a5e7 - Bump extensions/PortableInfobox from `b9f59b5` to `25c6c90` [18:10:40] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_37/extensions/PortableInfobox-25c6c90 - 13https://git.io/vbL5b [18:10:42] [02mediawiki] 07dependabot[bot] commented on pull request 03#4676: Bump extensions/PortableInfobox from `b9f59b5` to `4a0bac9` - 13https://git.io/JD6Ud [18:10:43] [02mediawiki] 07dependabot[bot] closed pull request 03#4676: Bump extensions/PortableInfobox from `b9f59b5` to `4a0bac9` - 13https://git.io/JDr3a [18:10:45] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_37/extensions/PortableInfobox-4a0bac9 [18:10:46] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_37/extensions/PortableInfobox-4a0bac9 - 13https://git.io/vbL5b [18:11:00] [02mediawiki] 07github-actions[bot] labeled pull request 03#4677: Bump extensions/PortableInfobox from `b9f59b5` to `25c6c90` - 13https://git.io/JD6U7 [18:11:01] [02mediawiki] 07github-actions[bot] labeled pull request 03#4677: Bump extensions/PortableInfobox from `b9f59b5` to `25c6c90` - 13https://git.io/JD6U7 [18:11:03] [02mediawiki] 07github-actions[bot] labeled pull request 03#4677: Bump extensions/PortableInfobox from `b9f59b5` to `25c6c90` - 13https://git.io/JD6U7 [18:11:25] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.57, 4.06, 3.62 [18:12:19] PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found [18:12:26] PROBLEM - cloud5 Current Load on cloud5 is WARNING: WARNING - load average: 21.18, 20.52, 19.05 [18:13:23] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.81, 4.00, 3.65 [18:16:26] RECOVERY - cloud5 Current Load on cloud5 is OK: OK - load average: 18.95, 20.05, 19.24 [18:19:08] RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: SSL OK - lcn.zfc.id.lv reverse DNS resolves to cp21.miraheze.org - CNAME OK [18:19:14] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.36, 3.72, 3.60 [18:20:18] PROBLEM - mw13 Current Load on mw13 is CRITICAL: CRITICAL - load average: 8.30, 7.14, 5.97 [18:21:32] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [18:22:15] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 5.67, 6.61, 5.92 [18:25:44] [02mediawiki] 07Universal-Omega closed pull request 03#4677: Bump extensions/PortableInfobox from `b9f59b5` to `25c6c90` - 13https://git.io/JD6U7 [18:25:45] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_37/extensions/PortableInfobox-25c6c90 - 13https://git.io/vbL5b [18:25:47] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_37/extensions/PortableInfobox-25c6c90 [18:25:48] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_37 [+0/-0/±1] 13https://git.io/JD6IL [18:25:50] [02miraheze/mediawiki] 07dependabot[bot] 037fef6f3 - Bump extensions/PortableInfobox from `b9f59b5` to `25c6c90` (#4677) [18:26:32] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [18:26:54] !log [universalomega@test3] starting deploy of {'world': True, 'l10n': True} to skip [18:26:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:27:02] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.55, 3.57, 3.68 [18:27:25] !log [universalomega@mw11] starting deploy of {'world': True, 'l10n': True} to all [18:27:30] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:30:56] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.53, 3.44, 3.55 [18:32:53] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.35, 3.35, 3.50 [18:35:48] [02ManageWiki] 07Universal-Omega closed pull request 03#320: Create phpunit test for ApiQueryWikiConfig - 13https://git.io/JDrZy [18:35:49] [02miraheze/ManageWiki] 07Universal-Omega pushed 031 commit to 03master [+1/-0/±1] 13https://git.io/JD6LQ [18:35:51] [02miraheze/ManageWiki] 07Universal-Omega 03631b9c0 - Create phpunit test for ApiQueryWikiConfig (#320) [18:35:55] [02ManageWiki] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 - 13https://git.io/vpSns [18:35:56] [02miraheze/ManageWiki] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 [18:36:47] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.77, 3.10, 3.37 [18:40:06] miraheze/ManageWiki - Universal-Omega the build passed. [18:41:50] PROBLEM - cloud5 Current Load on cloud5 is CRITICAL: CRITICAL - load average: 25.84, 21.40, 19.59 [18:42:35] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.01, 3.63, 3.50 [18:43:47] PROBLEM - cloud5 Current Load on cloud5 is WARNING: WARNING - load average: 20.36, 20.66, 19.53 [18:44:33] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.01, 3.52, 3.49 [18:45:43] RECOVERY - cloud5 Current Load on cloud5 is OK: OK - load average: 19.94, 20.36, 19.56 [18:50:01] !log [universalomega@test3] finished deploy of {'world': True, 'l10n': True} to skip - SUCCESS in 1386s [18:50:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:52:32] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [18:54:21] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.10, 3.81, 3.65 [18:54:36] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 21.16, 18.71, 15.73 [18:56:36] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 16.09, 17.65, 15.71 [18:57:32] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [18:58:21] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.33, 3.43, 3.53 [19:00:21] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.95, 3.67, 3.58 [19:01:23] PROBLEM - cloud5 Current Load on cloud5 is CRITICAL: CRITICAL - load average: 32.44, 26.01, 22.22 [19:02:13] !log [universalomega@mw11] finished deploy of {'world': True, 'l10n': True} to all - SUCCESS in 2087s [19:02:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:02:21] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.00, 3.32, 3.46 [19:05:16] PROBLEM - cloud5 Current Load on cloud5 is WARNING: WARNING - load average: 18.69, 22.67, 21.79 [19:08:21] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.13, 3.58, 3.53 [19:10:21] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.37, 3.34, 3.44 [19:12:21] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.27, 3.61, 3.53 [19:14:21] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.12, 3.36, 3.44 [19:16:21] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.89, 3.20, 3.37 [19:20:21] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.85, 3.46, 3.42 [19:21:20] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 8.62, 6.79, 5.41 [19:21:31] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.91, 7.23, 5.64 [19:22:02] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [19:22:21] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.52, 3.04, 3.26 [19:23:20] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 6.20, 6.44, 5.45 [19:23:29] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.67, 6.55, 5.59 [19:27:02] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [19:27:22] PROBLEM - gluster4 Current Load on gluster4 is WARNING: WARNING - load average: 5.43, 5.21, 4.60 [19:29:21] PROBLEM - gluster4 Current Load on gluster4 is CRITICAL: CRITICAL - load average: 6.12, 5.31, 4.69 [19:29:32] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [19:29:49] !log [universalomega@test3] DEPLOY ABORTED: Canary check failed for localhost [19:29:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:30:29] PROBLEM - cloud5 Current Load on cloud5 is CRITICAL: CRITICAL - load average: 24.20, 22.33, 22.12 [19:31:20] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 7.68, 7.64, 6.40 [19:31:24] !log [universalomega@test3] DEPLOY ABORTED: Canary check failed for localhost [19:31:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:31:58] PROBLEM - test3 MediaWiki Rendering on test3 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 431 bytes in 0.142 second response time [19:32:21] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.72, 3.45, 3.30 [19:32:27] PROBLEM - cloud5 Current Load on cloud5 is WARNING: WARNING - load average: 23.99, 22.77, 22.31 [19:33:19] PROBLEM - gluster4 Current Load on gluster4 is WARNING: WARNING - load average: 5.48, 5.79, 5.04 [19:33:20] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 4.69, 6.70, 6.21 [19:35:58] RECOVERY - test3 MediaWiki Rendering on test3 is OK: HTTP OK: HTTP/1.1 200 OK - 20552 bytes in 0.217 second response time [19:39:14] RECOVERY - gluster4 Current Load on gluster4 is OK: OK - load average: 3.96, 4.85, 4.87 [19:39:32] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [19:40:21] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.19, 3.32, 3.33 [19:42:32] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [19:47:32] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [19:50:27] PROBLEM - cloud5 Current Load on cloud5 is CRITICAL: CRITICAL - load average: 24.72, 21.92, 21.43 [19:50:32] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [19:51:04] PROBLEM - gluster4 Current Load on gluster4 is CRITICAL: CRITICAL - load average: 7.40, 5.66, 5.13 [19:51:15] CosmicAlpha: what's test3 hate? [19:51:41] RhinosF1: me testing something. [19:51:51] Ok [19:52:03] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 9.50, 7.12, 6.21 [19:52:06] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.34, 3.68, 3.48 [19:52:27] PROBLEM - cloud5 Current Load on cloud5 is WARNING: WARNING - load average: 21.59, 21.75, 21.43 [19:54:01] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 5.57, 6.47, 6.08 [19:54:03] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.43, 4.13, 3.68 [19:55:32] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [19:57:00] PROBLEM - gluster4 Current Load on gluster4 is WARNING: WARNING - load average: 4.27, 5.52, 5.32 [19:57:57] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.19, 3.80, 3.66 [20:02:59] RECOVERY - gluster4 Current Load on gluster4 is OK: OK - load average: 3.77, 4.67, 5.06 [20:03:48] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 5.00, 3.99, 3.76 [20:04:26] RECOVERY - cloud5 Current Load on cloud5 is OK: OK - load average: 19.49, 19.24, 20.33 [20:07:42] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.90, 3.75, 3.69 [20:13:33] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 1.70, 2.78, 3.31 [20:23:13] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 3.93, 4.20, 3.75 [20:27:07] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.01, 3.72, 3.68 [20:32:38] PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 79, in check_records cname = str(dns_resolver.query(hostname, 'CNAME')[0]) File "/usr/lib/python3/d [20:32:39] ackages/dns/resolver.py", line 1002, in query raise NXDOMAIN(qnames=qnames_to_try, responses=nxdomain_responses)dns.resolver.NXDOMAIN: None of DNS query names exist: lcn.zfc.id.lv., lcn.zfc.id.lv. [20:32:58] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.68, 4.04, 3.81 [20:34:55] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.89, 3.78, 3.74 [20:36:52] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.91, 4.18, 3.88 [20:42:43] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.66, 3.82, 3.85 [20:46:17] RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: SSL OK - lcn.zfc.id.lv reverse DNS resolves to cp20.miraheze.org - CNAME OK [20:46:37] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.29, 3.68, 3.76 [20:48:34] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.43, 3.62, 3.73 [20:52:28] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.33, 3.87, 3.79 [20:55:34] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.25, 3.85, 2.85 [20:56:22] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.78, 3.47, 3.66 [20:57:32] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.49, 4.65, 3.26 [20:59:31] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 4.66, 4.58, 3.40 [21:06:08] [02mediawiki] 07HM100 commented on pull request 03#4672: Bump skins/Evelution from `2cc6564` to `b96948e` - 13https://git.io/JD6uH [21:06:21] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 1.41, 2.56, 3.20 [21:11:31] [02mediawiki] 07RhinosF1 commented on pull request 03#4672: Bump skins/Evelution from `2cc6564` to `b96948e` - 13https://git.io/JD6z0 [21:11:32] [02mediawiki] 07RhinosF1 closed pull request 03#4672: Bump skins/Evelution from `2cc6564` to `b96948e` - 13https://git.io/JDafe [21:11:34] [02mediawiki] 07dependabot[bot] commented on pull request 03#4672: Bump skins/Evelution from `2cc6564` to `b96948e` - 13https://git.io/JD6zE [21:11:35] [02miraheze/mediawiki] 07RhinosF1 deleted branch 03dependabot/submodules/REL1_37/skins/Evelution-b96948e [21:11:37] [02mediawiki] 07RhinosF1 deleted branch 03dependabot/submodules/REL1_37/skins/Evelution-b96948e - 13https://git.io/vbL5b [21:12:21] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.54, 3.29, 3.32 [21:14:21] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.01, 3.04, 3.22 [21:18:21] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.43, 3.37, 3.32 [21:24:21] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.61, 3.35, 3.35 [21:34:58] PROBLEM - test3 MediaWiki Rendering on test3 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 17383 bytes in 0.162 second response time [21:38:53] RECOVERY - test3 MediaWiki Rendering on test3 is OK: HTTP OK: HTTP/1.1 200 OK - 20552 bytes in 0.167 second response time [21:51:52] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.30, 3.46, 3.30 [21:59:40] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.01, 3.23, 3.29 [22:20:02] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [22:21:04] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.80, 7.35, 5.85 [22:22:00] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.77, 4.01, 3.49 [22:22:14] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.86, 6.81, 5.86 [22:22:44] PROBLEM - mw13 Current Load on mw13 is CRITICAL: CRITICAL - load average: 8.26, 7.17, 5.75 [22:23:04] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.60, 8.27, 6.39 [22:24:44] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 5.95, 6.68, 5.75 [22:25:02] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [22:25:04] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 5.91, 7.28, 6.25 [22:25:09] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.03, 7.06, 6.12 [22:26:06] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 6.13, 6.69, 6.04 [22:27:04] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 9.06, 7.68, 6.51 [22:27:52] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.35, 3.85, 3.62 [22:29:04] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.20, 6.59, 6.25 [22:29:05] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 4.13, 6.02, 5.94 [22:35:40] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.93, 3.21, 3.40 [23:00:01] !log [@test3] starting deploy of {'l10nupdate': True} to skip [23:00:02] !log [@mw11] starting deploy of {'l10nupdate': True} to all [23:00:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:00:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:03:32] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [23:05:17] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.92, 6.92, 5.39 [23:06:26] PROBLEM - cloud5 Current Load on cloud5 is WARNING: WARNING - load average: 22.68, 19.29, 16.20 [23:08:06] PROBLEM - test3 APT on test3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [23:08:27] RECOVERY - cloud5 Current Load on cloud5 is OK: OK - load average: 19.66, 19.86, 16.82 [23:08:32] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [23:08:45] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.07, 3.73, 3.35 [23:09:20] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 11.86, 9.66, 6.94 [23:10:42] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.18, 3.62, 3.36 [23:11:20] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 5.75, 7.72, 6.53 [23:12:16] RECOVERY - test3 APT on test3 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [23:13:20] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 4.18, 6.53, 6.24 [23:13:52] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.02, 6.64, 5.36 [23:14:36] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.50, 3.07, 3.20 [23:15:17] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 5.34, 6.53, 6.15 [23:15:53] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.96, 6.84, 5.55 [23:17:51] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.43, 6.83, 5.73 [23:18:45] PROBLEM - test3 APT on test3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [23:19:02] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [23:19:49] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 11.49, 7.91, 6.21 [23:20:10] PROBLEM - test3 Current Load on test3 is CRITICAL: CRITICAL - load average: 4.28, 3.29, 1.92 [23:20:27] PROBLEM - cloud5 Current Load on cloud5 is CRITICAL: CRITICAL - load average: 25.66, 21.62, 18.71 [23:20:46] RECOVERY - test3 APT on test3 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [23:21:16] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 8.18, 7.64, 6.74 [23:21:47] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 5.43, 6.97, 6.08 [23:22:10] RECOVERY - test3 Current Load on test3 is OK: OK - load average: 0.91, 2.33, 1.74 [23:22:27] PROBLEM - cloud5 Current Load on cloud5 is WARNING: WARNING - load average: 21.96, 20.81, 18.71 [23:23:45] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 3.79, 5.87, 5.78 [23:24:02] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [23:24:27] RECOVERY - cloud5 Current Load on cloud5 is OK: OK - load average: 18.35, 19.61, 18.52 [23:27:14] PROBLEM - test3 APT on test3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [23:27:16] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 5.89, 7.87, 7.35 [23:28:11] PROBLEM - test3 Current Load on test3 is CRITICAL: CRITICAL - load average: 7.81, 4.23, 2.56 [23:28:27] PROBLEM - cloud5 Current Load on cloud5 is CRITICAL: CRITICAL - load average: 28.12, 22.77, 19.91 [23:29:12] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.94, 3.64, 3.38 [23:29:15] RECOVERY - test3 APT on test3 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [23:30:27] PROBLEM - cloud5 Current Load on cloud5 is WARNING: WARNING - load average: 23.39, 22.57, 20.17 [23:30:30] !log [universalomega@test3] starting deploy of {'folders': 'w/extensions/CreateWiki'} to skip [23:30:31] !log [universalomega@test3] finished deploy of {'folders': 'w/extensions/CreateWiki'} to skip - SUCCESS in 0s [23:30:32] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [23:30:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:30:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:31:09] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.97, 3.38, 3.32 [23:32:10] RECOVERY - test3 Current Load on test3 is OK: OK - load average: 1.68, 3.21, 2.58 [23:32:15] !log [@mw11] finished deploy of {'l10nupdate': True} to all - SUCCESS in 1931s [23:32:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:33:31] [02CreateWiki] 07Universal-Omega closed pull request 03#267: Replace unused `requestwikiedit` log with `requestreopen` - 13https://git.io/JDEAT [23:33:33] [02miraheze/CreateWiki] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±3] 13https://git.io/JD6Se [23:33:34] [02miraheze/CreateWiki] 07Universal-Omega 03b1f7890 - Replace unused `requestwikiedit` log with `requestreopen` (#267) [23:33:44] [02CreateWiki] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 - 13https://git.io/vpJTL [23:33:45] [02miraheze/CreateWiki] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 [23:34:27] RECOVERY - cloud5 Current Load on cloud5 is OK: OK - load average: 15.18, 18.74, 19.17 [23:34:33] !log [@test3] finished deploy of {'l10nupdate': True} to skip - SUCCESS in 2071s [23:34:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:35:16] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 5.53, 6.06, 6.69 [23:35:32] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [23:38:36] miraheze/CreateWiki - Universal-Omega the build passed. [23:40:36] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_37 [+0/-0/±1] 13https://git.io/JD6Sa [23:40:37] [02miraheze/mediawiki] 07Universal-Omega 03dcdbb27 - Update MirahezeMagic [23:42:53] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_37 [+0/-0/±1] 13https://git.io/JD6Sy [23:42:55] [02miraheze/mediawiki] 07Universal-Omega 03151fac8 - Update Cosmos [23:45:09] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_37 [+0/-0/±1] 13https://git.io/JD69i [23:45:11] [02miraheze/mediawiki] 07Universal-Omega 0381a97c5 - Update CreateWiki [23:45:32] !log [universalomega@test3] starting deploy of {'world': True, 'l10n': True} to skip [23:45:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:45:44] !log [universalomega@mw11] starting deploy of {'world': True, 'l10n': True} to all [23:45:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:53:28] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.37, 3.50, 3.30 [23:55:25] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.98, 3.35, 3.27 [23:55:40] !log [universalomega@test3] finished deploy of {'world': True, 'l10n': True} to skip - SUCCESS in 608s [23:55:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log