[00:00:00] !log [www-data@test151] Began automatic backing up [00:00:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:00:02] !log [www-data@mwtask181] Began automatic backing up [00:00:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:02:30] miraheze/CreateWiki - Universal-Omega the build has errored. [00:03:11] RECOVERY - cp37 Disk Space on cp37 is OK: DISK OK - free space: / 57373MiB (12% inode=99%); [00:05:41] RECOVERY - cp38 Disk Space on cp38 is OK: DISK OK - free space: / 60410MiB (13% inode=99%); [00:10:59] [02CreateWiki] 07Universal-Omega pushed 1 new commit to 03validator 13https://github.com/miraheze/CreateWiki/commit/665a12077c2fb6e8023062fd2c6d5ce3fe5ece6e [00:10:59] 02CreateWiki/03validator 07CosmicAlpha 03665a120 Fix [00:12:56] [02CreateWiki] 07Universal-Omega pushed 1 new commit to 03validator 13https://github.com/miraheze/CreateWiki/commit/9e08480639ce5953d86a84655ffcbbb7f308a64a [00:12:56] 02CreateWiki/03validator 07CosmicAlpha 039e08480 Fix [00:14:40] [02CreateWiki] 07github-actions[bot] pushed 1 new commit to 03validator 13https://github.com/miraheze/CreateWiki/commit/5d39c228d1494e3a88491e4a7e23e505465a1d7a [00:14:40] 02CreateWiki/03validator 07github-actions 035d39c22 CI: lint code to MediaWiki standards… [00:23:47] [02MirahezeMagic] 07OAuthority pushed 1 new commit to 03master 13https://github.com/miraheze/MirahezeMagic/commit/810ad7c9565a1024230e4b629fc2cd700e666daf [00:23:47] 02MirahezeMagic/03master 07OriginalAuthority 03810ad7c Change url for dark mode bug reporting in vector (closes T13285) [00:24:25] PROBLEM - poupedia.com - LetsEncrypt on sslhost is CRITICAL: connect to address poupedia.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:24:30] PROBLEM - kingdomdeath.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address kingdomdeath.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:24:39] PROBLEM - cp36 Nginx Backend for mwtask161 on cp36 is CRITICAL: connect to address localhost and port 8163: Connection refused [00:24:39] PROBLEM - beidipedia.com - LetsEncrypt on sslhost is CRITICAL: connect to address beidipedia.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:24:40] PROBLEM - cp36 Nginx Backend for mw183 on cp36 is CRITICAL: connect to address localhost and port 8127: Connection refused [00:24:40] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 38.46.223.205/cpweb, 2602:294:0:b13::110/cpweb [00:24:42] PROBLEM - cp36 Nginx Backend for mwtask171 on cp36 is CRITICAL: connect to address localhost and port 8161: Connection refused [00:24:48] PROBLEM - cp36 Nginx Backend for mw152 on cp36 is CRITICAL: connect to address localhost and port 8114: Connection refused [00:24:50] PROBLEM - cp36 Nginx Backend for test151 on cp36 is CRITICAL: connect to address localhost and port 8181: Connection refused [00:24:55] PROBLEM - cp36 Nginx Backend for swiftproxy171 on cp36 is CRITICAL: connect to address localhost and port 8207: Connection refused [00:24:59] PROBLEM - dragdown.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address dragdown.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:25:04] PROBLEM - cp36 Nginx Backend for mw163 on cp36 is CRITICAL: connect to address localhost and port 8123: Connection refused [00:25:05] PROBLEM - cp36 HTTP 4xx/5xx ERROR Rate on cp36 is CRITICAL: CRITICAL - NGINX Error Rate is 63% [00:25:06] PROBLEM - archivesofhavnor.org - LetsEncrypt on sslhost is CRITICAL: connect to address archivesofhavnor.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:25:07] PROBLEM - kagaga.jp - LetsEncrypt on sslhost is CRITICAL: connect to address kagaga.jp and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:25:08] PROBLEM - cp36 Nginx Backend for mw182 on cp36 is CRITICAL: connect to address localhost and port 8120: Connection refused [00:25:10] miraheze/CreateWiki - Universal-Omega the build passed. [00:25:21] PROBLEM - cp36 Nginx Backend for mw153 on cp36 is CRITICAL: connect to address localhost and port 8121: Connection refused [00:25:24] PROBLEM - cp36 Nginx Backend for reports171 on cp36 is CRITICAL: connect to address localhost and port 8205: Connection refused [00:25:26] PROBLEM - cp36 Nginx Backend for mw173 on cp36 is CRITICAL: connect to address localhost and port 8125: Connection refused [00:25:27] PROBLEM - cp36 Nginx Backend for mw201 on cp36 is CRITICAL: connect to address localhost and port 8132: Connection refused [00:25:27] PROBLEM - cp36 Nginx Backend for mw172 on cp36 is CRITICAL: connect to address localhost and port 8118: Connection refused [00:25:27] PROBLEM - ssbuniverses.com - LetsEncrypt on sslhost is CRITICAL: connect to address ssbuniverses.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:25:30] PROBLEM - monarchists.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address monarchists.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:25:31] PROBLEM - rippaverse.wikitide.org - LetsEncrypt on sslhost is CRITICAL: connect to address rippaverse.wikitide.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:25:32] PROBLEM - cp36 Nginx Backend for mw202 on cp36 is CRITICAL: connect to address localhost and port 8133: Connection refused [00:25:34] PROBLEM - cp36 Nginx Backend for matomo151 on cp36 is CRITICAL: connect to address localhost and port 8203: Connection refused [00:25:36] PROBLEM - infectowiki.com - LetsEncrypt on sslhost is CRITICAL: connect to address infectowiki.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:25:38] PROBLEM - cp36 Nginx Backend for mw162 on cp36 is CRITICAL: connect to address localhost and port 8116: Connection refused [00:25:39] PROBLEM - cp36 HTTPS on cp36 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 7 - Failed to connect to cp36.wikitide.net port 443 after 0 ms: Couldn't connect to server [00:25:42] PROBLEM - cp36 Nginx Backend for mw203 on cp36 is CRITICAL: connect to address localhost and port 8134: Connection refused [00:25:45] PROBLEM - cp36 Nginx Backend for mw171 on cp36 is CRITICAL: connect to address localhost and port 8117: Connection refused [00:25:48] PROBLEM - cp36 Nginx Backend for mw192 on cp36 is CRITICAL: connect to address localhost and port 8130: Connection refused [00:25:48] PROBLEM - cp36 Nginx Backend for mwtask151 on cp36 is CRITICAL: connect to address localhost and port 8162: Connection refused [00:25:51] PROBLEM - cp36 Nginx Backend for mw181 on cp36 is CRITICAL: connect to address localhost and port 8119: Connection refused [00:25:51] PROBLEM - cp36 Nginx Backend for mon181 on cp36 is CRITICAL: connect to address localhost and port 8201: Connection refused [00:25:54] PROBLEM - rosettacode.org - LetsEncrypt on sslhost is CRITICAL: connect to address rosettacode.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:25:56] PROBLEM - cp36 Nginx Backend for mw193 on cp36 is CRITICAL: connect to address localhost and port 8131: Connection refused [00:25:57] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 38.46.223.205/cpweb, 2602:294:0:b13::110/cpweb [00:25:58] PROBLEM - cp36 Nginx Backend for mw161 on cp36 is CRITICAL: connect to address localhost and port 8115: Connection refused [00:26:00] PROBLEM - cp36 Nginx Backend for puppet181 on cp36 is CRITICAL: connect to address localhost and port 8204: Connection refused [00:26:02] PROBLEM - mockgovernments.com - LetsEncrypt on sslhost is CRITICAL: connect to address mockgovernments.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:26:03] PROBLEM - wikigenius.org - LetsEncrypt on sslhost is CRITICAL: connect to address wikigenius.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:26:04] PROBLEM - cp36 Nginx Backend for phorge171 on cp36 is CRITICAL: connect to address localhost and port 8202: Connection refused [00:26:04] PROBLEM - *.wikitide.org - LetsEncrypt on sslhost is CRITICAL: connect to address wikitide.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:26:07] PROBLEM - cp36 Nginx Backend for swiftproxy161 on cp36 is CRITICAL: connect to address localhost and port 8206: Connection refused [00:26:13] PROBLEM - n64brew.dev - LetsEncrypt on sslhost is CRITICAL: connect to address n64brew.dev and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:26:13] PROBLEM - cp36 Nginx Backend for mw151 on cp36 is CRITICAL: connect to address localhost and port 8113: Connection refused [00:26:16] PROBLEM - dccomicswiki.com - LetsEncrypt on sslhost is CRITICAL: connect to address dccomicswiki.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:26:16] PROBLEM - cp36 Nginx Backend for mwtask181 on cp36 is CRITICAL: connect to address localhost and port 8160: Connection refused [00:26:16] PROBLEM - androidwiki.info - LetsEncrypt on sslhost is CRITICAL: connect to address androidwiki.info and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:26:20] PROBLEM - keitaiwiki.com - LetsEncrypt on sslhost is CRITICAL: connect to address keitaiwiki.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:26:22] PROBLEM - cp36 Varnish Backends on cp36 is CRITICAL: 19 backends are down. mw151 mw152 mw161 mw162 mw171 mw172 mw181 mw182 mw153 mw163 mw173 mw183 mw191 mw192 mw193 mw201 mw202 mw203 mediawiki [00:26:24] PROBLEM - cp36 Nginx Backend for mw191 on cp36 is CRITICAL: connect to address localhost and port 8129: Connection refused [00:26:25] PROBLEM - darkwaterswiki.com - LetsEncrypt on sslhost is CRITICAL: connect to address darkwaterswiki.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:26:27] PROBLEM - inourownwords.online - LetsEncrypt on sslhost is CRITICAL: connect to address inourownwords.online and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:26:42] PROBLEM - wikappedia.cc - LetsEncrypt on sslhost is CRITICAL: connect to address wikappedia.cc and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:26:42] PROBLEM - sptwiki.org - LetsEncrypt on sslhost is CRITICAL: connect to address sptwiki.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:26:47] PROBLEM - kodiak.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address kodiak.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:27:04] PROBLEM - podpedia.org - LetsEncrypt on sslhost is CRITICAL: connect to address podpedia.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:27:25] PROBLEM - decimatedrive.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address decimatedrive.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:27:27] PROBLEM - allthetropes.org - LetsEncrypt on sslhost is CRITICAL: connect to address allthetropes.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:27:34] PROBLEM - acuralegendwiki.org - LetsEncrypt on sslhost is CRITICAL: connect to address acuralegendwiki.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:27:51] PROBLEM - raidrush.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address raidrush.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:27:51] PROBLEM - worldtriggerwiki.com - LetsEncrypt on sslhost is CRITICAL: connect to address worldtriggerwiki.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:27:57] !log [oa@mwtask181] starting deploy of {'world': True, 'l10n': True, 'versions': '1.43', 'upgrade_extensions': 'MirahezeMagic'} to all [00:27:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:28:00] PROBLEM - worldsanskrit.net - LetsEncrypt on sslhost is CRITICAL: connect to address worldsanskrit.net and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:28:10] PROBLEM - yoshipedia.com - LetsEncrypt on sslhost is CRITICAL: connect to address yoshipedia.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:28:11] PROBLEM - mwcosmos.com - LetsEncrypt on sslhost is CRITICAL: connect to address mwcosmos.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:28:15] PROBLEM - wikimoma.art - LetsEncrypt on sslhost is CRITICAL: connect to address wikimoma.art and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:28:17] PROBLEM - grayzonewarfare.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address grayzonewarfare.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:28:19] PROBLEM - puzzles.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address puzzles.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:28:37] PROBLEM - richterian.com - LetsEncrypt on sslhost is CRITICAL: connect to address richterian.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:28:39] PROBLEM - hastursnotebook.org - LetsEncrypt on sslhost is CRITICAL: connect to address hastursnotebook.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:28:41] PROBLEM - farthestfrontier.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address farthestfrontier.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:29:22] PROBLEM - edapedia.net - LetsEncrypt on sslhost is CRITICAL: connect to address edapedia.net and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:29:34] PROBLEM - looneypyramids.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address looneypyramids.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:29:37] PROBLEM - hoseki.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address hoseki.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:29:46] PROBLEM - geeu.org - LetsEncrypt on sslhost is CRITICAL: connect to address geeu.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:29:53] PROBLEM - neurologiahpm.com - LetsEncrypt on sslhost is CRITICAL: connect to address neurologiahpm.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:30:01] PROBLEM - binrayarchives.com - LetsEncrypt on sslhost is CRITICAL: connect to address binrayarchives.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:30:04] PROBLEM - osrwiki.org - LetsEncrypt on sslhost is CRITICAL: connect to address osrwiki.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:30:10] PROBLEM - caverncrusher.com - LetsEncrypt on sslhost is CRITICAL: connect to address caverncrusher.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:30:21] PROBLEM - rarewarewiki.com - LetsEncrypt on sslhost is CRITICAL: connect to address rarewarewiki.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:30:44] PROBLEM - cnidaria.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address cnidaria.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:31:05] PROBLEM - gielinorgames.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address gielinorgames.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:31:06] PROBLEM - blackmagic.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address blackmagic.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:31:17] PROBLEM - az-wiki.com - LetsEncrypt on sslhost is CRITICAL: connect to address az-wiki.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:31:31] PROBLEM - rainverse.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address rainverse.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:31:41] PROBLEM - trollpasta.com - LetsEncrypt on sslhost is CRITICAL: connect to address trollpasta.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:32:08] PROBLEM - holocron.net - LetsEncrypt on sslhost is CRITICAL: connect to address holocron.net and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:32:28] PROBLEM - burnout.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address burnout.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:32:37] PROBLEM - mariopedia.org - LetsEncrypt on sslhost is CRITICAL: connect to address mariopedia.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:32:46] PROBLEM - lostmediawiki.ru - LetsEncrypt on sslhost is CRITICAL: connect to address lostmediawiki.ru and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:32:49] PROBLEM - lads.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address lads.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:32:57] PROBLEM - wizardia.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address wizardia.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:33:11] PROBLEM - transgender.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address transgender.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:33:14] PROBLEM - metroidpedia.com - LetsEncrypt on sslhost is CRITICAL: connect to address metroidpedia.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:33:21] PROBLEM - fairytail.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address fairytail.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:33:42] PROBLEM - opendatascot.org - LetsEncrypt on sslhost is CRITICAL: connect to address opendatascot.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:33:49] PROBLEM - squareenixwiki.com - LetsEncrypt on sslhost is CRITICAL: connect to address squareenixwiki.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:33:57] !log increase cp36 ram to 10gib [00:34:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:34:19] PROBLEM - voecwiki.com - LetsEncrypt on sslhost is CRITICAL: connect to address voecwiki.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [00:34:22] RECOVERY - cp36 Varnish Backends on cp36 is OK: All 31 backends are healthy [00:34:39] RECOVERY - cp36 Nginx Backend for mwtask161 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8163 [00:34:40] RECOVERY - cp36 Nginx Backend for mw183 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8127 [00:34:43] RECOVERY - cp36 Nginx Backend for mwtask171 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8161 [00:34:48] RECOVERY - cp36 Nginx Backend for mw152 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8114 [00:34:50] RECOVERY - cp36 Nginx Backend for test151 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8181 [00:34:55] RECOVERY - cp36 Nginx Backend for swiftproxy171 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8207 [00:35:04] RECOVERY - cp36 Nginx Backend for mw163 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8123 [00:35:05] RECOVERY - cp36 HTTP 4xx/5xx ERROR Rate on cp36 is OK: OK - NGINX Error Rate is 7% [00:35:08] RECOVERY - cp36 Nginx Backend for mw182 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8120 [00:35:21] RECOVERY - cp36 Nginx Backend for mw153 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8121 [00:35:24] RECOVERY - cp36 Nginx Backend for reports171 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8205 [00:35:26] RECOVERY - cp36 Nginx Backend for mw201 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8132 [00:35:27] RECOVERY - cp36 Nginx Backend for mw172 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8118 [00:35:27] RECOVERY - cp36 Nginx Backend for mw173 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8125 [00:35:32] RECOVERY - cp36 Nginx Backend for mw202 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8133 [00:35:34] RECOVERY - cp36 Nginx Backend for matomo151 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8203 [00:35:38] RECOVERY - cp36 Nginx Backend for mw162 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8116 [00:35:39] RECOVERY - cp36 HTTPS on cp36 is OK: HTTP OK: HTTP/2 404 - Status line output matched "HTTP/2 404" - 4073 bytes in 0.061 second response time [00:35:42] miraheze/MirahezeMagic - OAuthority the build passed. [00:35:42] RECOVERY - cp36 Nginx Backend for mw203 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8134 [00:35:45] RECOVERY - cp36 Nginx Backend for mw171 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8117 [00:35:47] !log add 1gb of swap to cp36 [00:35:48] RECOVERY - cp36 Nginx Backend for mw192 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8130 [00:35:48] RECOVERY - cp36 Nginx Backend for mwtask151 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8162 [00:35:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:35:57] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [00:36:11] RECOVERY - cp36 Nginx Backend for swiftproxy161 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8206 [00:36:12] RECOVERY - cp36 Nginx Backend for mw193 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8131 [00:36:13] RECOVERY - cp36 Nginx Backend for mw181 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8119 [00:36:13] RECOVERY - cp36 Nginx Backend for mon181 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8201 [00:36:13] RECOVERY - cp36 Nginx Backend for mw151 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8113 [00:36:14] RECOVERY - cp36 Nginx Backend for phorge171 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8202 [00:36:15] RECOVERY - cp36 Nginx Backend for mw161 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8115 [00:36:16] RECOVERY - cp36 Nginx Backend for mwtask181 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8160 [00:36:16] RECOVERY - cp36 Nginx Backend for puppet181 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8204 [00:36:24] RECOVERY - cp36 Nginx Backend for mw191 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8129 [00:36:40] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [00:37:24] !log [oa@mwtask181] finished deploy of {'world': True, 'l10n': True, 'versions': '1.43', 'upgrade_extensions': 'MirahezeMagic'} to all - SUCCESS in 566s [00:37:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:38:07] RECOVERY - voecwiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'voecwiki.com' will expire on Tue 25 Mar 2025 12:53:32 AM GMT +0000. [00:38:10] RECOVERY - squareenixwiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'squareenixwiki.com' will expire on Thu 22 May 2025 11:46:44 PM GMT +0000. [00:38:12] RECOVERY - opendatascot.org - LetsEncrypt on sslhost is OK: OK - Certificate 'opendatascot.org' will expire on Mon 26 May 2025 08:12:32 AM GMT +0000. [00:38:14] RECOVERY - fairytail.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'fairytail.wiki' will expire on Thu 20 Mar 2025 09:03:07 PM GMT +0000. [00:38:16] RECOVERY - lads.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'lads.wiki' will expire on Tue 20 May 2025 04:01:34 PM GMT +0000. [00:38:18] RECOVERY - wizardia.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'wizardia.wiki' will expire on Wed 19 Mar 2025 02:21:35 AM GMT +0000. [00:38:20] RECOVERY - transgender.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'transgender.wiki' will expire on Sun 30 Mar 2025 04:29:43 PM GMT +0000. [00:38:21] RECOVERY - mariopedia.org - LetsEncrypt on sslhost is OK: OK - Certificate 'mariopedia.org' will expire on Thu 22 May 2025 09:56:46 PM GMT +0000. [00:38:23] RECOVERY - burnout.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'burnout.wiki' will expire on Tue 18 Mar 2025 03:16:41 PM GMT +0000. [00:38:25] RECOVERY - holocron.net - LetsEncrypt on sslhost is OK: OK - Certificate 'holocron.net' will expire on Sat 10 May 2025 02:45:38 AM GMT +0000. [00:38:27] RECOVERY - cnidaria.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'cnidaria.wiki' will expire on Sun 04 May 2025 05:19:11 PM GMT +0000. [00:38:29] RECOVERY - gielinorgames.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'gielinorgames.wiki' will expire on Fri 23 May 2025 10:01:26 PM GMT +0000. [00:38:31] RECOVERY - rainverse.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'rainverse.wiki' will expire on Mon 21 Apr 2025 12:52:28 AM GMT +0000. [00:38:33] RECOVERY - trollpasta.com - LetsEncrypt on sslhost is OK: OK - Certificate 'trollpasta.com' will expire on Tue 18 Mar 2025 03:42:24 PM GMT +0000. [00:38:37] RECOVERY - lostmediawiki.ru - LetsEncrypt on sslhost is OK: OK - Certificate 'lostmediawiki.ru' will expire on Mon 07 Apr 2025 01:25:09 PM GMT +0000. [00:38:39] RECOVERY - metroidpedia.com - LetsEncrypt on sslhost is OK: OK - Certificate 'metroidpedia.com' will expire on Fri 23 May 2025 03:01:56 PM GMT +0000. [00:38:42] RECOVERY - az-wiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'az-wiki.com' will expire on Sun 16 Mar 2025 04:04:50 PM GMT +0000. [00:38:44] RECOVERY - blackmagic.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'blackmagic.wiki' will expire on Fri 09 May 2025 06:36:06 PM GMT +0000. [00:38:46] RECOVERY - rarewarewiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'rarewarewiki.com' will expire on Sat 24 May 2025 03:08:20 PM GMT +0000. [00:38:48] RECOVERY - caverncrusher.com - LetsEncrypt on sslhost is OK: OK - Certificate 'caverncrusher.com' will expire on Thu 29 May 2025 01:34:47 PM GMT +0000. [00:38:50] RECOVERY - osrwiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'osrwiki.org' will expire on Mon 07 Apr 2025 07:37:52 PM GMT +0000. [00:38:51] RECOVERY - binrayarchives.com - LetsEncrypt on sslhost is OK: OK - Certificate 'binrayarchives.com' will expire on Fri 25 Apr 2025 09:14:14 PM GMT +0000. [00:38:53] RECOVERY - geeu.org - LetsEncrypt on sslhost is OK: OK - Certificate 'geeu.org' will expire on Sun 27 Apr 2025 02:05:19 PM GMT +0000. [00:38:56] RECOVERY - neurologiahpm.com - LetsEncrypt on sslhost is OK: OK - Certificate 'neurologiahpm.com' will expire on Sun 11 May 2025 11:05:08 AM GMT +0000. [00:38:59] RECOVERY - looneypyramids.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'looneypyramids.wiki' will expire on Fri 23 May 2025 07:45:51 AM GMT +0000. [00:39:01] RECOVERY - hoseki.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'hoseki.wiki' will expire on Thu 22 May 2025 12:25:41 PM GMT +0000. [00:39:04] RECOVERY - mwcosmos.com - LetsEncrypt on sslhost is OK: OK - Certificate 'mwcosmos.com' will expire on Mon 19 May 2025 09:42:04 PM GMT +0000. [00:39:06] RECOVERY - wikimoma.art - LetsEncrypt on sslhost is OK: OK - Certificate 'wikimoma.art' will expire on Wed 28 May 2025 01:48:54 PM GMT +0000. [00:39:08] RECOVERY - grayzonewarfare.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'grayzonewarfare.wiki' will expire on Mon 07 Apr 2025 07:26:26 PM GMT +0000. [00:39:10] RECOVERY - puzzles.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'puzzles.wiki' will expire on Tue 08 Apr 2025 10:21:57 PM GMT +0000. [00:39:12] RECOVERY - richterian.com - LetsEncrypt on sslhost is OK: OK - Certificate 'richterian.com' will expire on Wed 23 Apr 2025 08:25:04 PM GMT +0000. [00:39:14] RECOVERY - hastursnotebook.org - LetsEncrypt on sslhost is OK: OK - Certificate 'hastursnotebook.org' will expire on Sat 24 May 2025 07:23:18 AM GMT +0000. [00:39:16] RECOVERY - allthetropes.org - LetsEncrypt on sslhost is OK: OK - Certificate 'allthetropes.org' will expire on Tue 20 May 2025 08:00:38 PM GMT +0000. [00:39:17] RECOVERY - acuralegendwiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'acuralegendwiki.org' will expire on Fri 23 May 2025 01:54:36 AM GMT +0000. [00:39:19] RECOVERY - raidrush.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'raidrush.wiki' will expire on Wed 19 Mar 2025 04:19:52 PM GMT +0000. [00:39:20] PROBLEM - worldtriggerwiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'worldtriggerwiki.com' expires in 14 day(s) (Sun 16 Mar 2025 12:03:12 AM GMT +0000). [00:39:22] RECOVERY - worldsanskrit.net - LetsEncrypt on sslhost is OK: OK - Certificate 'worldsanskrit.net' will expire on Wed 28 May 2025 01:54:54 PM GMT +0000. [00:39:24] RECOVERY - yoshipedia.com - LetsEncrypt on sslhost is OK: OK - Certificate 'yoshipedia.com' will expire on Thu 17 Apr 2025 06:15:41 PM GMT +0000. [00:39:27] RECOVERY - darkwaterswiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'darkwaterswiki.com' will expire on Mon 24 Mar 2025 03:09:45 PM GMT +0000. [00:39:28] RECOVERY - inourownwords.online - LetsEncrypt on sslhost is OK: OK - Certificate 'inourownwords.online' will expire on Thu 22 May 2025 08:35:49 AM GMT +0000. [00:39:30] RECOVERY - sptwiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'sptwiki.org' will expire on Fri 11 Apr 2025 05:33:12 PM GMT +0000. [00:39:32] RECOVERY - wikappedia.cc - LetsEncrypt on sslhost is OK: OK - Certificate 'wikappedia.cc' will expire on Sun 23 Mar 2025 01:58:38 PM GMT +0000. [00:39:32] [02ssl] 07WikiTideSSLBot pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/c9e1f925510befd5e59009c63bd1d0baebe40ec4 [00:39:32] 02ssl/03master 07WikiTideSSLBot 03c9e1f92 Bot: Update SSL cert for worldtriggerwiki.com [00:39:34] RECOVERY - kodiak.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'kodiak.wiki' will expire on Wed 09 Apr 2025 02:48:10 AM GMT +0000. [00:39:36] RECOVERY - podpedia.org - LetsEncrypt on sslhost is OK: OK - Certificate 'podpedia.org' will expire on Wed 21 May 2025 06:59:07 PM GMT +0000. [00:39:38] RECOVERY - decimatedrive.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'decimatedrive.wiki' will expire on Fri 21 Mar 2025 09:50:10 PM GMT +0000. [00:39:39] RECOVERY - androidwiki.info - LetsEncrypt on sslhost is OK: OK - Certificate 'androidwiki.info' will expire on Sat 12 Apr 2025 03:18:53 PM GMT +0000. [00:39:44] RECOVERY - n64brew.dev - LetsEncrypt on sslhost is OK: OK - Certificate 'n64brew.dev' will expire on Wed 21 May 2025 05:40:16 PM GMT +0000. [00:39:45] RECOVERY - dccomicswiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'dccomicswiki.com' will expire on Sun 18 May 2025 12:10:49 PM GMT +0000. [00:39:46] RECOVERY - rippaverse.wikitide.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wikitide.org' will expire on Sat 12 Apr 2025 07:36:30 PM GMT +0000. [00:39:48] RECOVERY - infectowiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'infectowiki.com' will expire on Wed 21 May 2025 02:17:52 PM GMT +0000. [00:39:50] RECOVERY - keitaiwiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'keitaiwiki.com' will expire on Wed 28 May 2025 01:53:26 PM GMT +0000. [00:39:52] RECOVERY - farthestfrontier.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'farthestfrontier.wiki' will expire on Mon 28 Apr 2025 07:20:21 PM GMT +0000. [00:39:54] RECOVERY - edapedia.net - LetsEncrypt on sslhost is OK: OK - Certificate 'edapedia.net' will expire on Mon 07 Apr 2025 01:52:00 AM GMT +0000. [00:40:00] RECOVERY - wikigenius.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wikigenius.org' will expire on Mon 07 Apr 2025 04:35:18 PM GMT +0000. [00:40:02] RECOVERY - *.wikitide.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wikitide.org' will expire on Sat 12 Apr 2025 07:36:30 PM GMT +0000. [00:40:04] RECOVERY - archivesofhavnor.org - LetsEncrypt on sslhost is OK: OK - Certificate 'archivesofhavnor.org' will expire on Thu 24 Apr 2025 01:55:22 PM GMT +0000. [00:40:06] RECOVERY - beidipedia.com - LetsEncrypt on sslhost is OK: OK - Certificate 'beidipedia.com' will expire on Mon 07 Apr 2025 02:07:29 PM GMT +0000. [00:40:09] RECOVERY - kingdomdeath.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'kingdomdeath.wiki' will expire on Wed 28 May 2025 07:58:21 AM GMT +0000. [00:40:11] RECOVERY - poupedia.com - LetsEncrypt on sslhost is OK: OK - Certificate 'poupedia.com' will expire on Mon 28 Apr 2025 01:12:42 PM GMT +0000. [00:40:13] RECOVERY - kagaga.jp - LetsEncrypt on sslhost is OK: OK - Certificate 'kagaga.jp' will expire on Thu 20 Mar 2025 04:51:28 PM GMT +0000. [00:40:15] RECOVERY - monarchists.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'monarchists.wiki' will expire on Wed 21 May 2025 08:10:32 AM GMT +0000. [00:40:16] RECOVERY - ssbuniverses.com - LetsEncrypt on sslhost is OK: OK - Certificate 'ssbuniverses.com' will expire on Sat 24 May 2025 05:44:10 PM GMT +0000. [00:40:20] RECOVERY - rosettacode.org - LetsEncrypt on sslhost is OK: OK - Certificate 'rosettacode.org' will expire on Sun 06 Apr 2025 01:20:04 AM GMT +0000. [00:40:24] RECOVERY - mockgovernments.com - LetsEncrypt on sslhost is OK: OK - Certificate 'mockgovernments.com' will expire on Tue 01 Apr 2025 05:55:36 PM GMT +0000. [00:40:29] RECOVERY - dragdown.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'dragdown.wiki' will expire on Mon 17 Mar 2025 03:23:49 PM GMT +0000. [00:45:12] [02mw-config] 07OAuthority merged 07BlankEclair's pull request #5832: T13250: Use Wikimedia Commons for JsonConfig on openfrontwiki (07miraheze:03master...07BlankEclair:03T13250) 13https://github.com/miraheze/mw-config/pull/5832 [00:45:12] [02mw-config] 07OAuthority pushed 1 new commit to 03master 13https://github.com/miraheze/mw-config/commit/346ad8fdab75881e16ee9ed9cdabce86493aa758 [00:45:12] 02mw-config/03master 07Claire 03346ad8f T13250: Use Wikimedia Commons for JsonConfig on openfrontwiki (#5832)… [00:46:09] miraheze/mw-config - OAuthority the build passed. [00:46:45] !log [oa@mwtask181] starting deploy of {'world': True, 'versions': '1.43', 'upgrade_extensions': 'PortableInfobox'} to all [00:46:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:47:16] PROBLEM - mw153 Puppet on mw153 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 2 minutes ago with 2 failures. Failed resources (up to 3 shown): Exec[git_pull_femiwiki-deploy-1.43],Exec[git_pull_3d2png] [00:48:33] !log [oa@mwtask181] finished deploy of {'world': True, 'versions': '1.43', 'upgrade_extensions': 'PortableInfobox'} to all - SUCCESS in 108s [00:48:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:00:12] RECOVERY - mattermost1 Backups Mattermost DB on mattermost1 is OK: FILE_AGE OK: /var/log/mattermost-backup/mattermost-db/mattermost-db.log is 0 seconds old and 199 bytes [01:01:13] RECOVERY - mattermost1 Backups Mattermost Data on mattermost1 is OK: FILE_AGE OK: /var/log/mattermost-backup/mattermost-data/mattermost-data.log is 34 seconds old and 308 bytes [01:01:27] RECOVERY - puppet181 Backups PuppetDB on puppet181 is OK: FILE_AGE OK: /var/log/puppetdb-backup/puppetdb-backup/puppetdb.log is 64 seconds old and 195 bytes [01:15:15] RECOVERY - mw153 Puppet on mw153 is OK: OK: Puppet is currently enabled, last run 6 seconds ago with 0 failures [01:26:21] [02CreateWiki] 07BlankEclair left a file comment in pull request #683 035d39c22: ```suggestion […] 13https://github.com/miraheze/CreateWiki/pull/683#discussion_r1976201900 [01:26:23] [02CreateWiki] 07BlankEclair left a file comment in pull request #683 035d39c22: ```suggestion […] 13https://github.com/miraheze/CreateWiki/pull/683#discussion_r1976202403 [01:28:43] [02CreateWiki] 07Universal-Omega left a file comment in pull request #683 035d39c22: Doesn't this mean it doesn't work if for some reason a subdomain is example.miraheze.org.miraheze.org (which can happen if they copy and paste to much or something), it would only trim the last part and then try to create example.miraheze.orgwiki 13https://github.com/miraheze/CreateWiki/pull/683#discussion_r1976203779 [01:36:34] PROBLEM - mwtask161 Current Load on mwtask161 is CRITICAL: LOAD CRITICAL - total load average: 26.94, 15.36, 6.94 [01:38:28] RECOVERY - worldtriggerwiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'worldtriggerwiki.com' will expire on Thu 29 May 2025 11:40:56 PM GMT +0000. [01:38:34] RECOVERY - mwtask161 Current Load on mwtask161 is OK: LOAD OK - total load average: 12.54, 15.44, 8.11 [01:41:45] PROBLEM - mwtask151 Current Load on mwtask151 is CRITICAL: LOAD CRITICAL - total load average: 25.07, 16.02, 8.27 [01:47:45] RECOVERY - mwtask151 Current Load on mwtask151 is OK: LOAD OK - total load average: 8.34, 17.17, 11.85 [04:27:16] [02ssl] 07WikiTideSSLBot pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/34d9fec7d93f26630f7e84dd99b12318b18af7c4 [04:27:16] 02ssl/03master 07WikiTideSSLBot 0334d9fec Bot: Update SSL cert for christipedia.nl [04:27:40] PROBLEM - mwtask171 Current Load on mwtask171 is CRITICAL: LOAD CRITICAL - total load average: 25.17, 15.59, 7.04 [04:35:40] RECOVERY - mwtask171 Current Load on mwtask171 is OK: LOAD OK - total load average: 4.36, 15.61, 11.56 [04:36:04] [02ssl] 07MacFan4000 pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/54bbdaa3b59d0ae1a67b0203a8c5685146bfe45c [04:36:04] 02ssl/03master 07MacFan4000 0354bbdaa add wiki.primearca.de and dunepedia.net [05:13:49] RECOVERY - christipedia.nl - LetsEncrypt on sslhost is OK: OK - Certificate 'christipedia.nl' will expire on Fri 30 May 2025 03:28:39 AM GMT +0000. [05:44:58] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 25.19, 15.21, 9.40 [05:58:58] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 17.75, 21.44, 17.96 [06:00:58] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 25.58, 23.48, 19.16 [06:08:58] RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 8.10, 18.83, 19.57 [06:14:58] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.30, 20.92, 19.83 [06:16:58] RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 10.63, 16.36, 18.26 [08:02:58] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 27.78, 17.63, 11.34 [08:04:58] RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 17.23, 17.32, 12.03 [10:20:58] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 25.66, 17.99, 10.33 [10:47:22] [02CreateWiki] 07Universal-Omega pushed 1 new commit to 03validator 13https://github.com/miraheze/CreateWiki/commit/09882493cfecae2c15d7b78cebff533c2848ca31 [10:47:22] 02CreateWiki/03validator 07CosmicAlpha 030988249 Pass exists [10:50:34] miraheze/CreateWiki - Universal-Omega the build has errored. [10:50:58] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 11.78, 22.55, 23.05 [10:51:15] [02CreateWiki] 07Universal-Omega pushed 1 new commit to 03validator 13https://github.com/miraheze/CreateWiki/commit/f278546a5cd3653023a8ab4a4b5f58862dd8f58f [10:51:16] 02CreateWiki/03validator 07CosmicAlpha 03f278546 Update [10:52:20] [Grafana] FIRING: The mediawiki job queue has more than 2000 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [10:54:19] miraheze/CreateWiki - Universal-Omega the build has errored. [10:54:45] [02CreateWiki] 07Universal-Omega pushed 1 new commit to 03validator 13https://github.com/miraheze/CreateWiki/commit/b2d79521c906eb08097f9aefc145e296b91bf025 [10:54:45] 02CreateWiki/03validator 07CosmicAlpha 03b2d7952 Update [10:54:58] RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 2.02, 11.30, 18.30 [10:56:40] [02CreateWiki] 07Universal-Omega pushed 1 new commit to 03validator 13https://github.com/miraheze/CreateWiki/commit/b2b5d85341ab76216dc6f272bf54af7bab83ef5d [10:56:40] 02CreateWiki/03validator 07CosmicAlpha 03b2b5d85 Remove [10:57:20] [Grafana] RESOLVED: High Job Queue Backlog https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [10:59:49] miraheze/CreateWiki - Universal-Omega the build has errored. [10:59:54] [02CreateWiki] 07Universal-Omega pushed 1 new commit to 03validator 13https://github.com/miraheze/CreateWiki/commit/c0ed0836215f63a25f568fc8139a8ce6ad57ab49 [10:59:54] 02CreateWiki/03validator 07CosmicAlpha 03c0ed083 Use str_ends_with [11:02:22] [02CreateWiki] 07Universal-Omega pushed 1 new commit to 03validator 13https://github.com/miraheze/CreateWiki/commit/f7d62f25c36154adc5709b1c7141ead8940de8f9 [11:02:22] 02CreateWiki/03validator 07CosmicAlpha 03f7d62f2 Fix [11:02:57] miraheze/CreateWiki - Universal-Omega the build has errored. [11:11:24] miraheze/CreateWiki - Universal-Omega the build has errored. [11:12:03] [02CreateWiki] 07Universal-Omega pushed 1 new commit to 03validator 13https://github.com/miraheze/CreateWiki/commit/580a91b5d6e46b5c16aa2b81919feb35c1b9fd4f [11:12:03] 02CreateWiki/03validator 07CosmicAlpha 03580a91b Fix [11:20:51] miraheze/CreateWiki - Universal-Omega the build has errored. [11:21:14] [02CreateWiki] 07Universal-Omega pushed 1 new commit to 03validator 13https://github.com/miraheze/CreateWiki/commit/dc88a8b6fdb9315e4e6a9a75a7e5e4ae2ca98bcd [11:21:14] 02CreateWiki/03validator 07CosmicAlpha 03dc88a8b Fix [11:33:03] miraheze/CreateWiki - Universal-Omega the build passed. [12:27:48] PROBLEM - db171 Current Load on db171 is CRITICAL: LOAD CRITICAL - total load average: 13.23, 11.25, 7.15 [12:29:48] RECOVERY - db171 Current Load on db171 is OK: LOAD OK - total load average: 7.39, 10.00, 7.21 [13:39:43] [02puppet] 07paladox created 03paladox-patch-6 (+1 new commit) 13https://github.com/miraheze/puppet/commit/1c7903441754 [13:39:43] 02puppet/03paladox-patch-6 07paladox 031c79034 nginx: Fix set_read_ip_from [13:39:46] [02puppet] 07paladox opened pull request #4228: nginx: Fix set_read_ip_from (03master...03paladox-patch-6) 13https://github.com/miraheze/puppet/pull/4228 [13:40:16] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-6 13https://github.com/miraheze/puppet/commit/66524b1f4d1740118a02b1c7e36c717d072de720 [13:40:17] 02puppet/03paladox-patch-6 07paladox 0366524b1 Update nginx.conf.erb [13:45:12] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-6 13https://github.com/miraheze/puppet/commit/46693c2b02059b27af55947d90a4d292622d449d [13:45:12] 02puppet/03paladox-patch-6 07paladox 0346693c2 Update nginx.conf.erb [13:46:19] !log [macfan@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php /srv/mediawiki/1.43/maintenance/rebuildall.php --wiki=spongebobfanonwiki (END - exit=0) [13:46:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [13:46:21] !log [macfan@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php /srv/mediawiki/1.43/maintenance/initSiteStats.php --wiki=spongebobfanonwiki --update (END - exit=0) [13:46:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [13:47:26] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-6 13https://github.com/miraheze/puppet/commit/f3a092ea1574f2019c581179c51564346d263e54 [13:47:26] 02puppet/03paladox-patch-6 07paladox 03f3a092e Update nginx.conf.erb [13:47:37] [02puppet] 07paladox merged pull request #4228: nginx: Fix set_read_ip_from (03master...03paladox-patch-6) 13https://github.com/miraheze/puppet/pull/4228 [13:47:37] [02puppet] 07paladox pushed 1 new commit to 03master 13https://github.com/miraheze/puppet/commit/143a22d845d6fbd02376ab503e8df51995b77a1a [13:47:37] 02puppet/03master 07paladox 03143a22d nginx: Fix set_read_ip_from (#4228) [13:47:38] [02puppet] 07paladox 04deleted 03paladox-patch-6 at 03f3a092e 13https://github.com/miraheze/puppet/commit/f3a092e [13:47:43] PROBLEM - mw171 Puppet on mw171 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:48:36] PROBLEM - mw153 Puppet on mw153 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:49:09] [02puppet] 07coderabbitai[bot] commented on pull request #4228: --- […] 13https://github.com/miraheze/puppet/pull/4228#issuecomment-2692240460 [13:49:43] RECOVERY - mw171 Puppet on mw171 is OK: OK: Puppet is currently enabled, last run 23 seconds ago with 0 failures [13:50:36] RECOVERY - mw153 Puppet on mw153 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [13:58:58] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 25.02, 14.89, 7.77 [14:00:58] RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 4.63, 10.49, 7.02 [14:06:42] !log increased cp38 ram to 30gib [14:06:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:20:58] [02mw-config] 07paladox created 03paladox-patch-2 (+1 new commit) 13https://github.com/miraheze/mw-config/commit/d737359ef268 [14:20:58] 02mw-config/03paladox-patch-2 07paladox 03d737359 Add mw191,mw192,mw193,mw201,mw202 and mw203 to wgLinterSubmitterWhitelist… [14:21:08] [02mw-config] 07paladox opened pull request #5839: Add mw191,mw192,mw193,mw201,mw202 and mw203 to wgLinterSubmitterWhitelist (03master...03paladox-patch-2) 13https://github.com/miraheze/mw-config/pull/5839 [14:21:36] [02mw-config] 07paladox merged pull request #5839: Add mw191,mw192,mw193,mw201,mw202 and mw203 to wgLinterSubmitterWhitelist (03master...03paladox-patch-2) 13https://github.com/miraheze/mw-config/pull/5839 [14:21:36] [02mw-config] 07paladox pushed 1 new commit to 03master 13https://github.com/miraheze/mw-config/commit/4ff37ce779af3bac7d707ac058a374d50828145d [14:21:36] 02mw-config/03master 07paladox 034ff37ce Add mw191,mw192,mw193,mw201,mw202 and mw203 to wgLinterSubmitterWhitelist (#5839)… [14:21:37] [02mw-config] 07paladox 04deleted 03paladox-patch-2 at 03d737359 13https://github.com/miraheze/mw-config/commit/d737359 [14:21:40] [02mw-config] 07github-actions[bot] created 03paladox-patch-2 (+2 new commits) 13https://github.com/miraheze/mw-config/compare/d737359ef268^...e7b69cf8ebb3 [14:21:40] 02mw-config/03paladox-patch-2 07paladox 03d737359 Add mw191,mw192,mw193,mw201,mw202 and mw203 to wgLinterSubmitterWhitelist… [14:21:41] 02mw-config/03paladox-patch-2 07github-actions 03e7b69cf CI: lint code to MediaWiki standards… [14:22:00] !log [paladox@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all [14:22:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:22:09] miraheze/mw-config - paladox the build passed. [14:22:13] !log [paladox@test151] starting deploy of {'pull': 'config', 'config': True} to test151 [14:22:14] !log [paladox@test151] finished deploy of {'pull': 'config', 'config': True} to test151 - SUCCESS in 0s [14:22:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:22:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:22:22] !log [paladox@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 22s [14:22:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:22:30] miraheze/mw-config - paladox the build passed. [16:20:05] PROBLEM - matomo151 Current Load on matomo151 is WARNING: LOAD WARNING - total load average: 7.35, 6.51, 5.12 [16:23:59] RECOVERY - matomo151 Current Load on matomo151 is OK: LOAD OK - total load average: 6.40, 6.69, 5.52 [16:27:06] PROBLEM - az-wiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'az-wiki.com' expires in 14 day(s) (Sun 16 Mar 2025 04:04:50 PM GMT +0000). [16:27:18] [02ssl] 07WikiTideSSLBot pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/63698d86c304fefa317d265c19fa10f6767d7fb2 [16:27:18] 02ssl/03master 07WikiTideSSLBot 0363698d8 Bot: Update SSL cert for az-wiki.com [16:29:39] PROBLEM - magistro.co - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'magistro.co' expires in 14 day(s) (Sun 16 Mar 2025 03:59:30 PM GMT +0000). [16:29:49] [02ssl] 07WikiTideSSLBot pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/ea1b3532dc01f6af299f19cc8cfd1df1040b75a8 [16:29:49] 02ssl/03master 07WikiTideSSLBot 03ea1b353 Bot: Update SSL cert for magistro.co [16:56:39] RECOVERY - az-wiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'az-wiki.com' will expire on Fri 30 May 2025 03:28:41 PM GMT +0000. [16:59:32] RECOVERY - magistro.co - LetsEncrypt on sslhost is OK: OK - Certificate 'magistro.co' will expire on Fri 30 May 2025 03:31:14 PM GMT +0000. [17:02:17] [02mw-config] 07Universal-Omega pushed 1 new commit to 03master 13https://github.com/miraheze/mw-config/commit/f544f3f8d427efa3b79f48b99c1f693b2965bd22 [17:02:17] 02mw-config/03master 07CosmicAlpha 03f544f3f Reschedule maintenance [17:03:11] miraheze/mw-config - Universal-Omega the build passed. [17:03:22] !log [universalomega@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all [17:03:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:03:42] !log [universalomega@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 20s [17:03:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:28:58] PROBLEM - cp38 Disk Space on cp38 is WARNING: DISK WARNING - free space: / 49926MiB (10% inode=99%); [18:04:33] [02CreateWiki] 07Universal-Omega created 03rename-hook (+1 new commit) 13https://github.com/miraheze/CreateWiki/commit/b261af668ac7 [18:04:33] 02CreateWiki/03rename-hook 07CosmicAlpha 03b261af6 Don't run onCreateWikiRename in maintenance script… [18:04:36] [02CreateWiki] 07Universal-Omega opened pull request #684: Don't run onCreateWikiRename in maintenance script (03master...03rename-hook) 13https://github.com/miraheze/CreateWiki/pull/684 [18:04:42] [02CreateWiki] 07coderabbitai[bot] commented on pull request #684: --- […] 13https://github.com/miraheze/CreateWiki/pull/684#issuecomment-2692346594 [18:06:49] [02ssl] 07MacFan4000 pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/3f8451bce6e551fdd1c06a577c3a1ba8bf17ba99 [18:06:50] 02ssl/03master 07MacFan4000 033f8451b add marbleblast.wiki [18:11:54] [02CreateWiki] 07Universal-Omega pushed 1 new commit to 03validator 13https://github.com/miraheze/CreateWiki/commit/cdc1be76a1b35ec16725826c773b970fa97032d6 [18:11:54] 02CreateWiki/03validator 07CosmicAlpha 03cdc1be7 Update comment [18:16:38] miraheze/CreateWiki - Universal-Omega the build passed. [18:23:36] miraheze/CreateWiki - Universal-Omega the build passed. [18:27:06] [02puppet] 07Reception123 merged 07ellieknowsyou's pull request #4208: Add drive.google.com to the CSP (07miraheze:03master...07ellieknowsyou:03patch-1) 13https://github.com/miraheze/puppet/pull/4208 [18:27:06] [02puppet] 07Reception123 pushed 1 new commit to 03master 13https://github.com/miraheze/puppet/commit/f9ea5a310a434bae3438d831b7ba1f1f6278963d [18:27:07] 02puppet/03master 07Ellie 03f9ea5a3 Add drive.google.com to the CSP (#4208)… [18:38:52] [02MirahezeMagic] 07Reception123 pushed 1 new commit to 03master 13https://github.com/miraheze/MirahezeMagic/commit/103ea65269b7ee392c3d58c3750a50cd0400ba50 [18:38:52] 02MirahezeMagic/03master 07Reception123 03103ea65 correct closed and inactive sitenotices with new DP thresholds… [18:44:52] [02CreateWiki] 07Universal-Omega merged pull request #684: Don't run onCreateWikiRename in maintenance script (03master...03rename-hook) 13https://github.com/miraheze/CreateWiki/pull/684 [18:44:53] [02CreateWiki] 07Universal-Omega pushed 1 new commit to 03master 13https://github.com/miraheze/CreateWiki/commit/98563f4d3a0a369a99b9ed64062ef70c1d96f11b [18:44:54] 02CreateWiki/03master 07CosmicAlpha 0398563f4 Don't run onCreateWikiRename in maintenance script (#684)… [18:44:56] [02CreateWiki] 07Universal-Omega 04deleted 03rename-hook at 03b261af6 13https://github.com/miraheze/CreateWiki/commit/b261af6 [18:50:31] miraheze/MirahezeMagic - Reception123 the build passed. [18:56:45] miraheze/CreateWiki - Universal-Omega the build passed. [19:08:54] [02ssl] 07Reception123 pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/5867860b2f5503abaaaea38eb9cfafde714d33f9 [19:08:54] 02ssl/03master 07Reception123 035867860 www.langrisser.wiki redirect [21:12:47] PROBLEM - mwtask171 Current Load on mwtask171 is CRITICAL: LOAD CRITICAL - total load average: 25.69, 18.40, 9.28 [21:16:46] PROBLEM - mwtask171 Current Load on mwtask171 is WARNING: LOAD WARNING - total load average: 22.14, 21.59, 12.74 [21:18:45] RECOVERY - mwtask171 Current Load on mwtask171 is OK: LOAD OK - total load average: 3.26, 14.57, 11.24 [21:23:28] [02CreateWiki] 07Universal-Omega pushed 1 new commit to 03validator 13https://github.com/miraheze/CreateWiki/commit/c49280e5823db7b1c1c8599a0cff4b0be79a804c [21:23:28] 02CreateWiki/03validator 07CosmicAlpha 03c49280e Rename methods and some other cleanup [21:25:45] PROBLEM - mwtask151 Current Load on mwtask151 is CRITICAL: LOAD CRITICAL - total load average: 25.19, 16.29, 7.71 [21:31:45] PROBLEM - mwtask151 Current Load on mwtask151 is WARNING: LOAD WARNING - total load average: 23.69, 21.87, 13.11 [21:33:45] PROBLEM - mwtask151 Current Load on mwtask151 is CRITICAL: LOAD CRITICAL - total load average: 24.98, 22.90, 14.55 [21:35:45] PROBLEM - mwtask151 Current Load on mwtask151 is WARNING: LOAD WARNING - total load average: 15.57, 20.84, 14.87 [21:36:24] miraheze/CreateWiki - Universal-Omega the build passed. [21:39:45] RECOVERY - mwtask151 Current Load on mwtask151 is OK: LOAD OK - total load average: 11.59, 18.97, 15.65 [22:07:05] [02mw-config] 07pixDeVl opened pull request #5840: Update date format for ReplaceText message (07miraheze:03master...07pixDeVl:03patch-2) 13https://github.com/miraheze/mw-config/pull/5840 [22:07:57] miraheze/mw-config - pixDeVl the build passed. [22:15:23] miraheze/mw-config - pixDeVl the build passed. [22:16:18] [02mw-config] 07pixDeVl retitled PR #5840: "Update date format for ReplaceText message" ➜ "Update restriction text for some extensions" 13https://github.com/miraheze/mw-config/pull/5840 [22:37:53] [02CreateWiki] 07pixDeVl left a file comment in pull request #605 037ad5153: @Universal-Omega configured incorrectly being not defining no edits or default or just not defining no edits? 13https://github.com/miraheze/CreateWiki/pull/605#discussion_r1976502276