[00:07:00] PROBLEM - wiki.digitaldesignhq.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.digitaldesignhq.com could not be found [00:07:01] PROBLEM - antiguabarbudacalypso.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for antiguabarbudacalypso.com could not be found [00:07:01] PROBLEM - zh.internetpedia.tk - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for zh.internetpedia.tk could not be found [00:07:02] PROBLEM - savage-wiki.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for savage-wiki.com could not be found [00:13:51] RECOVERY - wiki.digitaldesignhq.com - reverse DNS on sslhost is OK: SSL OK - wiki.digitaldesignhq.com reverse DNS resolves to cp13.miraheze.org - CNAME OK [00:13:55] RECOVERY - antiguabarbudacalypso.com - reverse DNS on sslhost is OK: SSL OK - antiguabarbudacalypso.com reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [00:13:56] RECOVERY - savage-wiki.com - reverse DNS on sslhost is OK: SSL OK - savage-wiki.com reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [00:13:56] RECOVERY - zh.internetpedia.tk - reverse DNS on sslhost is OK: SSL OK - zh.internetpedia.tk reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [00:18:44] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 5.28, 7.20, 7.92 [00:19:04] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 5.27, 6.60, 7.88 [00:19:56] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 3.72, 5.86, 7.73 [00:22:48] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 3.95, 5.62, 7.69 [00:24:59] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 3.20, 4.63, 6.60 [00:25:55] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 4.49, 4.74, 6.59 [00:26:45] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 4.31, 5.12, 6.56 [00:28:48] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 3.56, 4.61, 6.58 [03:23:11] PROBLEM - cp12 Current Load on cp12 is WARNING: WARNING - load average: 2.00, 1.91, 1.39 [03:25:10] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JcPVJ [03:25:12] [02miraheze/services] 07MirahezeSSLBot 035b30ee0 - BOT: Updating services config for wikis [03:27:10] RECOVERY - cp12 Current Load on cp12 is OK: OK - load average: 1.05, 1.54, 1.37 [06:05:19] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 9.31, 7.15, 5.25 [06:11:19] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.08, 7.12, 5.93 [06:13:18] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 6.14, 6.70, 5.91 [06:15:45] PROBLEM - mw8 Disk Space on mw8 is WARNING: DISK WARNING - free space: / 2081 MB (10% inode=74%); [06:17:44] RECOVERY - mw8 Disk Space on mw8 is OK: DISK OK - free space: / 2114 MB (11% inode=74%); [06:25:16] RECOVERY - services3 APT on services3 is OK: APT OK: 39 packages available for upgrade (0 critical updates). [06:33:10] !log [reception@jobrunner3] sudo -u www-data php /srv/mediawiki/w/maintenance/refreshLinks.php --wiki crappygameswiki [06:33:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:35:32] RECOVERY - services4 APT on services4 is OK: APT OK: 39 packages available for upgrade (0 critical updates). [06:36:07] !log [reception@jobrunner4] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki picrosswiki /home/reception/Picross_Wiki-20210630143653.xml --username-prefix=wikia:picross [06:36:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:41:17] [02miraheze/dns] 07Reception123 pushed 031 commit to 03master [+1/-0/±0] 13https://git.io/JcXT5 [06:41:18] [02miraheze/dns] 07Reception123 03208f073 - add fortis.wiki zone [06:45:48] [02miraheze/ssl] 07Reception123 pushed 031 commit to 03master [+1/-0/±2] 13https://git.io/JcXIe [06:45:50] [02miraheze/ssl] 07Reception123 03243f032 - add fortis.wiki cert [06:48:44] !log [reception@jobrunner4] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki biggeruniversewiki /home/reception/biggeruniverse_pages_full.xml --username-prefix=wikia:biggeruniverse [06:48:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [07:19:16] !log [reception@jobrunner3] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki trollpastawiki /home/reception/detrollpasta_pages_full.xml --username-prefix=wikia:trollpasta [07:19:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [07:39:56] [02MirahezeMagic] 07Reception123 commented on pull request 03#268: add Recaptcha V3 - 13https://git.io/JcXCB [07:57:22] !log [reception@jobrunner3] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki trollpastawiki /home/reception/detrollpasta_pages_full.xml --username-prefix=wikia:trollpasta --help [07:57:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [08:02:34] PROBLEM - mw8 APT on mw8 is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. [08:04:35] PROBLEM - mw8 APT on mw8 is CRITICAL: APT CRITICAL: 62 packages available for upgrade (18 critical updates). [13:06:16] PROBLEM - mw8 Disk Space on mw8 is WARNING: DISK WARNING - free space: / 2046 MB (10% inode=74%); [13:08:15] RECOVERY - mw8 Disk Space on mw8 is OK: DISK OK - free space: / 2131 MB (11% inode=74%); [15:33:05] PROBLEM - mw8 APT on mw8 is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. [15:35:06] PROBLEM - mw8 APT on mw8 is CRITICAL: APT CRITICAL: 62 packages available for upgrade (18 critical updates). [16:03:00] @Reception123 can you take another look at this task? I have no idea why the link now returns a user facing error on meta, but on test3 you get redirected correct. The error the task is about still happens on test3 though. https://phabricator.miraheze.org/T7386 [16:03:02] [url] ⚓ T7386 PHP Fatal Error from /srv/mediawiki/w/extensions/OAuth/vendor/league/oauth2-server/src/ResponseTypes/BearerTokenResponse.php: Method MediaWiki\Extensions\OAuth\Entity\AccessTokenEntity::__toS | phabricator.miraheze.org [16:07:12] @MarkusRost Sorry about how long that task is taking. When I try the same link but with test3 instead of Meta it seems to be redirecting me to https://settings.wikibot.de/login?action=failed for some reason [16:07:14] [url] Login – Wiki-Bot Settings | settings.wikibot.de [16:08:05] yes, that's correct [16:08:40] But when my app is trying to fetch the token I'm getting a 500 error [16:09:30] Ah, I understand. Well it really doesn't seem easy to test so I think the best way to move forward is probably for me to just try to set those two variables on test3 and see if it's fixed once the two variables are set [16:11:23] sure, can you let me know when that is done so I can run my test app with miraheze oauth enabled? Otherwise you would have to set up an oauth2 app yourself for testing as mine would still give you an error for unknown state 😅 [16:22:19] [02puppet] 07MusikAnimal synchronize pull request 03#1799: Whitelist *.archive.org - 13https://git.io/Jctix [16:28:15] sure! [16:35:55] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 8.16, 6.48, 5.66 [16:37:14] PROBLEM - test3 MediaWiki Rendering on test3 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 324 bytes in 0.029 second response time [16:37:54] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 6.65, 6.58, 5.80 [16:39:13] RECOVERY - test3 MediaWiki Rendering on test3 is OK: HTTP OK: HTTP/1.1 200 OK - 20019 bytes in 0.212 second response time [16:41:30] interesting, now there's a Application Connection Error [16:41:50] where? [16:42:05] on test3 [16:42:18] after I tried setting those two settings [16:45:57] > The authorization server encountered an unexpected condition that prevented it from fulfilling the request. [16:45:59] hm... [16:46:05] that's weird [16:46:28] yeah, not sure why that is :( and unfortunately we're also encountering a logging issue at the moment (see https://phabricator.miraheze.org/T7599) so I can't look for errors in the log [16:46:29] [url] ⚓ T7599 Graylog search not working | phabricator.miraheze.org [16:50:33] Why does oauth have to be so complicated 😦 [16:55:01] Reception123: disk space [16:57:23] RhinosF1: well yes but not sure where, it's definitely not graylog's disk space as a whole as that's fine [16:57:30] and I also don't want to accidentally delete actual logs [16:57:41] @MarkusRost I have no idea, but it does seem to be 😦 [16:58:28] And I can't even change the redirect url for an app for testing [19:14:30] PROBLEM - uncyclopediakr.miraheze.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for uncyclopediakr.miraheze.org could not be found [19:14:31] PROBLEM - madzebrascience.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for madzebrascience.wiki could not be found [19:14:32] PROBLEM - patternarchive.online - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for patternarchive.online could not be found [19:14:32] PROBLEM - gp.ct777.cf - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for gp.ct777.cf could not be found [19:14:35] PROBLEM - wiki.graalmilitary.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.graalmilitary.com could not be found [19:14:35] PROBLEM - commons.gyaanipedia.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for commons.gyaanipedia.com could not be found [19:14:43] PROBLEM - orain.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for orain.org could not be found [19:14:48] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 14.67, 10.45, 6.84 [19:15:11] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 13.18, 9.93, 6.92 [19:15:28] PROBLEM - thestarsareright.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for thestarsareright.org could not be found [19:16:15] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 12.09, 8.51, 6.33 [19:16:49] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 7.36, 7.41, 6.01 [19:18:06] PROBLEM - permanentfuturelab.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for permanentfuturelab.wiki could not be found [19:18:46] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 9.22, 8.09, 6.43 [19:20:44] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 7.46, 7.81, 6.53 [19:21:09] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 5.91, 7.73, 7.00 [19:21:14] RECOVERY - madzebrascience.wiki - reverse DNS on sslhost is OK: SSL OK - madzebrascience.wiki reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [19:21:15] RECOVERY - patternarchive.online - reverse DNS on sslhost is OK: SSL OK - patternarchive.online reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [19:21:17] RECOVERY - gp.ct777.cf - reverse DNS on sslhost is OK: SSL OK - gp.ct777.cf reverse DNS resolves to cp13.miraheze.org - CNAME OK [19:21:20] RECOVERY - commons.gyaanipedia.com - reverse DNS on sslhost is OK: SSL OK - commons.gyaanipedia.com reverse DNS resolves to cp13.miraheze.org - CNAME OK [19:21:22] RECOVERY - wiki.graalmilitary.com - reverse DNS on sslhost is OK: SSL OK - wiki.graalmilitary.com reverse DNS resolves to cp13.miraheze.org - CNAME OK [19:21:27] RECOVERY - uncyclopediakr.miraheze.org - reverse DNS on sslhost is OK: SSL OK - uncyclopediakr.miraheze.org reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [19:21:33] RECOVERY - orain.org - reverse DNS on sslhost is OK: SSL OK - orain.org reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [19:22:31] @owen you have mail [19:22:34] RECOVERY - thestarsareright.org - reverse DNS on sslhost is OK: SSL OK - thestarsareright.org reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [19:22:44] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 8.25, 7.91, 6.71 [19:23:09] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.44, 8.59, 7.43 [19:23:52] PROBLEM - en.petrawiki.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for en.petrawiki.org could not be found [19:24:17] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 5.64, 7.61, 6.96 [19:24:18] PROBLEM - www.erikapedia.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.erikapedia.com could not be found [19:24:18] PROBLEM - nonbinary.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for nonbinary.wiki could not be found [19:24:23] PROBLEM - www.rangpurpedia.xyz - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.rangpurpedia.xyz could not be found [19:24:23] PROBLEM - podpedia.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for podpedia.org could not be found [19:24:24] PROBLEM - wiki.mxlinuxusers.de - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.mxlinuxusers.de could not be found [19:24:25] PROBLEM - lukaibrisimovic.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lukaibrisimovic.org could not be found [19:24:25] PROBLEM - ecole.science - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for ecole.science could not be found [19:24:41] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 6.43, 7.18, 6.58 [19:24:47] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 5.77, 7.74, 7.21 [19:24:50] RECOVERY - permanentfuturelab.wiki - reverse DNS on sslhost is OK: SSL OK - permanentfuturelab.wiki reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [19:25:08] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 5.76, 7.40, 7.13 [19:26:18] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 8.77, 7.88, 7.12 [19:26:41] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 8.88, 7.46, 6.73 [19:26:49] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 11.01, 8.42, 7.48 [19:27:10] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 10.34, 8.49, 7.54 [19:30:33] RECOVERY - en.petrawiki.org - reverse DNS on sslhost is OK: SSL OK - en.petrawiki.org reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [19:31:06] RECOVERY - nonbinary.wiki - reverse DNS on sslhost is OK: SSL OK - nonbinary.wiki reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [19:31:12] RECOVERY - www.rangpurpedia.xyz - reverse DNS on sslhost is OK: SSL OK - www.rangpurpedia.xyz reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [19:31:13] RECOVERY - podpedia.org - reverse DNS on sslhost is OK: SSL OK - podpedia.org reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [19:31:14] RECOVERY - wiki.mxlinuxusers.de - reverse DNS on sslhost is OK: SSL OK - wiki.mxlinuxusers.de reverse DNS resolves to cp13.miraheze.org - CNAME OK [19:31:15] RECOVERY - ecole.science - reverse DNS on sslhost is OK: SSL OK - ecole.science reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [19:31:19] RECOVERY - www.erikapedia.com - reverse DNS on sslhost is OK: SSL OK - www.erikapedia.com reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [19:31:24] RECOVERY - lukaibrisimovic.org - reverse DNS on sslhost is OK: SSL OK - lukaibrisimovic.org reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [19:36:15] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 5.59, 7.63, 7.82 [19:36:39] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 6.62, 7.72, 7.45 [19:36:45] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 4.71, 7.72, 7.96 [19:39:11] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 5.27, 7.22, 7.79 [19:42:16] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 9.67, 7.02, 7.34 [19:42:44] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 11.07, 7.83, 7.37 [19:42:48] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 9.16, 7.07, 7.42 [19:43:10] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 10.36, 8.33, 8.04 [19:44:15] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 6.30, 6.83, 7.25 [19:44:41] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 6.49, 7.34, 7.26 [19:44:47] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 4.88, 6.15, 7.04 [19:45:09] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 5.17, 7.14, 7.64 [19:46:13] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 3.42, 5.63, 6.76 [19:46:47] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.19, 5.65, 6.73 [19:48:39] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 5.78, 6.22, 6.78 [19:53:10] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 3.51, 4.96, 6.42 [20:05:57] Responded [20:42:59] @owen noted [23:42:13] PROBLEM - cp12 Current Load on cp12 is WARNING: WARNING - load average: 1.67, 1.84, 1.40 [23:44:12] RECOVERY - cp12 Current Load on cp12 is OK: OK - load average: 1.22, 1.60, 1.37