[00:01:00] [02statichelp] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/statichelp/commit/2cf4804b69adc5ac4909614bbc356c784519edfe [00:01:00] 02statichelp/03main 07WikiTideBot 032cf4804 Bot: Auto-update Tech namespace pages 2026-05-22 00:00:57 [00:04:47] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php MirahezeMagic:RenameDatabase --wiki=loginwiki --rename --old=alentiniszerwiki --new=conquerorsofthecraftwiki --user=Skye (END - exit=0) [00:04:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:30:22] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/a16355968591f85741ce9d92ae079f25c2a74450 [00:30:23] 02ssl/03main 07WikiTideBot 03a163559 Bot: Auto-update domain lists [00:43:34] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php CreateWiki:SetContainersAccess --wiki=conquerorsofthecraftwiki (END - exit=0) [00:43:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:46:06] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php MirahezeMagic:RenameDatabase --wiki=loginwiki --rename --old=impostorsyndromewiki --new=malwaredlcwiki --user=Skye (END - exit=0) [00:46:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:49:36] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php CreateWiki:SetContainersAccess --wiki=malwaredlcwiki (END - exit=0) [00:49:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:57:04] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php MirahezeMagic:RenameDatabase --wiki=loginwiki --rename --old=osterholtwiki --new=havoreywiki --user=Skye (END - exit=0) [00:57:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:59:08] [02ManageWiki] 07lihaohong6 opened pull request #790: T15415: increase the time limit for ManageWiki maintenance scripts (07miraheze:03main...07lihaohong6:03clock) 13https://github.com/miraheze/ManageWiki/pull/790 [00:59:08] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php CreateWiki:SetContainersAccess --wiki=havoreywiki (END - exit=0) [00:59:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:04:19] miraheze/ManageWiki - lihaohong6 the build passed. [01:06:04] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php MirahezeMagic:RenameDatabase --wiki=loginwiki --rename --old=uncwikiwiki --new=vyetnapediawiki --user=Skye (END - exit=0) [01:06:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:06:31] Just a heads up I'm going to merge https://github.com/miraheze/mw-config/pull/6418. It should be low-risk and I've done it once on beta already. Just to be safe I'll create a test wiki on prod to be sure. [01:06:31] [GitHub] [miraheze/mw-config #6418] open PR by lihaohong6, created 1 day, 3 hours ago: T15367: changes to default MediaWiki extensions | None of these require database changes and should have a low risk of causing problems. [01:07:53] Please post this in #tech instead of here [01:08:47] [02mw-config] 07lihaohong6 merged pull request #6418: T15367: changes to default MediaWiki extensions (07miraheze:03main...07lihaohong6:03T15367) 13https://github.com/miraheze/mw-config/pull/6418 [01:08:47] [02mw-config] 07lihaohong6 pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/d678043eb4b3cca020ae4784239fba1a68982d9f [01:08:48] 02mw-config/03main 07Peter Li 03d678043 T15367: changes to default MediaWiki extensions (#6418)… [01:09:14] !log [petramagna@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all [01:09:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:09:39] !log [petramagna@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 24s [01:09:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:10:00] miraheze/mw-config - lihaohong6 the build passed. [01:12:56] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php CreateWiki:SetContainersAccess --wiki=vyetnapediawiki (END - exit=0) [01:12:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:21:01] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php MirahezeMagic:RenameDatabase --wiki=loginwiki --rename --old=negapaintwiki --new=aswonuwiki --user=Skye (END - exit=0) [01:21:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:23:14] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php CreateWiki:SetContainersAccess --wiki=aswonuwiki (END - exit=0) [01:23:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:33:12] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php MirahezeMagic:RenameDatabase --wiki=loginwiki --rename --old=amcfoundationofficialwiki --new=uaccfoundationofficialwiki --user=Skye (END - exit=0) [01:33:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:34:31] [02mw-config] 07lihaohong6 opened pull request #6421: T15367: changes to default MediaWiki extensions (part 2) (07miraheze:03main...07lihaohong6:03T15367) 13https://github.com/miraheze/mw-config/pull/6421 [01:35:38] miraheze/mw-config - lihaohong6 the build passed. [01:36:16] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php CreateWiki:SetContainersAccess --wiki=uaccfoundationofficialwiki (END - exit=0) [01:36:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:39:37] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php MirahezeMagic:RenameDatabase --wiki=loginwiki --rename --old=raccoinwiki --new=doomsdayhunterswiki --user=Skye (END - exit=0) [01:39:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:41:55] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php CreateWiki:SetContainersAccess --wiki=doomsdayhunterswiki (END - exit=0) [01:41:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:44:45] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php MirahezeMagic:RenameDatabase --wiki=loginwiki --rename --old=funiraccoongamewiki --new=hyperchargewiki --user=Skye (END - exit=0) [01:44:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:45:13] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php CreateWiki:SetContainersAccess --wiki=hyperchargewiki (END - exit=0) [01:45:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [11:28:40] [02CreateWiki] 07Reception123 created 03Reception123-patch-6 (+1 new commit) 13https://github.com/miraheze/CreateWiki/commit/de9333c66215 [11:28:41] 02CreateWiki/03Reception123-patch-6 07Reception123 03de9333c allow wiki requestors to abandon their request (T14115) - ci redo [11:29:15] [02CreateWiki] 07Reception123 opened pull request #824: allow wiki requestors to abandon their request (T14115) - ci redo (03main...03Reception123-patch-6) 13https://github.com/miraheze/CreateWiki/pull/824 [11:33:58] [02CreateWiki] 07codecov[bot] commented on pull request #824: :x: Patch coverage is `0%` with `32 lines` in your changes missing coverage. Please review. […] 13https://github.com/miraheze/CreateWiki/pull/824#issuecomment-4518284582 [11:36:43] miraheze/CreateWiki - Reception123 the build passed. [11:38:04] [02CreateWiki] 07Reception123 closed pull request #824: allow wiki requestors to abandon their request (T14115) - ci redo (03main...03Reception123-patch-6) 13https://github.com/miraheze/CreateWiki/pull/824 [11:38:05] [02CreateWiki] 07Reception123 commented on pull request #824: looks like that won't work 13https://github.com/miraheze/CreateWiki/pull/824#issuecomment-4518313032 [11:38:08] [02CreateWiki] 07Reception123 04deleted 03Reception123-patch-6 at 03de9333c 13https://api.github.com/repos/miraheze/CreateWiki/commit/de9333c [11:45:57] [02mw-config] 07Conscience93 left a file comment in pull request #6405 03a5bb569: oh they are different? Oops lemme change it 13https://github.com/miraheze/mw-config/pull/6405#discussion_r3288111320 [11:49:13] miraheze/mw-config - Conscience93 the build passed. [15:33:32] PROBLEM - puppet181 Check unit status of listdomains_github_push on puppet181 is CRITICAL: CRITICAL: Status of the systemd unit listdomains_github_push [16:31:32] RECOVERY - puppet181 Check unit status of listdomains_github_push on puppet181 is OK: OK: Status of the systemd unit listdomains_github_push [16:53:50] [02CreateWiki] 07Reception123 commented on pull request #818: The final patch has been successfully tested on Beta. The comments/suggestions have all been addressed and while it would've been ideal to also resolve the codecov CI test related issues, PRs have been approved in the past without them […] 13https://github.com/miraheze/CreateWiki/pull/818#issuecomment-4520842945 [16:53:56] [02CreateWiki] 07Reception123 merged pull request #818: allow wiki requestors to abandon their request (T14115) (03main...03Reception123-patch-4) 13https://github.com/miraheze/CreateWiki/pull/818 [16:53:56] [02CreateWiki] 07Reception123 pushed 1 new commit to 03main 13https://github.com/miraheze/CreateWiki/commit/d3f8ad010316b7d8cc5cb1e6e89d48c15d44b1d2 [16:53:56] 02CreateWiki/03main 07Reception123 03d3f8ad0 allow wiki requestors to abandon their request (T14115) (#818)… [16:53:57] [02CreateWiki] 07Reception123 04deleted 03Reception123-patch-4 at 036235685 13https://api.github.com/repos/miraheze/CreateWiki/commit/6235685 [16:56:35] [02CreateWiki] 07Reception123 pushed 1 new commit to 03Reception123-patch-1 13https://github.com/miraheze/CreateWiki/commit/4bec7f21dfac67003c8e6221dc8f2cf1828d335f [16:56:35] 02CreateWiki/03Reception123-patch-1 07Reception123 034bec7f2 change wording slightly [16:59:13] [02puppet] 07Reception123 pushed 1 new commit to 03Reception123-patch-1 13https://github.com/miraheze/puppet/commit/e4ab4d6bfce7003118efe86b5b03c3cb03bbc2b1 [16:59:13] 02puppet/03Reception123-patch-1 07Reception123 03e4ab4d6 new name for maint script [17:00:16] miraheze/CreateWiki - Reception123 the build has errored. [17:02:13] miraheze/CreateWiki - Reception123 the build passed. [17:03:49] !log [reception@mwtask181] starting deploy of {'l10n': True, 'versions': '1.45', 'upgrade_extensions': 'CreateWiki'} to all [17:03:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:10:07] [02CreateWiki] 07Reception123 04force-pushed 03Reception123-patch-1 from 034bec7f2 to 032b1d46c 13https://api.github.com/repos/miraheze/CreateWiki/commits/Reception123-patch-1 [17:10:07] 02CreateWiki/03Reception123-patch-1 07Reception123 031cc624d maint script for abandoned wiki requests T11735… [17:10:08] 02CreateWiki/03Reception123-patch-1 07github-actions 03320df60 CI: lint code to MediaWiki standards… [17:10:08] 02CreateWiki/03Reception123-patch-1 07Reception123 0357e8271 Update DeclineStaleWikiRequests.php [17:10:09] 02CreateWiki/03Reception123-patch-1 07Reception123 033ee1cbe localise decline message [17:10:10] 02CreateWiki/03Reception123-patch-1 07Reception123 036948875 localise [17:11:25] [02CreateWiki] 07Reception123 pushed 1 new commit to 03Reception123-patch-1 13https://github.com/miraheze/CreateWiki/commit/5f71c3eee189c30b5c586bec9fbd8d2ef20f66e4 [17:11:26] 02CreateWiki/03Reception123-patch-1 07Reception123 035f71c3e forgot to allow comments when abandoning a req [17:12:36] !log [reception@mwtask181] finished deploy of {'l10n': True, 'versions': '1.45', 'upgrade_extensions': 'CreateWiki'} to all - SUCCESS in 527s [17:12:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:13:20] !log [reception@mwtask181] starting deploy of {'world': True, 'force': True, 'versions': '1.45', 'upgrade_extensions': 'CreateWiki'} to all [17:13:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:13:26] !log [reception@mwtask181] DEPLOY ABORTED: Non-Zero Exit Code in prep, see output. [17:13:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:14:08] !log [reception@mwtask181] starting deploy of {'world': True, 'versions': '1.45', 'upgrade_extensions': 'CreateWiki'} to all [17:14:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:14:11] !log [reception@mwtask181] DEPLOY ABORTED: Non-Zero Exit Code in prep, see output. [17:14:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:14:51] !log [reception@mwtask181] starting deploy of {'world': True, 'force': True, 'versions': '1.45'} to all [17:14:54] miraheze/CreateWiki - Reception123 the build has errored. [17:14:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:14:54] !log [reception@mwtask181] DEPLOY ABORTED: Non-Zero Exit Code in prep, see output. [17:14:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:21:08] [02CreateWiki] 07Reception123 pushed 1 new commit to 03Reception123-patch-1 13https://github.com/miraheze/CreateWiki/commit/5ca1fb73684586fc9ddcb30549442d9c7ba35ee4 [17:21:08] 02CreateWiki/03Reception123-patch-1 07Reception123 035ca1fb7 +addcomment [17:24:54] miraheze/CreateWiki - Reception123 the build has errored. [17:48:36] @reception123 do you need help with the deployment fail? [18:04:39] [02CreateWiki] 07Reception123 pushed 1 new commit to 03Reception123-patch-1 13https://github.com/miraheze/CreateWiki/commit/5c3a6a8d9382bcb7b2fd9d042480d1a460697416 [18:04:39] 02CreateWiki/03Reception123-patch-1 07Reception123 035c3a6a8 Update WikiRequestViewer.php [18:08:38] miraheze/CreateWiki - Reception123 the build has errored. [18:08:42] I managed to do it with just upgrade-ext but world is indeed failing [18:09:25] [1/4] ```Execute: sudo -u www-data http_proxy=http://bastion.fsslc.wtnet:8080 https_proxy=http://bastion.fsslc.wtnet:8080 composer update --no-dev --quiet [18:09:25] [2/4] Your requirements could not be resolved to an installable set of packages. [18:09:26] [3/4] Running update with --no-dev does not mean require-dev is ignored, it just means the packages will not be installed. If dev requirements are blocking the update you have to resolve those problems. [18:09:26] [4/4] ``` [18:26:56] [02CreateWiki] 07Reception123 pushed 1 new commit to 03Reception123-patch-1 13https://github.com/miraheze/CreateWiki/commit/b2e4835352809521e7a543c9fdd10c80fe58dadf [18:26:56] 02CreateWiki/03Reception123-patch-1 07Reception123 03b2e4835 maybe this will work? [18:31:05] miraheze/CreateWiki - Reception123 the build has errored. [18:35:25] [02CreateWiki] 07Reception123 pushed 1 new commit to 03Reception123-patch-1 13https://github.com/miraheze/CreateWiki/commit/216e8b01a58ebb58d0c7fef4efe9e2e6d3c879fb [18:35:25] 02CreateWiki/03Reception123-patch-1 07Reception123 03216e8b0 Update en.json [18:39:20] miraheze/CreateWiki - Reception123 the build has errored. [18:47:55] Run that command please without --quiet [18:48:00] In your shell [18:49:06] A failing deployment should be treated as UBN though [18:54:45] [02CreateWiki] 07pskyechology requested changes on pull request #822 13https://github.com/miraheze/CreateWiki/pull/822#pullrequestreview-4348196678 [18:54:46] [02CreateWiki] 07pskyechology left a file comment in pull request #822 03216e8b0: Warning in phpcs causes CI failure, separate wfMessage to a new variable? 13https://github.com/miraheze/CreateWiki/pull/822#discussion_r3290663383 [18:54:48] [02CreateWiki] 07pskyechology left a file comment in pull request #822 03216e8b0: Warning in phpcs causes CI failure 13https://github.com/miraheze/CreateWiki/pull/822#discussion_r3290659540 [19:00:23] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/563faa85c7af30726ab9c5c44878b652fca7dcfd [19:00:23] 02ssl/03main 07WikiTideBot 03563faa8 Bot: Auto-update domain lists [19:15:33] [1/12] ```reception@mwtask181:/srv/mediawiki/1.45$ sudo -u www-data http_proxy=http://bastion.fsslc.wtnet:8080 https_proxy=http://bastion.fsslc.wtnet:8080 composer update --no-dev [19:15:34] [2/12] Composer could not detect the root package (mediawiki/core) version, defaulting to '1.0.0'. See https://getcomposer.org/root-version [19:15:34] [3/12] > MediaWiki\Composer\VersionChecker::onEvent [19:15:34] [4/12] Loading composer repositories with package information [19:15:34] [5/12] Cannot create cache directory /home/reception/.cache/composer/repo/https---repo.packagist.org/, or directory is not writable. Proceeding without cache. See also cache-read-only config if your filesystem is read-only. [19:15:35] [6/12] Updating dependencies [19:15:35] [7/12] Your requirements could not be resolved to an installable set of packages. [19:15:35] [8/12] Problem 1 [19:15:36] [9/12] - Root composer.json requires mediawiki/chameleon-skin 5.x-dev (exact version match), found mediawiki/chameleon-skin[dev-dependabot/composer/mediawiki/mediawiki-phan-config-0.20.0, dev-master, dev-logo, dev-bs5-followup-fixes, dev-bs5-385, dev-bs53, 1.0.0, ..., 1.x-dev, 2.0, 2.1.0, 2.2.0, 2.3.0, 3.0.0, ..., 3.x-dev, 4.0.0, ..., 4.x-dev, 5.0.0, 5.0.1, 5. [19:15:36] [10/12] 0.2, 6.x-dev (alias of dev-master)] but these do not match your constraint and are therefore not installable. Make sure you either fix the constraint or avoid updating this package to keep the one present in the lock file (mediawiki/chameleon-skin[5.x-dev (alias of dev-master)]). [19:15:36] [11/12] Running update with --no-dev does not mean require-dev is ignored, it just means the packages will not be installed. If dev requirements are blocking the update you have to resolve those problems. [19:15:37] [12/12] ``` [19:17:39] PROBLEM - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is WARNING: WARNING - NGINX Error Rate is 44% [19:18:10] How the fuck does that suddenly start failing [19:18:47] I wonder if a version has been released [19:18:54] And they deleted an old one [19:18:58] I've got no idea... I guess --world isn't really used much so no one would've noticed [19:19:39] RECOVERY - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is OK: OK - NGINX Error Rate is 33% [19:19:40] --world is fairly key [19:19:50] If --world isn't working it should be UBN! [19:20:00] [02CreateWiki] 07Reception123 pushed 1 new commit to 03Reception123-patch-1 13https://github.com/miraheze/CreateWiki/commit/652e8cc0b081f46c61b624f1bd9c1e04f85d0558 [19:20:00] 02CreateWiki/03Reception123-patch-1 07Reception123 03652e8cc use separate var for message [19:20:26] There is no 5.x-dev [19:21:20] [02CreateWiki] 07github-actions[bot] pushed 1 new commit to 03Reception123-patch-1 13https://github.com/miraheze/CreateWiki/commit/57774c90ab1eb2dacc18680213e2d4041ddf2f49 [19:21:20] 02CreateWiki/03Reception123-patch-1 07github-actions 0357774c9 CI: lint code to MediaWiki standards… [19:22:14] [02mediawiki-repos] 07RhinosF1 created 03RhinosF1-patch-1 (+1 new commit) 13https://github.com/miraheze/mediawiki-repos/commit/70256ceecb17 [19:22:14] 02mediawiki-repos/03RhinosF1-patch-1 07RhinosF1 0370256ce Pin chameleon-skin version to 5.0.2… [19:22:19] [02mediawiki-repos] 07RhinosF1 opened pull request #146: Pin chameleon-skin version to 5.0.2 (03main...03RhinosF1-patch-1) 13https://github.com/miraheze/mediawiki-repos/pull/146 [19:22:29] @reception123 deploy https://github.com/miraheze/mediawiki-repos/pull/146 please [19:22:29] [GitHub] [miraheze/mediawiki-repos #146] open PR by RhinosF1, created 12 seconds ago: Pin chameleon-skin version to 5.0.2 | 5.X-Dev doesn't exist anymore as far as I can see [19:22:36] You'll have to pull it on puppet181 first [19:22:41] Then run task puppet [19:22:45] And deploy world again [19:23:02] Okay, deploying now [19:23:12] [02mediawiki-repos] 07Reception123 merged 07RhinosF1's pull request #146: Pin chameleon-skin version to 5.0.2 (03main...03RhinosF1-patch-1) 13https://github.com/miraheze/mediawiki-repos/pull/146 [19:23:12] [02mediawiki-repos] 07Reception123 pushed 1 new commit to 03main 13https://github.com/miraheze/mediawiki-repos/commit/fd2fc127483c423c5b75f890ed50bd3467eb5d71 [19:23:13] 02mediawiki-repos/03main 07RhinosF1 03fd2fc12 Pin chameleon-skin version to 5.0.2 (#146)… [19:23:14] And please for the love of god treat failed deploys as UBN [19:23:15] [02mediawiki-repos] 07Reception123 04deleted 03RhinosF1-patch-1 at 0370256ce 13https://api.github.com/repos/miraheze/mediawiki-repos/commit/70256ce [19:23:19] That was a bad issue [19:25:03] miraheze/CreateWiki - Reception123 the build has errored. [19:26:47] [1/3] ```Error: /Stage[main]/Mediawiki/Tidy[/tmp]: Failed to generate additional resources using 'generate': The directory '/tmp' contains 5348 entries, which exceeds the limit of 3000 specified by the max_files parameter for this resource. The limit may be increased, but be aware that large number of file resources can result in excessive resource consumption and degraded performance [19:26:47] [2/3] . Consider using an alternate method to manage large directory trees [19:26:47] [3/3] ``` I've never seen this before but I guess I should clear /tmp [19:27:16] /tmp is 9GB... [19:28:34] That's been happening quite a bit recently [19:28:40] And yes it's a concern [19:28:54] it used to happen a lot a long time ago [19:29:15] !log [wwr@mwtask181] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php MirahezeMagic:ResetWiki --wiki=loginwiki --dbname=scratchwiki --requester=MC mrjone (END - exit=0) [19:29:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:30:16] !log yes I did use quote marks just logsal doesn't like to notice that [19:30:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:30:36] !log [reception@mwtask181] starting deploy of {'world': True, 'versions': '1.45'} to all [19:30:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:31:06] @rhinosf1 should be fixed now, deploy is proceeding fine [19:33:32] !log [wwr@mwtask181] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php CreateWiki:SetContainersAccess --wiki=scratchwiki (END - exit=0) [19:33:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:33:40] Good [19:33:48] I should fix that one day [19:34:07] !log [reception@mwtask181] finished deploy of {'world': True, 'versions': '1.45'} to all - SUCCESS in 210s [19:34:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:36:32] [02CreateWiki] 07Reception123 pushed 1 new commit to 03Reception123-patch-1 13https://github.com/miraheze/CreateWiki/commit/a00ae61726a9ff8e67bf38e7e044972599f33f41 [19:36:32] 02CreateWiki/03Reception123-patch-1 07Reception123 03a00ae61 Update DeclineStaleWikiRequests.php [19:37:59] [02CreateWiki] 07github-actions[bot] pushed 1 new commit to 03Reception123-patch-1 13https://github.com/miraheze/CreateWiki/commit/a0dbe4f1fc776960e82fc383713bfe3395c60325 [19:37:59] 02CreateWiki/03Reception123-patch-1 07github-actions 03a0dbe4f CI: lint code to MediaWiki standards… [19:40:05] miraheze/CreateWiki - Reception123 the build has errored. [19:41:24] [02CreateWiki] 07Reception123 pushed 1 new commit to 03Reception123-patch-1 13https://github.com/miraheze/CreateWiki/commit/90005d222d6253160a6e396d49db59c99ee4800a [19:41:24] 02CreateWiki/03Reception123-patch-1 07Reception123 0390005d2 :( [19:45:26] miraheze/CreateWiki - Reception123 the build has errored. [20:26:23] PROBLEM - cp191 APT on cp191 is CRITICAL: APT CRITICAL: 96 packages available for upgrade (1 critical updates). [20:34:11] [02CreateWiki] 07pskyechology created 03skyetests (+1 new commit) 13https://github.com/miraheze/CreateWiki/commit/a1fd95176b3d [20:34:11] 02CreateWiki/03skyetests 07Skye 03a1fd951 Add test for abandoning a request [20:34:58] [02CreateWiki] 07pskyechology opened pull request #825: Add test for abandoning a request (03main...03skyetests) 13https://github.com/miraheze/CreateWiki/pull/825 [20:39:04] PROBLEM - cp201 APT on cp201 is CRITICAL: APT CRITICAL: 96 packages available for upgrade (1 critical updates). [20:39:38] PROBLEM - changeprop202 APT on changeprop202 is CRITICAL: APT CRITICAL: 94 packages available for upgrade (1 critical updates). [20:40:14] [02CreateWiki] 07codecov[bot] commented on pull request #825: :white_check_mark: All modified and coverable lines are covered by tests. […] 13https://github.com/miraheze/CreateWiki/pull/825#issuecomment-4522562241 [20:42:52] PROBLEM - cp171 APT on cp171 is CRITICAL: APT CRITICAL: 96 packages available for upgrade (1 critical updates). [20:46:37] [02CreateWiki] 07pskyechology pushed 1 new commit to 03skyetests 13https://github.com/miraheze/CreateWiki/commit/ce25571e933f402ad2a79b7d3a6d27085303f6b5 [20:46:37] 02CreateWiki/03skyetests 07Skye 03ce25571 Start query builder before getting [20:53:30] [02mw-config] 07Reception123 merged pull request #6417: change/clarify dormancy exemption reasons (03main...03Reception123-patch-1) 13https://github.com/miraheze/mw-config/pull/6417 [20:53:31] [02mw-config] 07Reception123 pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/667ca6ed5fa960229f1b87c841b57aff98d0b603 [20:53:31] 02mw-config/03main 07Reception123 03667ca6e change/clarify dormancy exemption reasons (#6417)… [20:53:32] [02mw-config] 07Reception123 04deleted 03Reception123-patch-1 at 034bdde75 13https://api.github.com/repos/miraheze/mw-config/commit/4bdde75 [20:53:43] !log [reception@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all [20:53:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:53:49] miraheze/CreateWiki - pskyechology the build has errored. [20:54:08] !log [reception@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 25s [20:54:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:54:45] miraheze/mw-config - Reception123 the build passed. [20:55:33] !log MariaDB [mhglobal]> UPDATE cw_wikis SET wiki_inactive_exempt_reason = 'comp' WHERE wiki_inactive_exempt_reason = 'mtr'; ('mtr' and 'comp' have been merged) [20:55:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:04:01] [02WikiDiscover] 07Reception123 pushed 1 new commit to 03Reception123-patch-2 13https://github.com/miraheze/WikiDiscover/commit/9648d0f7dad7daba7e2a14e77caae2a218aa1a47 [21:04:01] 02WikiDiscover/03Reception123-patch-2 07Reception123 039648d0f Update en.json [21:08:49] miraheze/WikiDiscover - Reception123 the build passed. [21:27:02] [02CreateWiki] 07The-Voidwalker pushed 1 new commit to 03Reception123-patch-1 13https://github.com/miraheze/CreateWiki/commit/142906cb175eb7cce6368faf11c085b3636c761a [21:27:02] 02CreateWiki/03Reception123-patch-1 07The-Voidwalker 03142906c shorten long lines [21:32:06] [02CreateWiki] 07codecov[bot] commented on pull request #822: :x: Patch coverage is `0%` with `69 lines` in your changes missing coverage. Please review. […] 13https://github.com/miraheze/CreateWiki/pull/822#issuecomment-4522835788 [21:32:34] PROBLEM - mw153 HTTPS on mw153 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [21:32:46] PROBLEM - mw202 MediaWiki Rendering on mw202 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.010 second response time [21:32:52] PROBLEM - mw202 HTTPS on mw202 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [21:32:58] PROBLEM - db161 Current Load on db161 is CRITICAL: LOAD CRITICAL - total load average: 130.41, 50.68, 19.92 [21:33:09] PROBLEM - cp191 HTTPS on cp191 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [21:33:11] PROBLEM - mw161 MediaWiki Rendering on mw161 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.017 second response time [21:33:19] PROBLEM - mw152 HTTPS on mw152 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [21:33:22] PROBLEM - mw192 MediaWiki Rendering on mw192 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.010 second response time [21:33:26] PROBLEM - mw192 HTTPS on mw192 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [21:33:26] PROBLEM - mw151 MediaWiki Rendering on mw151 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.010 second response time [21:33:27] PROBLEM - swiftobject161 APT on swiftobject161 is CRITICAL: APT CRITICAL: 103 packages available for upgrade (5 critical updates). [21:33:27] PROBLEM - cp201 HTTPS on cp201 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [21:33:28] PROBLEM - mw152 MediaWiki Rendering on mw152 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.012 second response time [21:33:29] PROBLEM - mw162 HTTPS on mw162 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [21:33:29] PROBLEM - mw182 HTTPS on mw182 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [21:33:31] PROBLEM - mw191 HTTPS on mw191 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [21:33:32] PROBLEM - mw191 MediaWiki Rendering on mw191 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.010 second response time [21:33:39] PROBLEM - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is CRITICAL: CRITICAL - NGINX Error Rate is 94% [21:33:39] PROBLEM - cp171 HTTPS on cp171 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [21:33:48] PROBLEM - test151 APT on test151 is CRITICAL: APT CRITICAL: 142 packages available for upgrade (5 critical updates). [21:33:51] PROBLEM - mw181 HTTPS on mw181 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [21:33:53] PROBLEM - cp201 Varnish Backends on cp201 is CRITICAL: 4 backends are down. mw172 mw163 mw173 mw201 [21:33:54] erm what the fuck [21:33:56] PROBLEM - mw162 MediaWiki Rendering on mw162 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.011 second response time [21:33:58] PROBLEM - mw153 MediaWiki Rendering on mw153 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.011 second response time [21:34:00] PROBLEM - mw183 MediaWiki Rendering on mw183 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.010 second response time [21:34:01] PROBLEM - cp191 Varnish Backends on cp191 is CRITICAL: 4 backends are down. mw172 mw163 mw173 mw201 [21:34:03] cc @Infrastructure Specialists [21:34:06] PROBLEM - mw193 MediaWiki Rendering on mw193 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.010 second response time [21:34:09] PROBLEM - mw161 HTTPS on mw161 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [21:34:11] PROBLEM - mw193 HTTPS on mw193 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [21:34:12] PROBLEM - mw182 MediaWiki Rendering on mw182 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.016 second response time [21:34:12] ugh [21:34:15] PROBLEM - swiftobject151 APT on swiftobject151 is CRITICAL: APT CRITICAL: 103 packages available for upgrade (5 critical updates). [21:34:15] PROBLEM - cp161 Varnish Backends on cp161 is CRITICAL: 4 backends are down. mw172 mw163 mw173 mw201 [21:34:16] PROBLEM - eventgate182 APT on eventgate182 is CRITICAL: APT CRITICAL: 98 packages available for upgrade (5 critical updates). [21:34:16] PROBLEM - mw203 HTTPS on mw203 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [21:34:23] PROBLEM - mw151 HTTPS on mw151 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [21:34:25] doing [21:34:27] PROBLEM - cp161 HTTPS on cp161 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [21:34:37] PROBLEM - mw201 MediaWiki Rendering on mw201 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:34:39] PROBLEM - mw203 MediaWiki Rendering on mw203 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.011 second response time [21:34:39] PROBLEM - cp171 Varnish Backends on cp171 is CRITICAL: 4 backends are down. mw172 mw163 mw173 mw201 [21:34:41] miraheze/CreateWiki - The-Voidwalker the build passed. [21:34:47] PROBLEM - mw201 HTTPS on mw201 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10006 milliseconds with 0 bytes received [21:34:56] cant look at if everything is down, ffs [21:34:57] oh i just saw its fcuking db161 again [21:35:00] yeah [21:35:03] PROBLEM - mw201 APT on mw201 is CRITICAL: APT CRITICAL: 139 packages available for upgrade (5 critical updates). [21:35:08] RECOVERY - cp191 HTTPS on cp191 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 1463 bytes in 0.074 second response time [21:35:11] PROBLEM - db192 APT on db192 is CRITICAL: APT CRITICAL: 116 packages available for upgrade (5 critical updates). [21:35:11] RECOVERY - mw161 MediaWiki Rendering on mw161 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.194 second response time [21:35:16] PROBLEM - mon181 APT on mon181 is CRITICAL: APT CRITICAL: 111 packages available for upgrade (5 critical updates). [21:35:17] RECOVERY - mw152 HTTPS on mw152 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.068 second response time [21:35:19] PROBLEM - mw152 APT on mw152 is CRITICAL: APT CRITICAL: 139 packages available for upgrade (5 critical updates). [21:35:20] RECOVERY - mw192 HTTPS on mw192 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.085 second response time [21:35:21] RECOVERY - mw192 MediaWiki Rendering on mw192 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.224 second response time [21:35:22] RECOVERY - mw151 MediaWiki Rendering on mw151 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.178 second response time [21:35:26] RECOVERY - mw182 HTTPS on mw182 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.106 second response time [21:35:29] !log [somerandomdeveloper@mwtask181] starting deploy of {'config': True, 'force': True} to all [21:35:33] around [21:35:39] RECOVERY - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is OK: OK - NGINX Error Rate is 39% [21:35:45] PROBLEM - mw183 APT on mw183 is CRITICAL: APT CRITICAL: 139 packages available for upgrade (5 critical updates). [21:35:47] !log [somerandomdeveloper@mwtask181] finished deploy of {'config': True, 'force': True} to all - SUCCESS in 17s [21:35:50] !log depool c2 [21:35:52] PROBLEM - mw172 MediaWiki Rendering on mw172 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:35:53] RECOVERY - cp201 Varnish Backends on cp201 is OK: All 31 backends are healthy [21:36:09] didn't get a proclist because db161 is unreachable [21:36:14] PROBLEM - cloud16 APT on cloud16 is CRITICAL: APT CRITICAL: 146 packages available for upgrade (5 critical updates). [21:36:17] PROBLEM - cloud20 APT on cloud20 is CRITICAL: APT CRITICAL: 152 packages available for upgrade (5 critical updates). [21:36:21] nvm it is reachable but very slow [21:36:23] PROBLEM - ldap171 APT on ldap171 is CRITICAL: APT CRITICAL: 98 packages available for upgrade (5 critical updates). [21:36:36] PROBLEM - bast181 APT on bast181 is CRITICAL: APT CRITICAL: 107 packages available for upgrade (5 critical updates). [21:36:38] PROBLEM - mw182 APT on mw182 is CRITICAL: APT CRITICAL: 139 packages available for upgrade (5 critical updates). [21:37:23] PROBLEM - swiftobject211 APT on swiftobject211 is CRITICAL: APT CRITICAL: 99 packages available for upgrade (5 critical updates). [21:37:27] [1/5] ``` [21:37:27] [2/5] [root@db161:/home/somerandomdeveloper]# uptime [21:37:27] [3/5] 21:37:15 up 19 days, 19:19, 3 users, load average: 392.42, 218.46, 95.46 [21:37:28] [4/5] ``` [21:37:28] [5/5] amazing [21:37:39] PROBLEM - prometheus151 APT on prometheus151 is CRITICAL: APT CRITICAL: 98 packages available for upgrade (5 critical updates). [21:38:15] at 416 now, it's depooled though?? [21:38:35] PROBLEM - cloud17 APT on cloud17 is CRITICAL: APT CRITICAL: 146 packages available for upgrade (5 critical updates). [21:38:50] RECOVERY - mw202 HTTPS on mw202 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 5.521 second response time [21:38:51] RECOVERY - mw202 MediaWiki Rendering on mw202 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 4.594 second response time [21:38:52] PROBLEM - mattermost2 APT on mattermost2 is CRITICAL: APT CRITICAL: 96 packages available for upgrade (4 critical updates). [21:39:16] PROBLEM - swiftproxy161 APT on swiftproxy161 is CRITICAL: APT CRITICAL: 103 packages available for upgrade (5 critical updates). [21:39:24] RECOVERY - cp201 HTTPS on cp201 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 1463 bytes in 0.081 second response time [21:39:25] RECOVERY - mw191 HTTPS on mw191 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.089 second response time [21:39:33] RECOVERY - mw152 MediaWiki Rendering on mw152 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.208 second response time [21:39:33] RECOVERY - mw191 MediaWiki Rendering on mw191 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.257 second response time [21:39:34] PROBLEM - matomo151 APT on matomo151 is CRITICAL: APT CRITICAL: 100 packages available for upgrade (5 critical updates). [21:39:34] RECOVERY - mw162 HTTPS on mw162 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.073 second response time [21:39:36] RECOVERY - cp171 HTTPS on cp171 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 1485 bytes in 0.080 second response time [21:39:39] PROBLEM - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is WARNING: WARNING - NGINX Error Rate is 43% [21:39:40] RECOVERY - mw172 MediaWiki Rendering on mw172 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.223 second response time [21:39:51] RECOVERY - mw153 MediaWiki Rendering on mw153 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.230 second response time [21:39:51] RECOVERY - mw181 HTTPS on mw181 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.090 second response time [21:39:56] RECOVERY - mw162 MediaWiki Rendering on mw162 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.226 second response time [21:39:57] RECOVERY - mw183 MediaWiki Rendering on mw183 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.244 second response time [21:39:57] PROBLEM - mem201 APT on mem201 is CRITICAL: APT CRITICAL: 98 packages available for upgrade (5 critical updates). [21:40:00] RECOVERY - cp191 Varnish Backends on cp191 is OK: All 31 backends are healthy [21:40:01] PROBLEM - db172 APT on db172 is CRITICAL: APT CRITICAL: 108 packages available for upgrade (5 critical updates). [21:40:01] RECOVERY - mw193 MediaWiki Rendering on mw193 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.246 second response time [21:40:04] RECOVERY - mw161 HTTPS on mw161 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.079 second response time [21:40:11] RECOVERY - mw193 HTTPS on mw193 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.083 second response time [21:40:12] RECOVERY - mw182 MediaWiki Rendering on mw182 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.250 second response time [21:40:12] RECOVERY - mw203 HTTPS on mw203 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.089 second response time [21:40:15] RECOVERY - cp161 Varnish Backends on cp161 is OK: All 31 backends are healthy [21:40:18] RECOVERY - mw151 HTTPS on mw151 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.074 second response time [21:40:22] RECOVERY - cp161 HTTPS on cp161 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 1518 bytes in 0.083 second response time [21:40:23]