[00:05:50] RECOVERY - cp38 Disk Space on cp38 is OK: DISK OK - free space: / 59013MiB (12% inode=99%); [00:25:37] !log [blankeclair@mwtask171] runRefreshLinks: [0/2756] Handling creditunionswiki [00:25:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:26:17] [02puppet] 07BlankEclair opened pull request #4274: T13404; Do not redirect for /w/index.php?action=... (except view) (07miraheze:03master...07BlankEclair:03T13404) 13https://github.com/miraheze/puppet/pull/4274 [01:26:23] [02puppet] 07coderabbitai[bot] commented on pull request #4274: --- […] 13https://github.com/miraheze/puppet/pull/4274#issuecomment-2745956045 [01:26:50] [02puppet] 07BlankEclair retitled PR #4274: "T13404; Do not redirect for /w/index.php?action=... (except view)" ➜ "T13404: Do not redirect for /w/index.php?action=... (except view)" 13https://github.com/miraheze/puppet/pull/4274 [01:28:15] [02MirahezeMagic] 07BlankEclair left a file comment in pull request #559 03dd280ed: ```suggestion […] 13https://github.com/miraheze/MirahezeMagic/pull/559#discussion_r2008987884 [01:29:00] [02MirahezeMagic] 07Universal-Omega pushed 1 new commit to 03generate_sitemap_index 13https://github.com/miraheze/MirahezeMagic/commit/6c4ee5c9328ecaf783a8eec08f3a7c518ab2f3a8 [01:29:00] 02MirahezeMagic/03generate_sitemap_index 07CosmicAlpha 036c4ee5c Apply suggestion from code review… [01:29:06] [02ManageWiki] 07BlankEclair approved pull request #539 13https://github.com/miraheze/ManageWiki/pull/539#pullrequestreview-2708416653 [01:30:46] [02CreateWiki] 07BlankEclair pushed 1 new commit to 03T13396 13https://github.com/miraheze/CreateWiki/commit/930257e1d913171b0f80c93186efb57a4dddf68b [01:30:46] 02CreateWiki/03T13396 07BlankEclair 03930257e Use CreateWikiValidator::getValidUrl() [01:30:54] [02puppet] 07AgentIsai commented on pull request #4274: There're [quite a few action parameters to index.php](https://www.mediawiki.org/wiki/Manual:Parameters_to_index.php#Actions). Are we sure we want to stop redirecting all of them? 13https://github.com/miraheze/puppet/pull/4274#issuecomment-2745957301 [01:31:14] [02puppet] 07Universal-Omega commented on pull request #4274: We want some actions like edit to redirect though to normalize URLs otherwise we may as well remove the whole thing as that was the entire point of it. 13https://github.com/miraheze/puppet/pull/4274#issuecomment-2745957405 [01:37:55] [02mw-config] 07BlankEclair left a file comment in pull request #5827 03f36d7a6: ```suggestion […] 13https://github.com/miraheze/mw-config/pull/5827#discussion_r2008988386 [01:37:55] [02mw-config] 07BlankEclair left a file comment in pull request #5827 03f36d7a6: ```suggestion […] 13https://github.com/miraheze/mw-config/pull/5827#discussion_r2008988543 [01:37:55] [02mw-config] 07BlankEclair left a file comment in pull request #5827 03f36d7a6: ```suggestion […] 13https://github.com/miraheze/mw-config/pull/5827#discussion_r2008988743 [01:37:57] [02mw-config] 07BlankEclair left a file comment in pull request #5827 03f36d7a6: ```suggestion […] 13https://github.com/miraheze/mw-config/pull/5827#discussion_r2008988457 [01:37:59] [02mw-config] 07BlankEclair left a file comment in pull request #5827 03f36d7a6: ```suggestion […] 13https://github.com/miraheze/mw-config/pull/5827#discussion_r2008988322 [01:39:24] [02MirahezeMagic] 07BlankEclair approved pull request #559 13https://github.com/miraheze/MirahezeMagic/pull/559#pullrequestreview-2708417685 [01:40:36] [02puppet] 07BlankEclair commented on pull request #4274: Fair 13https://github.com/miraheze/puppet/pull/4274#issuecomment-2745960293 [01:41:09] miraheze/MirahezeMagic - Universal-Omega the build passed. [01:41:49] [02puppet] 07BlankEclair retitled PR #4274: "T13404: Do not redirect for /w/index.php?action=... (except view)" ➜ "T13404: Do not redirect for /w/index.php?action=render" 13https://github.com/miraheze/puppet/pull/4274 [01:42:30] miraheze/CreateWiki - BlankEclair the build passed. [01:43:27] [02puppet] 07Universal-Omega approved pull request #4274 13https://github.com/miraheze/puppet/pull/4274#pullrequestreview-2708418197 [01:44:58] [02puppet] 07Universal-Omega merged 07BlankEclair's pull request #4274: T13404: Do not redirect for /w/index.php?action=render (07miraheze:03master...07BlankEclair:03T13404) 13https://github.com/miraheze/puppet/pull/4274 [01:44:58] [02puppet] 07Universal-Omega pushed 1 new commit to 03master 13https://github.com/miraheze/puppet/commit/fe33d7b77e6985685ce7b20109ba32464bc78af2 [01:44:58] 02puppet/03master 07Claire 03fe33d7b T13404: Do not redirect for /w/index.php?action=render (#4274)… [01:49:31] [02ssl] 07MacFan4000 pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/0fbabbf322304de928e1936c94f1d55fd7af1db2 [01:49:32] 02ssl/03master 07MacFan4000 030fbabbf add wiki.firetime.me [01:50:05] [02ssl] 07MacFan4000 pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/c515bc75bb05a9e206bb29e725a048bd2472cca8 [01:50:05] 02ssl/03master 07MacFan4000 03c515bc7 Update certs.yaml [01:53:52] PROBLEM - mw201 Puppet on mw201 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [01:54:23] PROBLEM - mw191 Puppet on mw191 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [01:54:33] @blankeclair ^ [01:54:39] Seems we broke nginx [01:55:47] [1/5] ``` [01:55:47] [2/5] 2025/03/23 01:55:29 [emerg] 1089224#1089224: invalid condition "$arg_action" [01:55:47] [3/5] in /etc/nginx/mediawiki-includes:57 [01:55:48] [4/5] nginx: configuration file /etc/nginx/nginx.conf test failed [01:55:48] [5/5] ``` [01:56:23] PROBLEM - test151 Puppet on test151 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [01:56:48] PROBLEM - mw152 Puppet on mw152 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [01:57:37] PROBLEM - mw181 Puppet on mw181 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [01:57:45] [02puppet] 07Universal-Omega pushed 1 new commit to 03master 13https://github.com/miraheze/puppet/commit/e87e859f8e6805aae5f7772e8e10b52097cf4f62 [01:57:45] 02puppet/03master 07CosmicAlpha 03e87e859 Fix nginx [01:58:28] !log [blankeclair@mwtask171] runRefreshLinks: [100/2756] Handling derfanwiki [01:58:30] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:59:29] Nice. [02:00:48] Yeah fixed it. I'll do a better fix in a bit just did a very quick one for now. [02:01:13] PROBLEM - mw202 Puppet on mw202 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [02:01:20] PROBLEM - mw182 Puppet on mw182 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [02:01:52] RECOVERY - test151 Puppet on test151 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:17:41] cosmicalpha: ope i made a typo :3 [02:20:16] PROBLEM - edapedia.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'edapedia.net' expires in 14 day(s) (Mon 07 Apr 2025 01:52:00 AM GMT +0000). [02:20:27] [02ssl] 07WikiTideSSLBot pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/1590c661e12b7c58f30c001c8576b18a2d7d6937 [02:20:27] 02ssl/03master 07WikiTideSSLBot 031590c66 Bot: Update SSL cert for edapedia.net [02:21:52] RECOVERY - mw201 Puppet on mw201 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:22:18] RECOVERY - mw191 Puppet on mw191 is OK: OK: Puppet is currently enabled, last run 51 seconds ago with 0 failures [02:24:44] RECOVERY - mw152 Puppet on mw152 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:25:31] RECOVERY - mw181 Puppet on mw181 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:27:59] RECOVERY - mw182 Puppet on mw182 is OK: OK: Puppet is currently enabled, last run 31 seconds ago with 0 failures [02:29:28] RECOVERY - mw202 Puppet on mw202 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:48:53] PROBLEM - corru.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'corru.wiki' expires in 14 day(s) (Mon 07 Apr 2025 02:27:02 AM GMT +0000). [02:49:38] [02ssl] 07WikiTideSSLBot pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/ac77fe7a2e107a513c32eda9dae2de76b6b1d32c [02:49:38] 02ssl/03master 07WikiTideSSLBot 03ac77fe7 Bot: Update SSL cert for corru.wiki [02:50:02] RECOVERY - edapedia.net - LetsEncrypt on sslhost is OK: OK - Certificate 'edapedia.net' will expire on Sat 21 Jun 2025 01:21:51 AM GMT +0000. [03:07:46] [02mediawiki-repos] 07BlankEclair opened pull request #64: Add patch from https://gerrit.wikimedia.org/r/1121572 (07miraheze:03master...07BlankEclair:03pagenotice-global-notices) 13https://github.com/miraheze/mediawiki-repos/pull/64 [03:07:58] [02mediawiki-repos] 07coderabbitai[bot] commented on pull request #64: --- […] 13https://github.com/miraheze/mediawiki-repos/pull/64#issuecomment-2745987894 [03:18:25] [02puppet] 07BlankEclair left a file comment in pull request #3885 03938e1e4: Do we have to do it in a list comprehension instead of the for loop? And did you mean startswith()? […] 13https://github.com/miraheze/puppet/pull/3885#discussion_r2008999868 [03:18:25] [02puppet] 07BlankEclair left a file comment in pull request #3885 03938e1e4: ```suggestion […] 13https://github.com/miraheze/puppet/pull/3885#discussion_r2008999893 [03:48:21] RECOVERY - corru.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'corru.wiki' will expire on Sat 21 Jun 2025 01:50:29 AM GMT +0000. [07:13:19] PROBLEM - matomo151 Current Load on matomo151 is CRITICAL: LOAD CRITICAL - total load average: 8.18, 7.29, 6.11 [07:15:17] PROBLEM - matomo151 Current Load on matomo151 is WARNING: LOAD WARNING - total load average: 7.57, 7.15, 6.19 [07:17:16] PROBLEM - matomo151 Current Load on matomo151 is CRITICAL: LOAD CRITICAL - total load average: 9.15, 7.85, 6.56 [07:25:09] PROBLEM - matomo151 Current Load on matomo151 is WARNING: LOAD WARNING - total load average: 7.35, 7.68, 6.98 [07:31:04] PROBLEM - matomo151 Current Load on matomo151 is CRITICAL: LOAD CRITICAL - total load average: 8.43, 7.78, 7.22 [07:36:59] PROBLEM - matomo151 Current Load on matomo151 is WARNING: LOAD WARNING - total load average: 5.60, 7.06, 7.11 [07:38:57] PROBLEM - matomo151 Current Load on matomo151 is CRITICAL: LOAD CRITICAL - total load average: 9.07, 7.73, 7.34 [07:39:51] miraheze/mw-config - Aeywoo the build passed. [07:40:55] PROBLEM - matomo151 Current Load on matomo151 is WARNING: LOAD WARNING - total load average: 6.78, 7.52, 7.32 [07:48:52] RECOVERY - matomo151 Current Load on matomo151 is OK: LOAD OK - total load average: 5.44, 6.20, 6.76 [07:56:52] PROBLEM - matomo151 Current Load on matomo151 is WARNING: LOAD WARNING - total load average: 6.10, 6.81, 6.79 [08:00:52] RECOVERY - matomo151 Current Load on matomo151 is OK: LOAD OK - total load average: 5.70, 6.46, 6.69 [08:02:29] [02mediawiki-repos] 07Aeywoo opened pull request #65: Uninstall RefreshSpecial (07miraheze:03master...07Aeywoo:03patch-5) 13https://github.com/miraheze/mediawiki-repos/pull/65 [08:02:35] [02mediawiki-repos] 07coderabbitai[bot] commented on pull request #65: --- […] 13https://github.com/miraheze/mediawiki-repos/pull/65#issuecomment-2746079522 [08:03:34] [02mediawiki-repos] 07Universal-Omega merged 07Aeywoo's pull request #65: Uninstall RefreshSpecial (07miraheze:03master...07Aeywoo:03patch-5) 13https://github.com/miraheze/mediawiki-repos/pull/65 [08:03:34] [02mediawiki-repos] 07Universal-Omega pushed 1 new commit to 03master 13https://github.com/miraheze/mediawiki-repos/commit/1d7997660644da243dd596f4af9bb59852410a59 [08:03:36] 02mediawiki-repos/03master 07Aeywoo 031d79976 Uninstall RefreshSpecial (#65)… [08:04:53] miraheze/mw-config - Aeywoo the build passed. [08:05:51] [02mw-config] 07Aeywoo opened pull request #5881: Uninstall RefreshSpecial (07miraheze:03master...07Aeywoo:03patch-4) 13https://github.com/miraheze/mw-config/pull/5881 [08:06:39] miraheze/mw-config - Aeywoo the build passed. [08:31:01] [02mediawiki-repos] 07Aeywoo opened pull request #66: Uninstall TwitterTag per T13405 (07miraheze:03master...07Aeywoo:03patch-6) 13https://github.com/miraheze/mediawiki-repos/pull/66 [08:31:07] [02mediawiki-repos] 07coderabbitai[bot] commented on pull request #66: --- […] 13https://github.com/miraheze/mediawiki-repos/pull/66#issuecomment-2746089884 [08:53:12] !log [rhinos@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php refreshLinks --wiki=bluepageswiki (END - exit=35072) [08:53:13] !log [rhinos@mwtask181] rhinosf1: bluepageswiki done [08:53:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [08:53:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [08:53:33] Fuck [08:53:50] @blankeclair you wanna deal with ^ [08:54:36] wait what [08:54:41] that exit code [08:55:54] @blankeclair ye I'm not sure if that'll ever rebuild [08:56:01] But you can take a look [08:56:24] can we put it in as a task so i can look at it later? [08:57:36] @blankeclair I split it off from your mass refresh [08:57:39] But if you want [08:57:58] we'll need to make a version of runRefreshLinks that does it in segments :p [08:58:04] Yup [08:58:17] https://issue-tracker.miraheze.org/T13406 [09:33:23] I guess while I'll wait for TwitterTag to be uninstalled, I'll read the MW 1.44 update logs. [10:16:24] [02mediawiki-repos] 07Aeywoo opened pull request #67: Updating Skin:Citizen branch (07miraheze:03master...07Aeywoo:03patch-1) 13https://github.com/miraheze/mediawiki-repos/pull/67 [10:16:31] [02mediawiki-repos] 07coderabbitai[bot] commented on pull request #67: --- […] 13https://github.com/miraheze/mediawiki-repos/pull/67#issuecomment-2746130794 [10:16:41] @blankeclair I accidentally made a booboo with one of my earlier PRs to mediawiki-repos. [10:16:46] meow? [10:17:07] When I did the PR to install Skin:Lakeus it reverted the branch change for Skin:Citizen from main to REL1_39, can you quickly approve this PR to fix it please? https://github.com/miraheze/mediawiki-repos/pull/67 [10:17:07] [GitHub] [miraheze/mediawiki-repos #67] open PR by Aeywoo, created 45 seconds ago: Updating Skin:Citizen branch | Updating the Citizen skin branch back to the main branch, I accidentally reverted it with https://github.com/miraheze/mediawiki-repos/pull/57 when installing Skin:Lakeus. [10:17:37] [02mediawiki-repos] 07BlankEclair merged 07Aeywoo's pull request #67: Updating Skin:Citizen branch (07miraheze:03master...07Aeywoo:03patch-1) 13https://github.com/miraheze/mediawiki-repos/pull/67 [10:17:37] [02mediawiki-repos] 07BlankEclair pushed 1 new commit to 03master 13https://github.com/miraheze/mediawiki-repos/commit/497d2f3c8771104289f0b5215942b7d9c797eb83 [10:17:38] 02mediawiki-repos/03master 07Aeywoo 03497d2f3 Updating Skin:Citizen branch (#67)… [10:17:41] :3 [10:17:49] Tank chu :3 [10:18:01] lol you beat me @blankeclair whoops didn't catch that originally [10:18:26] Also it's almost 4:30AM why am I up lol [10:18:28] I didn't even notice it until I went to edit the file again and I'm like, wtf I thought I fixed that. [10:18:28] https://doc.wikimedia.org/mediawiki-core/master/php/interfaceMediaWiki_1_1JobQueue_1_1IJobSpecification.html#a4d1d6eb5c3ecb345d3061ea03625c24e [10:18:38] is this how to make a job only essentially appear once in the queue? [10:19:05] No you do that in changeprop [10:19:13] I think. [10:19:33] I'm pretty sure changeprop doesn't honor that but I suppose it might. [10:19:54] > Since job runner implementations may vary in reliability, job classes should be idempotent, to maintain correctness even if the job happens to run more than once. [10:19:56] hmm [10:19:58] found on https://doc.wikimedia.org/mediawiki-core/master/php/jobqueuearch.html [10:19:59] Well I should feep [10:20:02] eeep [10:20:04] sleep [10:20:09] eepy cosmy =w= [10:20:09] lol 3 times [10:20:34] lol [10:34:41] PROBLEM - mwtask181 Puppet on mwtask181 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_checkout_MediaWiki-REL1_43 Citizen] [10:46:12] !log [reception@mwtask181] sudo -u www-data /usr/local/bin/foreachwikiindblist /tmp/twittertag.php /srv/mediawiki/1.43/maintenance/run.php /srv/mediawiki/1.43/extensions/ManageWiki/maintenance/toggleExtension.php twittertag --disable (START) [10:46:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:46:19] !log [reception@mwtask181] sudo -u www-data /usr/local/bin/foreachwikiindblist /tmp/twittertag.php /srv/mediawiki/1.43/maintenance/run.php /srv/mediawiki/1.43/extensions/ManageWiki/maintenance/toggleExtension.php twittertag --disable (END - exit=2) [10:46:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:47:03] !log [reception@mwtask181] sudo -u www-data /usr/local/bin/foreachwikiindblist /tmp/twittertag.php /srv/mediawiki/1.43/maintenance/run.php /srv/mediawiki/1.43/extensions/ManageWiki/maintenance/ToggleExtension.php twittertag --disable (START) [10:47:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:48:39] !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php /srv/mediawiki/1.43/maintenance/refreshLinks.php --wiki=bluepageswiki (START) [10:48:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:56:23] !log [reception@mwtask181] sudo -u www-data /usr/local/bin/foreachwikiindblist /tmp/twittertag.php /srv/mediawiki/1.43/maintenance/run.php /srv/mediawiki/1.43/extensions/ManageWiki/maintenance/ToggleExtension.php twittertag --disable (END - exit=0) [10:56:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:56:46] PROBLEM - test151 Puppet on test151 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 2 minutes ago with 2 failures. Failed resources (up to 3 shown): Exec[git_checkout_MediaWiki-REL1_43 Citizen],Exec[git_checkout_MediaWiki-master Citizen] [10:56:50] [02mediawiki-repos] 07Reception123 commented on pull request #66: @Aeywoo There also needs to be a PR to remove it from ManageWiki (https://github.com/miraheze/mw-config/blob/master/ManageWikiExtensions.php#L1088) before we can remove it here. 13https://github.com/miraheze/mediawiki-repos/pull/66#issuecomment-2746145384 [10:57:26] [02mw-config] 07Reception123 commented on pull request #5881: What's wrong with RefreshSpecial? 13https://github.com/miraheze/mw-config/pull/5881#issuecomment-2746145573 [11:12:29] !log [blankeclair@mwtask161] runRefreshLinks: [300/3227] Handling bysonlinewiki [11:12:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:37:21] !log [blankeclair@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all [12:37:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:37:35] miraheze/mw-config - BlankEclair the build passed. [12:37:44] !log [blankeclair@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 23s [12:37:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:38:10] miraheze/mw-config - BlankEclair the build passed. [13:39:33] PROBLEM - lostmediawiki.ru - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'lostmediawiki.ru' expires in 14 day(s) (Mon 07 Apr 2025 01:25:09 PM GMT +0000). [13:58:36] PROBLEM - mw182 APT on mw182 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [13:58:39] PROBLEM - reports171 APT on reports171 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [13:59:30] Oh fuckety fuck [14:00:34] PROBLEM - swiftobject181 APT on swiftobject181 is CRITICAL: APT CRITICAL: 34 packages available for upgrade (1 critical updates). [14:00:59] PROBLEM - mw202 APT on mw202 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [14:01:25] PROBLEM - mw193 APT on mw193 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [14:01:39] PROBLEM - puppet181 APT on puppet181 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [14:02:11] PROBLEM - cloud16 APT on cloud16 is CRITICAL: APT CRITICAL: 41 packages available for upgrade (2 critical updates). [14:03:03] PROBLEM - swiftproxy161 APT on swiftproxy161 is CRITICAL: APT CRITICAL: 35 packages available for upgrade (1 critical updates). [14:03:36] PROBLEM - mwtask181 APT on mwtask181 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [14:04:03] PROBLEM - mw163 APT on mw163 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [14:05:11] PROBLEM - cloud19 APT on cloud19 is CRITICAL: APT CRITICAL: 41 packages available for upgrade (2 critical updates). [14:05:48] PROBLEM - swiftproxy171 APT on swiftproxy171 is CRITICAL: APT CRITICAL: 36 packages available for upgrade (1 critical updates). [14:06:42] PROBLEM - mw151 APT on mw151 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [14:06:46] PROBLEM - cloud15 APT on cloud15 is CRITICAL: APT CRITICAL: 41 packages available for upgrade (2 critical updates). [14:06:51] alright icinga [14:06:53] shut it [14:06:57] i'm looking [14:09:05] !log upgrading security packages on all relevant servers [14:09:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:09:23] !log [rhinos@puppet181] Upgraded packages on cloud19: libxslt1.1 and xsltproc [14:09:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:09:30] PROBLEM - mwtask171 APT on mwtask171 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [14:09:56] !log [rhinos@puppet181] Upgraded packages on cloud15: libxslt1.1 and xsltproc [14:09:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:10:17] !log [rhinos@puppet181] Upgraded packages on cloud16: libxslt1.1 and xsltproc [14:10:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:10:30] PROBLEM - phorge171 APT on phorge171 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [14:10:32] PROBLEM - mw162 APT on mw162 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [14:10:48] PROBLEM - mw161 APT on mw161 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [14:10:49] RECOVERY - cloud15 APT on cloud15 is OK: APT OK: 39 packages available for upgrade (0 critical updates). [14:10:53] PROBLEM - mw192 APT on mw192 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [14:11:03] !log [rhinos@puppet181] Upgraded packages on cloud18: libxslt1.1 and xsltproc [14:11:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:11:11] RECOVERY - cloud19 APT on cloud19 is OK: APT OK: 39 packages available for upgrade (0 critical updates). [14:11:56] !log [rhinos@puppet181] Upgraded packages on mw151: libxslt1.1 [14:11:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:12:13] RECOVERY - cloud16 APT on cloud16 is OK: APT OK: 39 packages available for upgrade (0 critical updates). [14:12:30] !log [rhinos@puppet181] Upgraded packages on mw163: libxslt1.1 [14:12:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:12:38] PROBLEM - swiftobject191 APT on swiftobject191 is CRITICAL: APT CRITICAL: 34 packages available for upgrade (1 critical updates). [14:12:41] RECOVERY - mw151 APT on mw151 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:12:43] !log [rhinos@puppet181] Upgraded packages on mw173: libxslt1.1 [14:12:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:13:11] !log [rhinos@puppet181] Upgraded packages on mw161: libxslt1.1 [14:13:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:13:16] PROBLEM - mw203 APT on mw203 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [14:13:24] !log [rhinos@puppet181] Upgraded packages on mw182: libxslt1.1 [14:13:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:13:41] !log [rhinos@puppet181] Upgraded packages on mw202: libxslt1.1 [14:13:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:13:56] PROBLEM - mattermost1 APT on mattermost1 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [14:13:59] !log [rhinos@puppet181] Upgraded packages on mw193: libxslt1.1 [14:14:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:14:02] RECOVERY - mw163 APT on mw163 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:14:11] !log [rhinos@puppet181] Upgraded packages on mw203: libxslt1.1 [14:14:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:14:35] RECOVERY - mw182 APT on mw182 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:14:35] !log [rhinos@puppet181] Upgraded packages on mw192: libxslt1.1 [14:14:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:14:47] RECOVERY - mw161 APT on mw161 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:14:48] !log [rhinos@puppet181] Upgraded packages on mw162: libxslt1.1 [14:14:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:14:52] RECOVERY - mw192 APT on mw192 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:14:58] RECOVERY - mw202 APT on mw202 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:15:00] !log [rhinos@puppet181] Upgraded packages on mwtask171: libxslt1.1 [14:15:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:15:04] PROBLEM - swiftobject171 APT on swiftobject171 is CRITICAL: APT CRITICAL: 33 packages available for upgrade (1 critical updates). [14:15:16] RECOVERY - mw203 APT on mw203 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:15:22] !log [rhinos@puppet181] Upgraded packages on phorge171: libxslt1.1 [14:15:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:15:24] RECOVERY - mw193 APT on mw193 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:15:29] RECOVERY - mwtask171 APT on mwtask171 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:15:34] PROBLEM - beidipedia.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'beidipedia.com' expires in 14 day(s) (Mon 07 Apr 2025 02:07:29 PM GMT +0000). [14:15:34] PROBLEM - otherkin.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'otherkin.wiki' expires in 14 day(s) (Mon 07 Apr 2025 01:52:12 PM GMT +0000). [14:15:36] !log [rhinos@puppet181] Upgraded packages on mwtask181: libxslt1.1 [14:15:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:15:45] PROBLEM - mon181 APT on mon181 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [14:15:53] !log [rhinos@puppet181] Upgraded packages on reports171: libxslt1.1 [14:15:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:16:07] PROBLEM - mw171 APT on mw171 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [14:16:14] PROBLEM - mw153 APT on mw153 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [14:16:22] !log [rhinos@puppet181] Upgraded packages on puppet181: libxslt1.1 [14:16:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:16:28] RECOVERY - phorge171 APT on phorge171 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:16:31] RECOVERY - mw162 APT on mw162 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:16:38] RECOVERY - reports171 APT on reports171 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:16:50] !log [rhinos@puppet181] Upgraded packages on swiftproxy161: libxslt1.1 [14:16:51] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:17:02] !log [rhinos@puppet181] Upgraded packages on swiftobject171: libxslt1.1 [14:17:03] RECOVERY - swiftproxy161 APT on swiftproxy161 is OK: APT OK: 34 packages available for upgrade (0 critical updates). [14:17:03] RECOVERY - swiftobject171 APT on swiftobject171 is OK: APT OK: 32 packages available for upgrade (0 critical updates). [14:17:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:17:24] !log [rhinos@puppet181] Upgraded packages on swiftproxy171: libxslt1.1 [14:17:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:17:35] RECOVERY - mwtask181 APT on mwtask181 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:17:38] RECOVERY - puppet181 APT on puppet181 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:17:44] PROBLEM - cloud17 APT on cloud17 is CRITICAL: APT CRITICAL: 41 packages available for upgrade (2 critical updates). [14:17:45] !log [rhinos@puppet181] Upgraded packages on swiftobject151: libxslt1.1 [14:17:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:17:48] RECOVERY - swiftproxy171 APT on swiftproxy171 is OK: APT OK: 35 packages available for upgrade (0 critical updates). [14:17:53] PROBLEM - mw172 APT on mw172 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [14:17:58] !log [rhinos@puppet181] Upgraded packages on swiftobject191: libxslt1.1 [14:17:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:18:19] !log [rhinos@puppet181] Upgraded packages on swiftobject181: libxslt1.1 [14:18:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:18:38] RECOVERY - swiftobject191 APT on swiftobject191 is OK: APT OK: 33 packages available for upgrade (0 critical updates). [14:18:42] RECOVERY - swiftobject181 APT on swiftobject181 is OK: APT OK: 33 packages available for upgrade (0 critical updates). [14:19:51] PROBLEM - mwtask161 APT on mwtask161 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [14:20:18] PROBLEM - mw183 APT on mw183 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [14:20:49] PROBLEM - mwtask151 APT on mwtask151 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [14:21:54] !log [rhinos@puppet181] Upgraded packages on cloud20: libxslt1.1 and xsltproc [14:21:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:22:32] !log [rhinos@puppet181] Upgraded packages on cloud17: libxslt1.1 and xsltproc [14:22:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:23:08] !log [rhinos@puppet181] Upgraded packages on mattermost1: libxslt1.1 [14:23:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:23:37] !log [rhinos@puppet181] Upgraded packages on mon181: libxslt1.1 [14:23:39] PROBLEM - mw201 APT on mw201 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [14:23:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:23:44] RECOVERY - mon181 APT on mon181 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:23:44] RECOVERY - cloud17 APT on cloud17 is OK: APT OK: 39 packages available for upgrade (0 critical updates). [14:23:54] !log [rhinos@puppet181] Upgraded packages on mw152: libxslt1.1 [14:23:56] RECOVERY - mattermost1 APT on mattermost1 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:23:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:24:09] PROBLEM - mw191 APT on mw191 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [14:24:16] !log [rhinos@puppet181] Upgraded packages on mw171: libxslt1.1 [14:24:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:24:33] !log [rhinos@puppet181] Upgraded packages on mw153: libxslt1.1 [14:24:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:24:50] !log [rhinos@puppet181] Upgraded packages on mw172: libxslt1.1 [14:24:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:24:55] PROBLEM - test151 APT on test151 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [14:25:07] !log [rhinos@puppet181] Upgraded packages on mw191: libxslt1.1 [14:25:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:25:19] !log [rhinos@puppet181] Upgraded packages on mw201: libxslt1.1 [14:25:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:25:35] PROBLEM - mw181 APT on mw181 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [14:25:38] RECOVERY - mw201 APT on mw201 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:25:52] RECOVERY - mw172 APT on mw172 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:25:53] !log [rhinos@puppet181] Upgraded packages on mw181: libxslt1.1 [14:25:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:26:02] PROBLEM - swiftobject201 APT on swiftobject201 is CRITICAL: APT CRITICAL: 34 packages available for upgrade (1 critical updates). [14:26:05] !log [rhinos@puppet181] Upgraded packages on mwtask151: libxslt1.1 [14:26:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:26:07] RECOVERY - mw171 APT on mw171 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:26:08] RECOVERY - mw191 APT on mw191 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:26:13] RECOVERY - mw153 APT on mw153 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:26:17] !log [rhinos@puppet181] Upgraded packages on mwtask161: libxslt1.1 [14:26:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:26:24] PROBLEM - bots171 APT on bots171 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [14:26:30] !log [rhinos@puppet181] Upgraded packages on mw183: libxslt1.1 [14:26:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:26:49] RECOVERY - mwtask151 APT on mwtask151 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:27:03] PROBLEM - swiftobject161 APT on swiftobject161 is CRITICAL: APT CRITICAL: 33 packages available for upgrade (1 critical updates). [14:27:34] RECOVERY - mw181 APT on mw181 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:27:47] PROBLEM - matomo151 APT on matomo151 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [14:27:48] !log [rhinos@puppet181] Upgraded packages on test151: libxslt1.1 [14:27:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:27:50] RECOVERY - mwtask161 APT on mwtask161 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:27:50] PROBLEM - cp38 Disk Space on cp38 is WARNING: DISK WARNING - free space: / 49940MiB (10% inode=99%); [14:28:08] !log [rhinos@puppet181] Upgraded packages on swiftac171: libxslt1.1 [14:28:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:28:18] RECOVERY - mw183 APT on mw183 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:28:21] !log [rhinos@puppet181] Upgraded packages on swiftobject201: libxslt1.1 [14:28:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:28:42] !log [rhinos@puppet181] Upgraded packages on swiftobject161: libxslt1.1 [14:28:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:28:56] RECOVERY - test151 APT on test151 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:29:03] RECOVERY - swiftobject161 APT on swiftobject161 is OK: APT OK: 32 packages available for upgrade (0 critical updates). [14:30:02] RECOVERY - swiftobject201 APT on swiftobject201 is OK: APT OK: 33 packages available for upgrade (0 critical updates). [14:30:10] RECOVERY - mwtask181 Puppet on mwtask181 is OK: OK: Puppet is currently enabled, last run 33 seconds ago with 0 failures [14:30:45] !log [blankeclair@mwtask161] runRefreshLinks: [400/3227] Handling clusterbrainswiki [14:30:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:31:06] !log [rhinos@puppet181] Upgraded packages on bots171: libxslt1.1 [14:31:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:31:47] !log [rhinos@puppet181] Upgraded packages on matomo151: libxslt1.1 [14:31:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:31:51] RECOVERY - bots171 APT on bots171 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:32:08] RECOVERY - matomo151 APT on matomo151 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:37:33] RECOVERY - lostmediawiki.ru - LetsEncrypt on sslhost is OK: OK - Certificate 'lostmediawiki.ru' will expire on Sat 21 Jun 2025 12:41:09 PM GMT +0000. [14:44:07] RECOVERY - beidipedia.com - LetsEncrypt on sslhost is OK: OK - Certificate 'beidipedia.com' will expire on Sat 21 Jun 2025 01:17:19 PM GMT +0000. [14:44:07] RECOVERY - otherkin.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'otherkin.wiki' will expire on Sat 21 Jun 2025 01:17:29 PM GMT +0000. [14:54:46] RECOVERY - test151 Puppet on test151 is OK: OK: Puppet is currently enabled, last run 42 seconds ago with 0 failures [15:02:10] https://cdn.discordapp.com/attachments/808001911868489748/1353383329675874418/20250121_130859.jpg?ex=67e173f2&is=67e02272&hm=3544ca70f205aba218dd34ce25d90dffd298bf43088d957af7c178d235277987& [15:06:19] [02dns] 07MacFan4000 pushed 1 new commit to 03master 13https://github.com/miraheze/dns/commit/5bc3530e460b96811c2c8bd4582f0577007157b0 [15:06:19] 02dns/03master 07MacFan4000 035bc3530 update [15:07:13] miraheze/dns - MacFan4000 the build passed. [15:34:59] [02dns] 07paladox created 03paladox-patch-1 (+1 new commit) 13https://github.com/miraheze/dns/commit/dfe4751f74ad [15:34:59] 02dns/03paladox-patch-1 07paladox 03dfe4751 Add healthcheck for appservers [15:35:03] [02dns] 07paladox opened pull request #660: Add healthcheck for appservers (03master...03paladox-patch-1) 13https://github.com/miraheze/dns/pull/660 [15:36:02] miraheze/dns - paladox the build has errored. [15:37:02] [02puppet] 07paladox created 03paladox-patch-1 (+1 new commit) 13https://github.com/miraheze/puppet/commit/6004c07c4d8b [15:37:02] 02puppet/03paladox-patch-1 07paladox 036004c07 mediawiki: allow name server to call healthcheck over port 80 [15:37:06] [02puppet] 07paladox opened pull request #4276: mediawiki: allow name server to call healthcheck over port 80 (03master...03paladox-patch-1) 13https://github.com/miraheze/puppet/pull/4276 [15:37:13] [02puppet] 07coderabbitai[bot] commented on pull request #4276: --- […] 13https://github.com/miraheze/puppet/pull/4276#issuecomment-2746277548 [15:39:01] RECOVERY - rosettacode.org - LetsEncrypt on sslhost is OK: OK - Certificate 'rosettacode.org' will expire on Sat 21 Jun 2025 01:51:14 PM GMT +0000. [15:40:39] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-1 13https://github.com/miraheze/puppet/commit/236be7f8a064d55faa4e9999287e215bd0ee4b24 [15:40:39] 02puppet/03paladox-patch-1 07paladox 03236be7f Update mediawiki.pp [15:41:56] [02puppet] 07paladox retitled PR #4276: "mediawiki: allow name server to call healthcheck over port 80" ➜ "mediawiki: allow name server to call health over port 80" 13https://github.com/miraheze/puppet/pull/4276 [15:47:43] [02dns] 07paladox pushed 1 new commit to 03paladox-patch-1 13https://github.com/miraheze/dns/commit/098450dcdbb7bdf0aec1e6af84bc2f649f3f5d21 [15:47:43]