[01:38:52] 10MediaWiki-Releasing, 10MediaWiki-Installer, 06MediaWiki-Stakeholders-Group, 05MW-1.47-release: Expand the set of bundled extensions and skins in MediaWiki 1.47 - https://phabricator.wikimedia.org/T392080#12435064 (10Mr._Starfleet_Command) 05Openβ†’03Resolved [01:50:46] 10MediaWiki-Releasing, 10MediaWiki-Installer, 06MediaWiki-Stakeholders-Group, 05MW-1.48-release: Expand the set of bundled extensions and skins in MediaWiki 1.48 - https://phabricator.wikimedia.org/T440762#12435070 (10Mr._Starfleet_Command) [02:04:29] 10MediaWiki-Releasing, 10ExtensionDistributor, 05MW-1.47-release: Add REL1_47 to ExtensionDistributor as the development snapshot - https://phabricator.wikimedia.org/T441077#12435076 (10ArielGlenn) Note to future MediaWiki release folks: ExtensionDistributor is basically two parts, the extension which co... [02:20:19] (03CR) 10C. Scott Ananian: "Ie262bc0397fc9182ab6940ffa37b6c6822ac0877 is the alternative which sets it in DevelopmentSettings.php." [integration/quibble] - 10https://gerrit.wikimedia.org/r/1354238 (https://phabricator.wikimedia.org/T440804) (owner: 10C. Scott Ananian) [08:01:28] 06Release-Engineering-Team (Priority Backlog πŸ“₯), 07Essential-Work, 05Release, 05Train Deployments: 1.47.0-wmf.24 deployment blockers - https://phabricator.wikimedia.org/T438220#12435486 (10hashar) Thank you @brennen! Looks like it is all quiet :) [08:25:47] 06Release-Engineering-Team (Priority Backlog πŸ“₯), 07Essential-Work, 13Patch-For-Review, 05Release, 05Train Deployments: 1.47.0-wmf.24 deployment blockers - https://phabricator.wikimedia.org/T438220#12435568 (10hashar) [08:30:11] 06Release-Engineering-Team (Priority Backlog πŸ“₯), 07Essential-Work, 13Patch-For-Review, 05Release, 05Train Deployments: 1.47.0-wmf.24 deployment blockers - https://phabricator.wikimedia.org/T438220#12435589 (10hashar) [08:44:11] 10Phabricator (Upstream), 07Upstream: Email Preferences should inform of default values - https://phabricator.wikimedia.org/T182172#12435652 (10Od1n) My mailbox is a core part of my workflow, and I use it as a record, not just as a stream of alerts. So, for me, this is about far more than just β€œmore or les... [08:51:16] 06Release-Engineering-Team (Doing 😎), 10MediaWiki-Change-tagging, 10QS-Test-Automation, 10Catalyst (Luka Ijo Pimeja Jan), 07Essential-Work: Catalyst the first edit using each tag is saved without its tag - https://phabricator.wikimedia.org/T439780#12435696 (10jnuche) >>! In T439780#12431648, @vaughnwalte... [08:53:09] 06Release-Engineering-Team (Doing 😎), 10MediaWiki-Change-tagging, 10QS-Test-Automation, 10Catalyst (Luka Ijo Pimeja Jan), 07Essential-Work: Catalyst the first edit using each tag is saved without its tag - https://phabricator.wikimedia.org/T439780#12435701 (10jnuche) 05Openβ†’03Resolved Verified th... [08:59:19] 06Release-Engineering-Team (Priority Backlog πŸ“₯), 07Essential-Work, 05Release, 05Train Deployments: 1.47.0-wmf.24 deployment blockers - https://phabricator.wikimedia.org/T438220#12435736 (10hashar) [09:20:10] 06Release-Engineering-Team (Priority Backlog πŸ“₯), 07Essential-Work, 05Release, 05Train Deployments: 1.47.0-wmf.24 deployment blockers - https://phabricator.wikimedia.org/T438220#12435813 (10hashar) [10:29:41] Project mediawiki-core-doxygen build #23673: 04FAILURE in 2 min 19 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen/23673/ [10:32:04] 06Project-Admins: Request new Phabricator tag: gerrit-reviewer-bot - https://phabricator.wikimedia.org/T441297 (10siebrand) 03NEW [10:35:08] 06Project-Admins: Request new Phabricator tag: gerrit-reviewer-bot - https://phabricator.wikimedia.org/T441297#12436136 (10Peachey88) As this is a toolforge tool, iirc you should be able to self manage this within striker, which also creates the appropriate db records in striker to link back to the project [10:56:04] 10GitLab (Project and group requests), 06Release-Engineering-Team: GitLab Private Repository Request for: product-safety-and-integrity/wmf-security-skills - https://phabricator.wikimedia.org/T441301 (10kostajh) 03NEW [11:27:06] Yippee, build fixed! [11:27:06] Project mediawiki-core-doxygen build #23674: 09FIXED in 9 min 4 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen/23674/ [11:57:08] (03PS1) 10Tryvix1509: Update email for Codename Noreste. [integration/config] - 10https://gerrit.wikimedia.org/r/1355577 [12:02:55] 06Release-Engineering-Team (Doing 😎), 10Catalyst (Luka Ijo Pimeja Jan), 07Essential-Work: Cinder: Migrate Wiki repositories pool to Cinder volume - https://phabricator.wikimedia.org/T439711#12436442 (10jnuche) 05Openβ†’03Declined Right now, a Cinder driver won't work for the repo pool. In order for th... [12:03:25] 06Release-Engineering-Team (Radar), 10Scap, 07Sustainability (Incident Followup): Improve scap/deploy rollback turnaround times - https://phabricator.wikimedia.org/T417166#12436447 (10thcipriani) Moving to Release Engineering's radar as the task is not currently actionable by the team. [12:16:21] 10Gerrit, 06Release-Engineering-Team, 06Collaboration-Services, 10Wikimedia-GitHub, and 2 others: gerrit: document replication re-runs to Github - https://phabricator.wikimedia.org/T424172#12436498 (10thcipriani) 05Openβ†’03Resolved a:03thcipriani https://wikitech.wikimedia.org/w/index.php?title=Ge... [12:30:01] 10Scap, 06Collaboration-Services: scap deploy-services is unable to find helmfile for other cluster_groups - https://phabricator.wikimedia.org/T441310 (10Jelto) 03NEW [12:30:23] 10Scap, 06Collaboration-Services: scap deploy-services is unable to find helmfile for other cluster_groups - https://phabricator.wikimedia.org/T441310#12436563 (10Jelto) [12:30:30] 06Release-Engineering-Team, 10Scap, 06Collaboration-Services, 06ServiceOps, and 3 others: Proposal: scap deploy-service - https://phabricator.wikimedia.org/T412941#12436564 (10Jelto) [12:36:58] 10Scap, 06Collaboration-Services: scap is blocking all deploy-service deployments when invalid yaml is used - https://phabricator.wikimedia.org/T441311 (10Jelto) 03NEW [12:37:11] 10Scap, 06Collaboration-Services: scap is blocking all deploy-service deployments when invalid yaml is used - https://phabricator.wikimedia.org/T441311#12436604 (10Jelto) [12:37:18] 06Release-Engineering-Team, 10Scap, 06Collaboration-Services, 06ServiceOps, and 3 others: Proposal: scap deploy-service - https://phabricator.wikimedia.org/T412941#12436605 (10Jelto) [13:16:40] 10Scap, 06ServiceOps, 10ServiceOps-Datastores, 06SRE-OnFire, 07Sustainability (Incident Followup): Add etcdmirror status check to scap - https://phabricator.wikimedia.org/T317403#12436773 (10JMeybohm) [13:35:46] 06Project-Admins, 10Tool-gerrit-reviewer-bot: Request new Phabricator tag: gerrit-reviewer-bot - https://phabricator.wikimedia.org/T441297#12436880 (10siebrand) 05Openβ†’03Resolved a:03siebrand >>! In T441297#12436136 schreef @Peachey88: > As this is a toolforge tool, iirc you should be able to self ma... [13:59:49] 10Gerrit, 10Tool-gerrit-reviewer-bot: Gerrit creates duplicate changes when pushing same commit twice in quick succession - https://phabricator.wikimedia.org/T242590#12437016 (10siebrand) [14:01:36] 10Gerrit, 10Tool-gerrit-reviewer-bot, 07TestMe: Gerrit creates duplicate changes when pushing same commit twice in quick succession - https://phabricator.wikimedia.org/T242590#12437032 (10siebrand) Test if the condition can still be created. Maybe it's fixed in Gerrit. [14:05:02] maintenance-disconnect-full-disks build 850931 integration-agent-docker-1095 (/: 97%, /srv: 6%, /var/lib/docker: 22%): OFFLINE due to disk space [14:10:46] (03CR) 10Hashar: [C:03+2] "Sorry I was holding this one to make sure the task was looked at which it did almost immediately (see T440288#12410272 and follow up comme" [integration/config] - 10https://gerrit.wikimedia.org/r/1347670 (https://phabricator.wikimedia.org/T440288) (owner: 10Sbisson) [14:12:05] (03Merged) 10jenkins-bot: inference-services: Trigger model tests on test-only changes [integration/config] - 10https://gerrit.wikimedia.org/r/1347670 (https://phabricator.wikimedia.org/T440288) (owner: 10Sbisson) [14:16:43] 10MediaWiki-Releasing, 06translatewiki.net, 05MW-1.47-release: Configure Translatewiki.net for REL1_47 translation backports - https://phabricator.wikimedia.org/T441079#12437139 (10ArielGlenn) 05Openβ†’03Resolved a:03ArielGlenn [14:25:02] maintenance-disconnect-full-disks build 850935 integration-agent-docker-1095 (/: 97%, /srv: 6%, /var/lib/docker: 22%): still OFFLINE due to disk space [14:44:41] 10Phabricator-Bot-Requests: Create a Phabricator bot account for Data Engineering dev-metrics (developer-activity-pipelines) - https://phabricator.wikimedia.org/T441326 (10Antoine_Quhen) 03NEW [14:47:04] 10Phabricator-Bot-Requests: Create a Phabricator bot account for Data Engineering dev-metrics (developer-activity-pipelines) - https://phabricator.wikimedia.org/T441326#12437344 (10Antoine_Quhen) [14:48:10] 10Phabricator-Bot-Requests: Create a Phabricator bot account for Data Engineering dev-metrics (developer-activity-pipelines) - https://phabricator.wikimedia.org/T441326#12437367 (10Antoine_Quhen) [14:48:17] 10Phabricator, 06Collaboration-Services, 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 13Patch-For-Review: data lake: phabricator events - https://phabricator.wikimedia.org/T434047#12437368 (10Antoine_Quhen) [14:50:02] maintenance-disconnect-full-disks build 850940 integration-agent-docker-1095 (/: 97%, /srv: 6%, /var/lib/docker: 22%): still OFFLINE due to disk space [14:54:26] 10Gerrit, 06Release-Engineering-Team, 06Security-Team, 07SecTeam-Processed, 07Security: Develop a means for preventing developers from accidentally pushing embargoed security patches to gerrit - https://phabricator.wikimedia.org/T388247#12437423 (10thcipriani) Note: there's also the gerrit [[https://gerr... [14:58:34] 10Continuous-Integration-Infrastructure, 07Sustainability (Incident Followup): Fail more gracefully when a job is out of resources - https://phabricator.wikimedia.org/T393201#12437434 (10thcipriani) p:05Triageβ†’03Low Since the job is running on a VM with insufficient resources (i.e., it's being killed by OO... [14:58:43] 10Phabricator-Bot-Requests: Create a Phabricator bot account for Data Engineering dev-metrics (developer-activity-pipelines) - https://phabricator.wikimedia.org/T441326#12437436 (10Antoine_Quhen) [14:59:52] 10Gerrit, 06Release-Engineering-Team, 06Security-Team, 07SecTeam-Processed, 07Security: Develop a means for preventing developers from accidentally pushing embargoed security patches to gerrit - https://phabricator.wikimedia.org/T388247#12437441 (10sbassett) >>! In T388247#12437423, @thcipriani wrote: >... [15:14:51] (03CR) 10Hashar: zuul: Add PersonalDashboard to GrowthExperiments browser tests (033 comments) [integration/config] - 10https://gerrit.wikimedia.org/r/1355048 (https://phabricator.wikimedia.org/T438582) (owner: 10Sergio Gimeno) [15:15:02] maintenance-disconnect-full-disks build 850945 integration-agent-docker-1095 (/: 97%, /srv: 6%, /var/lib/docker: 22%): still OFFLINE due to disk space [15:15:21] 10Scap, 06Collaboration-Services, 13Patch-For-Review: scap deploy-services is unable to find helmfile for other cluster_groups - https://phabricator.wikimedia.org/T441310#12437521 (10dancy) 05Openβ†’03In progress p:05Triageβ†’03High a:03dancy [15:26:31] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team (Doing 😎), 10Release Pipeline, 13Patch-For-Review: Upgrade integration-agent-docker instances to trixie - https://phabricator.wikimedia.org/T440670#12437627 (10hashar) Eventually got an alarm about disk filling up: ` maintenance-disconnect... [15:29:22] (03PS2) 10Tryvix1509: zuul: Update email for Codename Noreste. [integration/config] - 10https://gerrit.wikimedia.org/r/1355577 [15:29:25] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team (Doing 😎), 10Release Pipeline, 13Patch-For-Review: Upgrade integration-agent-docker instances to trixie - https://phabricator.wikimedia.org/T440670#12437634 (10hashar) Twist: Docker reports different image sizes. For: docker-registry.wikim... [15:30:04] dduvall: hi so Docker on Trixie writes a bunch of data on /var/lib/containerd which is part of the small `/` partition and it files it. I wrote on the task about it: https://phabricator.wikimedia.org/T440670#12437627 [15:30:49] no idea about a recommended solution, I merely reacted to the maintenance-disconnect-full-disks notification above [15:31:22] (03CR) 10Hashar: [C:03+2] zuul: Update email for Codename Noreste. [integration/config] - 10https://gerrit.wikimedia.org/r/1355577 (owner: 10Tryvix1509) [15:32:29] (03Merged) 10jenkins-bot: zuul: Update email for Codename Noreste. [integration/config] - 10https://gerrit.wikimedia.org/r/1355577 (owner: 10Tryvix1509) [15:37:23] (03CR) 10Hashar: [C:03+2] "Deployed!" [integration/config] - 10https://gerrit.wikimedia.org/r/1355577 (owner: 10Tryvix1509) [15:40:02] maintenance-disconnect-full-disks build 850950 integration-agent-docker-1095 (/: 97%, /srv: 6%, /var/lib/docker: 22%): still OFFLINE due to disk space [15:41:33] dancy: thx for https://phabricator.wikimedia.org/T441331 about "Broken variant table: sh-cyrl,sh-latn" Quite happy to see it filed as I was doing a last round of logs/graphs monitoring :) [15:41:45] yw [15:48:57] 06Release-Engineering-Team (Priority Backlog πŸ“₯), 07Essential-Work, 05Release, 05Train Deployments: 1.47.0-wmf.24 deployment blockers - https://phabricator.wikimedia.org/T438220#12437741 (10hashar) [15:50:17] 06Release-Engineering-Team (Priority Backlog πŸ“₯), 07Essential-Work, 05Release, 05Train Deployments: 1.47.0-wmf.24 deployment blockers - https://phabricator.wikimedia.org/T438220#12437755 (10hashar) shwiki has issues display images //RuntimeException: Broken variant table: sh-cyrl,sh-latn// which comes from... [16:01:13] 10Phabricator, 06Collaboration-Services, 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 13Patch-For-Review: data lake: phabricator events - https://phabricator.wikimedia.org/T434047#12437860 (10Ahoelzl) [16:05:02] maintenance-disconnect-full-disks build 850955 integration-agent-docker-1095 (/: 97%, /srv: 6%, /var/lib/docker: 22%): still OFFLINE due to disk space [16:06:12] 10Gerrit, 06Release-Engineering-Team, 06Security-Team, 07SecTeam-Processed, 07Security: Develop a means for preventing developers from accidentally pushing embargoed security patches to gerrit - https://phabricator.wikimedia.org/T388247#12438021 (10thcipriani) Ack. Checking for a subject starting with `S... [16:09:55] 06Release-Engineering-Team (Priority Backlog πŸ“₯), 07Essential-Work, 05Release, 05Train Deployments: 1.47.0-wmf.24 deployment blockers - https://phabricator.wikimedia.org/T438220#12438041 (10ihurbain) The reason for the ParserCache issue is https://phabricator.wikimedia.org/T441221 - I believe we're missing... [16:09:57] hashar: The stacktrace of T440951 starts in the Discord preview code, which is maintained by Future Audiences (https://gerrit.wikimedia.org/g/mediawiki/extensions/WikimediaEvents/+/HEAD/OWNERS.md, T423746). [16:09:57] T440951: PHP Deprecated: Use of MediaWiki\Page\ParserOutputAccess::getParserOutput with int options was deprecated in MediaWiki 1.47. [Called from WikimediaCustomizations\Discord\DiscordPreviewRestHandler::buildExtract] - https://phabricator.wikimedia.org/T440951 [16:09:58] T423746: Add HTML header for Discord previews - https://phabricator.wikimedia.org/T423746 [16:10:01] The WikimediaCustomizations repo is similar to wmf-config and WikimediaEvents. There isn't a single owner for the whole repo. [16:10:02] I've added a note to the mw:Maintainers page! [16:10:11] 10Phabricator: Tag (silently) all tasks of T432486 to workboard - https://phabricator.wikimedia.org/T441188#12438043 (10Aklapper) All and any subtasks (open or closed? direct/first-level vs all?) or some subset with some criteria? For *direct* (first-level) subtasks (but that's only five tasks), on https://phab... [16:19:26] 10Phabricator: Herald rule to add a task to a workboard if it is added as a subtask to an existing task - https://phabricator.wikimedia.org/T441165#12438110 (10Aklapper) 05Openβ†’03Declined Philosophical question, as usual, is where to put the cut-off (what is still a task (with subtasks) to track work, an... [16:23:49] 10Continuous-Integration-Infrastructure, 10Gerrit, 06Release-Engineering-Team, 06Collaboration-Services, and 2 others: Document how to restart CI when Gerrit had a production issue - https://phabricator.wikimedia.org/T424196#12438164 (10LSobanski) a:05hasharβ†’03LSobanski [16:24:39] 06Release-Engineering-Team (Priority Backlog πŸ“₯), 07Essential-Work, 05Release, 05Train Deployments, 07Wikimedia-production-error: Edits to articles seem to be causing page images to get dropped - https://phabricator.wikimedia.org/T441345 (10egardner) 03NEW p:05Triageβ†’03Medium [16:26:47] 06Release-Engineering-Team (Priority Backlog πŸ“₯), 07Essential-Work, 05Release, 05Train Deployments, 07Wikimedia-production-error: Edits to articles seem to be causing page images to get dropped - https://phabricator.wikimedia.org/T441345#12438212 (10Jdlrobson-WMF) p:05Mediumβ†’03Unbreak! [16:27:13] 06Release-Engineering-Team (Priority Backlog πŸ“₯), 06Content-Platform-Team, 07Essential-Work, 07Wikimedia-production-error: Edits to articles seem to be causing page images to get dropped - https://phabricator.wikimedia.org/T441345#12438219 (10Jdlrobson-WMF) [16:27:22] 06Release-Engineering-Team (Priority Backlog πŸ“₯), 06Content-Platform-Team, 07Essential-Work, 07Wikimedia-production-error: Edits to articles seem to be causing page images to get dropped - https://phabricator.wikimedia.org/T441345#12438220 (10Jdlrobson-WMF) [16:28:19] 06Release-Engineering-Team (Priority Backlog πŸ“₯), 06Content-Platform-Team, 07Essential-Work, 07Wikimedia-production-error: Edits to articles seem to be causing page images to get dropped - https://phabricator.wikimedia.org/T441345#12438239 (10Jdlrobson-WMF) @cscott @Jdforrester-WMF is it safe to revert http... [16:30:02] maintenance-disconnect-full-disks build 850960 integration-agent-docker-1095 (/: 97%, /srv: 6%, /var/lib/docker: 22%): still OFFLINE due to disk space [16:31:15] 06Release-Engineering-Team (Priority Backlog πŸ“₯), 06Content-Platform-Team, 10PageImages, 07Essential-Work, 07Wikimedia-production-error: Edits to articles seem to be causing page images to get dropped - https://phabricator.wikimedia.org/T441345#12438262 (10Jdlrobson-WMF) [16:45:16] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for Mercyjae - https://phabricator.wikimedia.org/T441191#12438344 (10Bodhisattwa) β†’14Duplicate dup:03T441171 [16:45:17] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for mercyjae - https://phabricator.wikimedia.org/T441171#12438342 (10Bodhisattwa) [16:45:27] 06Release-Engineering-Team (Priority Backlog πŸ“₯), 06Content-Platform-Team, 10PageImages, 07Essential-Work, and 2 others: Edits to articles seem to be causing page images to get dropped - https://phabricator.wikimedia.org/T441345#12438346 (10ABreault-WMF) @Jdlrobson-WMF That was supposed to be disabled in pr... [16:47:24] (03open) 10dancy: deploy: Do not write deploy server info into DEPLOY_HEAD [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1357 (https://phabricator.wikimedia.org/T197470) [16:50:21] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for mercyjae - https://phabricator.wikimedia.org/T441171#12438364 (10Mercyjae) Hi admin, I made a mistake by creating this request twice. That was the reason fot this duplicate requests. It has been merged to one. Kin... [16:52:02] 10Gerrit, 06Release-Engineering-Team, 06Security-Team, 07SecTeam-Processed, 07Security: Develop a means for preventing developers from accidentally pushing embargoed security patches to gerrit - https://phabricator.wikimedia.org/T388247#12438375 (10sbassett) >>! In T388247#12438021, @thcipriani wrote: >... [16:55:02] maintenance-disconnect-full-disks build 850965 integration-agent-docker-1095 (/: 97%, /srv: 6%, /var/lib/docker: 22%): still OFFLINE due to disk space [16:55:14] (03update) 10dancy: deploy: Do not write deploy server info into DEPLOY_HEAD [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1357 (https://phabricator.wikimedia.org/T197470) [16:58:03] 10Gerrit, 10Tool-gerrit-reviewer-bot, 13Patch-For-Review, 07TestMe: Gerrit creates duplicate changes when pushing same commit twice in quick succession - https://phabricator.wikimedia.org/T242590#12438420 (10siebrand) I wasn't able to reproduce creating a two changesets with the same change ID. Proposing t... [17:03:16] 10Gerrit, 06Release-Engineering-Team, 06Collaboration-Services, 07Sustainability (Incident Followup): Alert when Gerrit CI (Zuul, Jenkins, Gearman) is down/stuck - https://phabricator.wikimedia.org/T423123#12438441 (10thcipriani) As the task describes: CI not running may be a normal state. In the incident... [17:16:11] (03open) 10dancy: deploy-service: Report and skip bad service catalog entries [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1358 (https://phabricator.wikimedia.org/T441311) [17:18:33] (03update) 10dancy: deploy-service: Report and skip bad service catalog entries [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1358 (https://phabricator.wikimedia.org/T441311) [17:19:16] (03merge) 10dancy: deploy-service: Report and skip bad service catalog entries [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1358 (https://phabricator.wikimedia.org/T441311) [17:20:02] maintenance-disconnect-full-disks build 850970 integration-agent-docker-1095 (/: 97%, /srv: 6%, /var/lib/docker: 22%): still OFFLINE due to disk space [17:20:13] (03open) 10dancy: deploy-service: Check the helmfile directory before the deployment [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1359 (https://phabricator.wikimedia.org/T441310) [17:21:26] (03update) 10dancy: deploy-service: Check the helmfile directory before the deployment [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1359 (https://phabricator.wikimedia.org/T441310) [17:23:25] (03merge) 10dancy: deploy-service: Check the helmfile directory before the deployment [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1359 (https://phabricator.wikimedia.org/T441310) [17:25:54] (03open) 10dancy: deploy-service: Accept only a list of environments and releases [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1360 (https://phabricator.wikimedia.org/T412941) [17:26:42] (03update) 10dancy: deploy-service: Accept only a list of environments and releases [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1360 (https://phabricator.wikimedia.org/T412941) [17:27:43] (03merge) 10dancy: deploy-service: Accept only a list of environments and releases [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1360 (https://phabricator.wikimedia.org/T412941) [17:33:30] (03open) 10dancy: Release 4.299.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1361 [17:33:32] (03update) 10dancy: Release 4.299.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1361 [17:35:25] (03merge) 10dancy: Release 4.299.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1361 [17:45:02] maintenance-disconnect-full-disks build 850975 integration-agent-docker-1095 (/: 97%, /srv: 6%, /var/lib/docker: 22%): still OFFLINE due to disk space [18:10:02] maintenance-disconnect-full-disks build 850980 integration-agent-docker-1095 (/: 97%, /srv: 6%, /var/lib/docker: 22%): still OFFLINE due to disk space [18:27:48] 10Phabricator: Requesting acl*phabricator access for Catrope - https://phabricator.wikimedia.org/T441354 (10Catrope) 03NEW [18:30:49] 10Phabricator, 06Release-Engineering-Team, 07Essential-Work: Requesting acl*phabricator access for Catrope - https://phabricator.wikimedia.org/T441354#12438815 (10brennen) 05Openβ†’03Resolved a:03brennen Done. [18:35:02] maintenance-disconnect-full-disks build 850985 integration-agent-docker-1095 (/: 97%, /srv: 6%, /var/lib/docker: 22%): still OFFLINE due to disk space [19:00:02] maintenance-disconnect-full-disks build 850990 integration-agent-docker-1095 (/: 97%, /srv: 6%, /var/lib/docker: 22%): still OFFLINE due to disk space [19:17:36] 10MediaWiki-Releasing, 07Documentation, 05MW-1.47-release: Finish up RELEASE-NOTES for 1.47.0 - https://phabricator.wikimedia.org/T441084#12439013 (10Umherirrender) Please fill in the XXX in this sentence: ` === Deprecations in 1.47 === * (T166010) All PHP code in MediaWiki is slowly being moved to be in a... [19:25:02] maintenance-disconnect-full-disks build 850995 integration-agent-docker-1095 (/: 97%, /srv: 6%, /var/lib/docker: 22%): still OFFLINE due to disk space [19:50:02] maintenance-disconnect-full-disks build 851000 integration-agent-docker-1095 (/: 97%, /srv: 6%, /var/lib/docker: 22%): still OFFLINE due to disk space [20:10:21] !log performing rolling restart of docker on integration-agent-docker instances to deploy https://gerrit.wikimedia.org/r/c/operations/puppet/+/1355127 (T440670) [20:10:23] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:10:24] T440670: Upgrade integration-agent-docker instances to trixie - https://phabricator.wikimedia.org/T440670 [20:15:02] maintenance-disconnect-full-disks build 851005 integration-agent-docker-1095 (/: 97%, /srv: 6%, /var/lib/docker: 22%): still OFFLINE due to disk space [20:40:02] maintenance-disconnect-full-disks build 851010 integration-agent-docker-1095 (/: 97%, /srv: 6%, /var/lib/docker: 22%): still OFFLINE due to disk space [21:05:02] maintenance-disconnect-full-disks build 851015 integration-agent-docker-1095 (/: 97%, /srv: 6%, /var/lib/docker: 22%): still OFFLINE due to disk space [21:26:06] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team (Doing 😎), 10Release Pipeline: Upgrade integration-agent-docker instances to trixie - https://phabricator.wikimedia.org/T440670#12439623 (10dduvall) >>! In T440670#12437627, @hashar wrote: > With the newer Docker, images get written to `/var... [21:30:02] maintenance-disconnect-full-disks build 851020 integration-agent-docker-1095 (/: 97%, /srv: 6%, /var/lib/docker: 22%): still OFFLINE due to disk space [21:32:39] (03update) 10dancy: spiderpig: Export Prometheus metrics [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1356 [21:55:02] maintenance-disconnect-full-disks build 851025 integration-agent-docker-1095 (/: 97%, /srv: 6%, /var/lib/docker: 22%): still OFFLINE due to disk space [22:05:48] Hi, all. I run several large public MW wikis and I'm working on an upgrade to them. I have a shell script that for years I've used to git clone MW core and the skins and extensions we use (about 43 repos in total). I last ran a new modified version of it on Tuesday but today it started failing with 403 responses. This appears to be because my wikis [22:05:48] are in AWS, and my build script is run on a "manager" EC2 instance, and my understanding is that WM has tightened down access from hosting provider IPs. At several recommendations, I've ported my script to using the github.com mirror for my clones, but even there I hit some snags. Is there any way I'd be able to continue having my script clone from [22:05:49] gerrit on my EC2 instances? [22:20:02] maintenance-disconnect-full-disks build 851030 integration-agent-docker-1095 (/: 97%, /srv: 6%, /var/lib/docker: 22%): still OFFLINE due to disk space [22:45:02] maintenance-disconnect-full-disks build 851035 integration-agent-docker-1095 (/: 97%, /srv: 6%, /var/lib/docker: 22%): still OFFLINE due to disk space [23:10:02] maintenance-disconnect-full-disks build 851040 integration-agent-docker-1095 (/: 97%, /srv: 6%, /var/lib/docker: 22%): still OFFLINE due to disk space [23:35:02] maintenance-disconnect-full-disks build 851045 integration-agent-docker-1095 (/: 17%, /srv: 6%, /var/lib/docker: 22%): RECOVERY disk space OK