[00:04:58] !log creating new integration-agent-puppet-docker-{1006,1007} trixie based instances/agents (T440670) [00:05:00] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [00:05:01] T440670: Upgrade integration-agent-docker instances to trixie - https://phabricator.wikimedia.org/T440670 [00:05:01] (03open) 10project_16_bot_841eade40a22c56a6272172b8cb67384: Bump docker-registry.wikimedia.org/wikimedia/mediawiki-libs-shellbox image group to 2026-11-02-141242 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/797 [00:05:05] (03open) 10project_16_bot_841eade40a22c56a6272172b8cb67384: Bump docker-registry.wikimedia.org/repos/data-engineering/eventgate-wikimedia from v1.36.0 to v1.38.0 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/798 [00:05:08] (03open) 10project_16_bot_841eade40a22c56a6272172b8cb67384: Bump docker-registry.wikimedia.org/releng/quibble-bookworm-php83 from 1.22.0-s8 to 1.22.0-s9 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/799 [00:05:10] (03update) 10project_16_bot_841eade40a22c56a6272172b8cb67384: Bump docker-registry.wikimedia.org/repos/data-engineering/eventgate-wikimedia from v1.36.0 to v1.38.0 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/798 [00:05:12] (03update) 10project_16_bot_841eade40a22c56a6272172b8cb67384: Bump docker-registry.wikimedia.org/releng/quibble-bookworm-php83 from 1.22.0-s8 to 1.22.0-s9 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/799 [00:19:17] !log taking integration-agent-puppet-docker-{1004,1005} offline to be replaced by trixie agents (T440670) [00:19:19] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [00:19:20] T440670: Upgrade integration-agent-docker instances to trixie - https://phabricator.wikimedia.org/T440670 [00:21:11] !log deleting integration-agent-puppet-docker-{1004,1005} agents and instances (T440670) [00:21:13] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [00:21:29] !log registered new integration-agent-puppet-docker-{1006,1007} trixie based agents with jenkins (T440670) [00:21:32] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [00:27:41] !log all integration-agent-docker and integration-agent-puppet-docker agents have been migrated to trixie (T440670) [00:27:44] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [00:27:44] T440670: Upgrade integration-agent-docker instances to trixie - https://phabricator.wikimedia.org/T440670 [00:32:07] !log restarting containerd on contint{1003,2003} to deploy https://gerrit.wikimedia.org/r/c/operations/puppet/+/1356455 [00:32:08] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [00:47:02] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team (Doing 😎), 10Release Pipeline: Upgrade integration-agent-docker instances to trixie - https://phabricator.wikimedia.org/T440670#12520323 (10dduvall) 05In progressβ†’03Resolved [01:52:40] 10Continuous-Integration-Config, 07Developer Productivity, 10MobileFrontend (JavaScript core library): Export Popups dist files as CI artifacts - https://phabricator.wikimedia.org/T352240#12520522 (10Jdlrobson-WMF) [01:52:49] 10Continuous-Integration-Config, 10Page-Previews, 07Developer Productivity: Export Popups dist files as CI artifacts - https://phabricator.wikimedia.org/T352240#12520523 (10Jdlrobson-WMF) [02:02:35] (03CR) 10Jforrester: [C:03+2] build: Updating postcss-selector-parser to 6.1.5 [performance/WikimediaDebug] - 10https://gerrit.wikimedia.org/r/1363413 (owner: 10Libraryupgrader) [06:05:36] (03PS1) 10AntiCompositeNumber: zuul: Update email for AntiCompositeNumber [integration/config] - 10https://gerrit.wikimedia.org/r/1363856 [07:58:25] (03CR) 10Hashar: [C:03+2] zuul: Update email for AntiCompositeNumber [integration/config] - 10https://gerrit.wikimedia.org/r/1363856 (owner: 10AntiCompositeNumber) [07:59:33] (03Merged) 10jenkins-bot: zuul: Update email for AntiCompositeNumber [integration/config] - 10https://gerrit.wikimedia.org/r/1363856 (owner: 10AntiCompositeNumber) [08:46:16] 06Project-Admins, 06Movement-Insights: Create analytics-data-problem Phabricator tag - https://phabricator.wikimedia.org/T362839#12521237 (10Aklapper) [08:47:20] (03Abandoned) 10Hashar: dockerfiles: Rebuild all images for the ca-certificates update [integration/config] - 10https://gerrit.wikimedia.org/r/1361480 (https://phabricator.wikimedia.org/T443253) (owner: 10Jforrester) [09:01:52] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 06Test Platform, 06tools-infrastructure-team: Add more CPUs to quibble jobs - https://phabricator.wikimedia.org/T439808#12521262 (10fgiunchedi) >>! In T439808#12414727, @Peter wrote: > Lets add the instances when T440670 is done so the ne... [12:05:06] FIRING: [2x] PuppetAgentFailure: Puppet agent failure detected on instance deployment-cp-text09 in project deployment-prep - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentFailure [12:27:38] 10GitLab (Project and group requests), 06Release-Engineering-Team (Doing 😎), 10WMF security vulnerability pipeline, 05FY25-26 DE4.4, 06Product Safety and Integrity (Sprint Chrysanthemum (Nov 2 - Nov 20)): GitLab Private Repository Request for: repos/prod... - https://phabricator.wikimedia.org/T443448#12521967 [13:30:04] 06Release-Engineering-Team (Doing 😎), 06WMF-NDA-Requests: Add FSesini-WMF to the WMF-NDA Phab group - https://phabricator.wikimedia.org/T443779#12522141 (10Aklapper) 05Openβ†’03Resolved a:03Aklapper Hi and welcome! Added after verification. [14:03:51] (03CR) 10Hashar: [C:03+2] Zuul: [mediawiki/extensions/SemanticPageSeries] Mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1362270 (https://phabricator.wikimedia.org/T437245) (owner: 10Pppery) [14:03:54] (03CR) 10Hashar: [C:03+2] Zuul: [mediawiki/extensions/WikispeechSpeechDataCollector] Mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1362291 (https://phabricator.wikimedia.org/T432801) (owner: 10Pppery) [14:04:01] (03PS2) 10Hashar: utils/archive_repo.py: Add missing word [integration/config] - 10https://gerrit.wikimedia.org/r/1362293 (owner: 10Pppery) [14:04:04] (03CR) 10Hashar: [C:03+2] utils/archive_repo.py: Add missing word [integration/config] - 10https://gerrit.wikimedia.org/r/1362293 (owner: 10Pppery) [14:05:49] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/SemanticPageSeries] Mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1362270 (https://phabricator.wikimedia.org/T437245) (owner: 10Pppery) [14:05:51] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/WikispeechSpeechDataCollector] Mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1362291 (https://phabricator.wikimedia.org/T432801) (owner: 10Pppery) [14:05:53] (03Merged) 10jenkins-bot: utils/archive_repo.py: Add missing word [integration/config] - 10https://gerrit.wikimedia.org/r/1362293 (owner: 10Pppery) [14:10:23] 10Gerrit, 06Release-Engineering-Team, 07SecTeam-Processed, 07Security: Prevent Gerrit changes with top-level .patch files from being pushed - https://phabricator.wikimedia.org/T391269#12522314 (10Lucas_Werkmeister_WMDE) I thought we agreed to do this one as a low-hanging fruit without having to wait fo... [14:23:43] 06Release-Engineering-Team (Doing 😎), 06WMF-NDA-Requests: Add FSesini-WMF to the WMF-NDA Phab group - https://phabricator.wikimedia.org/T443779#12522397 (10FSesini-WMF) Thanks! [14:34:05] FIRING: PuppetAgentNoResources: No Puppet resources found on instance deployment-webperf31 on project deployment-prep - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentNoResources [15:45:45] 10Continuous-Integration-Config, 06Release-Engineering-Team, 06SRE, 07Puppet: Upgrade operations/puppet CI from Bullseye to Bookworm - https://phabricator.wikimedia.org/T443798 (10hashar) 03NEW [15:45:53] 10Continuous-Integration-Config, 06Release-Engineering-Team, 06SRE, 07Puppet: Upgrade operations/puppet CI from Bullseye to Bookworm - https://phabricator.wikimedia.org/T443798#12522973 (10hashar) [15:45:57] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 13Patch-For-Review: Migrate all CI jobs from Bullseye to Bookworm or later and drop Bullseye testing support - https://phabricator.wikimedia.org/T383337#12522972 (10hashar) [15:46:08] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 13Patch-For-Review: Migrate all CI jobs from Bullseye to Bookworm or later and drop Bullseye testing support - https://phabricator.wikimedia.org/T383337#12522976 (10hashar) [16:07:47] 10Scap (SpiderPig πŸ•ΈοΈ): Allow deployment of single releases with Spiderpig deploy-service - https://phabricator.wikimedia.org/T442586#12523246 (10Jelto) >>! In T442586#12492779, @dancy wrote: > @Jelto Just confirming that we want to limit this to single releases, not allowing the user to select multiple releases?... [16:12:51] (03open) 10dancy: spiderpig: Build each job of an API response the same way [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1409 [16:14:25] (03update) 10dancy: spiderpig: Build each job of an API response the same way [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1409 [16:14:28] (03update) 10dancy: spiderpig: Build each job of an API response the same way [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1409 [16:15:27] (03update) 10dancy: spiderpig: Build each job of an API response the same way [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1409 [16:17:51] (03merge) 10dancy: spiderpig: Build each job of an API response the same way [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1409 [16:33:48] (03open) 10dancy: image: Add `BUILD_CACHE_EXPORT_MODE` to set the cache export mode [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/169 [16:37:12] (03open) 10dancy: ci: Use golang1.22:1.22-20261101 for release-cli [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/170 [16:37:37] (03update) 10dancy: ci: Use golang1.22:1.22-20261101 for release-cli [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/170 [16:46:41] 10Continuous-Integration-Config, 10MediaWiki-Vagrant: Remove CI support for mediawiki/vagrant - https://phabricator.wikimedia.org/T443567#12523555 (10hashar) 05Openβ†’03Resolved a:03hashar [16:48:33] (03open) 10dancy: ci: Update stale base images in blubber.yaml [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/171 [16:52:12] (03close) 10dancy: ci: Use golang1.22:1.22-20261101 for release-cli [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/170 [16:52:12] (03update) 10dancy: ci: Use golang1.22:1.22-20261101 for release-cli [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/170 [16:54:08] (03update) 10dancy: ci: Update stale base images [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/171 [16:54:09] (03update) 10dancy: ci: Update stale base images [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/171 [16:57:41] (03merge) 10dancy: ci: Update stale base images [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/171 [16:58:12] (03update) 10dancy: spiderpig: web: Convert `api.js` and `state.js` to TypeScript [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1408 [16:58:21] (03update) 10dancy: spiderpig: web: Convert `api.js` and `state.js` to TypeScript [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1408 [16:59:09] (03update) 10dancy: spiderpig: web: Convert `api.js` and `state.js` to TypeScript [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1408 [17:02:33] (03update) 10dancy: image: Add `BUILD_CACHE_EXPORT_MODE` to set the cache export mode [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/169 [17:05:27] (03update) 10dancy: image: Add `BUILD_CACHE_EXPORT_MODE` to set the cache export mode [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/169 [17:08:59] (03merge) 10dancy: spiderpig: web: Convert `api.js` and `state.js` to TypeScript [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1408 [17:12:26] (03open) 10dancy: spiderpig: web: Remove the unreachable warning style of the job status [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1410 [17:12:27] (03update) 10dancy: spiderpig: web: Remove the unreachable warning style of the job status [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1410 [17:12:40] (03update) 10dancy: spiderpig: web: Remove the unreachable warning style of the job status [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1410 [17:15:28] (03merge) 10dancy: spiderpig: web: Remove the unreachable warning style of the job status [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1410 [17:19:10] (03open) 10dancy: CI: Reuse the verify-deps apt layer in the test jobs [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1411 [17:23:41] (03merge) 10dancy: CI: Reuse the verify-deps apt layer in the test jobs [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1411 [17:30:49] (03open) 10dancy: spiderpig: web: Show every failed job as an error [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1412 [17:31:09] (03update) 10dancy: spiderpig: web: Show every failed job as an error [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1412 [17:34:04] (03merge) 10dancy: spiderpig: web: Show every failed job as an error [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1412 [17:43:54] !log creating new integration-agent-docker-9998 trixie based instances/agent to replace integration-agent-docker-9999 (T440670) [17:43:56] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:43:56] T440670: Upgrade integration-agent-docker instances to trixie - https://phabricator.wikimedia.org/T440670 [17:48:04] (03open) 10dancy: scap: Beep after a failure only at a terminal [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1413 [17:48:15] (03update) 10dancy: scap: Beep after a failure only at a terminal [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1413 [17:48:30] (03update) 10brennen: fundraising: Re-build for libbrotli1, and install Node.js 22 from nodejs.org [repos/releng/dev-images] - 10https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/133 (https://phabricator.wikimedia.org/T442495) (owner: 10jforrester) [17:50:22] (03merge) 10dancy: scap: Beep after a failure only at a terminal [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1413 [17:59:52] (03open) 10dancy: spiderpig: web: Type-check the web UI in `npm test` [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1414 [18:00:55] (03update) 10dancy: spiderpig: web: Type-check the web UI in `npm test` [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1414 [18:02:31] (03merge) 10dancy: spiderpig: web: Type-check the web UI in `npm test` [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1414 [18:12:55] (03merge) 10dancy: make-container-image: Build image source files owned by 'app', not www-data [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/251 (https://phabricator.wikimedia.org/T411063) [18:41:06] 06Release-Engineering-Team (Radar), 07Essential-Work, 10Test Kitchen (Experiment Platform Sprint 32): Determine if we can expand TK experiment deployment windows - https://phabricator.wikimedia.org/T440612#12524134 (10Jdlrobson-WMF) @JVanderhoop-WMF it sounds good in practice, but I'm concerned that if a dep... [19:06:26] (03open) 10dancy: spiderpig: Add an event stream of job changes [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1415 [19:07:18] (03update) 10dancy: spiderpig: Add an event stream of job changes [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1415 [19:08:56] (03merge) 10dancy: spiderpig: Add an event stream of job changes [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1415 [19:27:32] (03open) 10dancy: spiderpig: Read the jobrunner status from the event stream [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1416 [19:29:44] (03merge) 10dancy: spiderpig: Read the jobrunner status from the event stream [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1416 [19:31:55] 06Release-Engineering-Team (Priority Backlog πŸ“₯), 07Essential-Work, 05Release, 05Train Deployments: 1.47.0-wmf.24 deployment blockers - https://phabricator.wikimedia.org/T438220#12524300 (10brennen) 05Openβ†’03Resolved [20:25:00] (03update) 10catrope: Add a script for checking which security patches can't be backported [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/282 [20:35:33] 10Phabricator (Upstream), 07Upstream: Typos on various codes? - https://phabricator.wikimedia.org/T430097#12524481 (10Pppery) 05Openβ†’03Resolved Closing as WMF has merged upstream. (They still haven't deployed yet but that's irrelevant here since this is a bug report from translatewiki.net which reads... [20:35:45] 10Phabricator (Upstream), 07Upstream: Typo in Arcanist's XUnitTestEngine.php:91 - https://phabricator.wikimedia.org/T428380#12524483 (10Pppery) 05Openβ†’03Resolved Closing as WMF has merged upstream. (They still haven't deployed yet but that's irrelevant here since this is a bug report from translatewik... [20:35:47] 10Phabricator (Upstream), 07Upstream: [Not a bug] typo on PhutilArgumentParser.php$419 - https://phabricator.wikimedia.org/T428328#12524486 (10Pppery) 05Openβ†’03Resolved Closing as WMF has merged upstream. (They still haven't deployed yet but that's irrelevant here since this is a bug report from trans... [20:51:15] 10GitLab (CI & Job Runners): runner-1036 & runner-1034.gitlab-runners.eqiad1.wikimedia.cloud running out of space - https://phabricator.wikimedia.org/T443833 (10Addshore) 03NEW [20:51:46] 10GitLab (CI & Job Runners): runner-1036 & runner-1034.gitlab-runners.eqiad1.wikimedia.cloud running out of space - https://phabricator.wikimedia.org/T443833#12524576 (10Addshore) [20:58:29] !log deleting integration-agent-docker-9999 following replacement (T440670) [20:58:32] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:58:32] T440670: Upgrade integration-agent-docker instances to trixie - https://phabricator.wikimedia.org/T440670 [20:59:13] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team (Doing 😎), 10Release Pipeline: Upgrade integration-agent-docker instances to trixie - https://phabricator.wikimedia.org/T440670#12524606 (10dduvall) 05Resolvedβ†’03Open Reopening since I missed that the integration-agent-qemu instance has... [21:05:42] !log creating integration-agent-qemu-1005 trixie based agent to replace integration-agent-qemu-1004 (T440670) [21:05:44] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:05:45] T440670: Upgrade integration-agent-docker instances to trixie - https://phabricator.wikimedia.org/T440670 [21:32:29] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team (Doing 😎), 10Release Pipeline: Upgrade integration-agent-docker instances to trixie - https://phabricator.wikimedia.org/T440670#12524798 (10dduvall) @hashar just noting that I ran into issues with provisioning a new integration-agent-qemu in... [21:35:29] !log registered integration-agent-qemu-1005 with jenkins. note the debian-11 image was copied from integration-agent-qemu-1004 due to build failures (T440670) [21:35:32] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:35:32] T440670: Upgrade integration-agent-docker instances to trixie - https://phabricator.wikimedia.org/T440670 [21:36:15] !log Decommission deployment-ms-fe04 - T436476 [21:36:17] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:36:18] T436476: Replace Swift hosts: deployment-ms-(be0[78]|fe04) (Bullseye deprecation) - https://phabricator.wikimedia.org/T436476 [21:36:52] !log deleting integration-agent-qemu-1004 agent and instance following its replacement (T440670) [21:36:55] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:39:17] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team (Doing 😎), 10Release Pipeline: Upgrade integration-agent-docker instances to trixie - https://phabricator.wikimedia.org/T440670#12524842 (10hashar) @dduvall yeah copying the qcow2 image is good enough. I don't think we should bother updating... [21:41:41] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team (Doing 😎), 10Release Pipeline: Upgrade integration-agent-docker instances to trixie - https://phabricator.wikimedia.org/T440670#12524846 (10dduvall) 05Openβ†’03Resolved Thanks, @hashar! Resolved once again. Note the following remainin... [21:43:22] 10Beta-Cluster-Infrastructure, 10Cloud-VPS (Debian Bullseye Deprecation), 13Patch-For-Review: Replace Swift hosts: deployment-ms-(be0[78]|fe04) (Bullseye deprecation) - https://phabricator.wikimedia.org/T436476#12524853 (10Southparkfan) 05In progressβ†’03Stalled Other than merging https://gerrit.wikimedia.... [21:44:12] Five bullseye instances left, we're getting there... [21:49:11] 10GitLab (CI & Job Runners), 06Release-Engineering-Team, 07Essential-Work: Buildkit v0.34.2 released - https://phabricator.wikimedia.org/T443845 (10dancy) 03NEW [21:58:59] Southparkfan: nice job and huge thanks! [21:59:13] (03open) 10dancy: staging/staging.tfvars: Use buildkitd wmf-v0.34.2 [auto-deploy] [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/659 (https://phabricator.wikimedia.org/T443845) [21:59:55] 06Release-Engineering-Team (Doing 😎), 10Release Pipeline, 13Patch-For-Review: ST6.1.4 Refactor Pipelinelib to use buildx and generate SBOMs - https://phabricator.wikimedia.org/T440668#12524948 (10Dzahn) @dduvall Your change is deployed on contint1003/contint2003 and puppet has edited the config. But as exp... [22:00:20] (03approved) 10dduvall: image: Add `BUILD_CACHE_EXPORT_MODE` to set the cache export mode [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/169 (owner: 10dancy) [22:00:41] 06Release-Engineering-Team (Doing 😎), 06Collaboration-Services, 10Release Pipeline, 13Patch-For-Review: ST6.1.4 Refactor Pipelinelib to use buildx and generate SBOMs - https://phabricator.wikimedia.org/T440668#12524949 (10Dzahn) [22:02:47] (03merge) 10dancy: staging/staging.tfvars: Use buildkitd wmf-v0.34.2 [auto-deploy] [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/659 (https://phabricator.wikimedia.org/T443845) [22:03:16] !log doing `docker system prune -af` followed by `systemctl restart docker` on contint1003.wikimedia.org to enable the containerd-snapshotter (T440668) [22:03:19] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:03:19] T440668: ST6.1.4 Refactor Pipelinelib to use buildx and generate SBOMs - https://phabricator.wikimedia.org/T440668 [22:04:07] dduvall: Thanks :-) [22:04:32] +1 :) [22:04:59] (03open) 10addshore: Draft: Add initial Enterprise commands [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/800 [22:05:18] !log doing `docker system prune -af` followed by `systemctl restart docker` on contint2003.wikimedia.org to enable the containerd-snapshotter (T440668) [22:05:20] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:12:59] (03open) 10dancy: production/prod.tfvars: Use buildkitd wmf-v0.34.2 [auto-deploy] [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/660 (https://phabricator.wikimedia.org/T443845) [22:13:26] (03update) 10dancy: production/prod.tfvars: Use buildkitd wmf-v0.34.2 [auto-deploy] [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/660 (https://phabricator.wikimedia.org/T443845) [22:13:48] (03merge) 10dancy: production/prod.tfvars: Use buildkitd wmf-v0.34.2 [auto-deploy] [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/660 (https://phabricator.wikimedia.org/T443845) [22:14:17] (03update) 10dancy: image: Add `BUILD_CACHE_EXPORT_MODE` to set the cache export mode [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/169 [22:14:31] (03merge) 10dancy: image: Add `BUILD_CACHE_EXPORT_MODE` to set the cache export mode [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/169 [22:24:13] 10GitLab, 06Release-Engineering-Team (Doing 😎), 10a Wikimedia CLI: Rename releng/mwcli and releng/mwcli gitlab repo - https://phabricator.wikimedia.org/T384781#12525045 (10Addshore) **Names** So I believe I'll move the main CLI to: https://gitlab.wikimedia.org/repos/a-wikimedia-cli/wm This leaves space to... [22:30:30] 10GitLab (CI & Job Runners), 06Release-Engineering-Team, 07Essential-Work, 13Patch-For-Review: Buildkit v0.34.2 released - https://phabricator.wikimedia.org/T443845#12525086 (10dancy) [22:30:43] !log Updated gitlab-cloud-runners buildkit to v0.34.2 (staging and production) [22:30:44] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:40:32] (03open) 10addshore: Include CLI versions in User-Agent identities [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/801 [22:43:30] (03update) 10catrope: Add a script for checking which security patches can't be backported [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/282 [22:51:06] FIRING: PuppetStaleCertificates: Found non-revoked Puppet certificates for 1 deleted instances on deployment-puppetserver-1 - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/PuppetStaleCertificates - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetStaleCertificates [23:01:02] 10Beta-Cluster-Infrastructure, 10Cloud-VPS (Debian Bullseye Deprecation): Replace deployment-echostore02 (Bullseye deprecation) - https://phabricator.wikimedia.org/T436479#12525178 (10Southparkfan) `deployment-echostore02` has an interesting history. It looks like that back in 2024, I forgot to create that ins... [23:17:56] (03open) 10dancy: version: 2.18.0 [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/172 [23:46:22] 10Beta-Cluster-Infrastructure, 10Cloud-VPS, 06tools-infrastructure-team, 13Patch-For-Review, 07Puppet: Cannot rollout Trixie hosts in deployment-prep without rm -rf /var/lib/puppet/ssl - https://phabricator.wikimedia.org/T443373#12525336 (10Southparkfan) >As far as I know, agents on Bookworm hosts do not...