[00:08:28] 10Beta-Cluster-Infrastructure, 10Cloud-VPS (Debian Bullseye Deprecation): Replace deployment-echostore02 (Bullseye deprecation) - https://phabricator.wikimedia.org/T436479#12525383 (10Southparkfan) Fortunately, the process of setting up Cassandra is not as involved as in T436471#12519535 and T370461#10007736.... [00:15:18] !log Decommission deployment-echostore02 - T436479 [00:15:20] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [00:15:20] T436479: Replace deployment-echostore02 (Bullseye deprecation) - https://phabricator.wikimedia.org/T436479 [00:21:24] 10Beta-Cluster-Infrastructure, 10Cloud-VPS (Debian Bullseye Deprecation), 13Patch-For-Review: Replace deployment-echostore02 (Bullseye deprecation) - https://phabricator.wikimedia.org/T436479#12525401 (10Southparkfan) 05In progress→03Resolved [00:39:00] Project mediawiki-core-doxygen build #24081: 04FAILURE in 9 min 57 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen/24081/ [00:47:24] Yippee, build fixed! [00:47:25] Project mediawiki-core-doxygen build #24082: 09FIXED in 8 min 23 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen/24082/ [01:54:14] (03open) 10mhurd: Add a dashboard for build time saved by the cache proxy [repos/releng/develstats] - 10https://gitlab.wikimedia.org/repos/releng/develstats/-/merge_requests/103 [02:26:48] 10Continuous-Integration-Infrastructure, 10Gerrit, 10Projects-Cleanup, 10Wikimedia-GitHub: Extension archival triage - https://phabricator.wikimedia.org/T443861#12525682 (10Pppery) (My goal in compiling this list was to hopefully cajole somebody into doing it once it's clear what needed to be done rather t... [02:29:06] 10Continuous-Integration-Infrastructure, 10Gerrit, 10Projects-Cleanup, 10Wikimedia-GitHub: Extension archival triage - https://phabricator.wikimedia.org/T443861#12525685 (10Pppery) [03:56:32] 10GitLab, 10Bitu, 06Infrastructure-Foundations: After changing an email address, Bitu suggests logging out of Gitlab using a link that 404s - https://phabricator.wikimedia.org/T443753#12525761 (10AntiCompositeNumber) [06:30:36] 06Project-Admins: Create project tag for CollabPatrol - https://phabricator.wikimedia.org/T443867 (10Muffy) 03NEW [07:28:44] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team (Doing 😎): Puppet fail on integration-agent-qemu-1004: due to ci-build-images - https://phabricator.wikimedia.org/T436775#12525930 (10hashar) 05Open→03Resolved That got solved earlier in September, I had the task left open pending the... [07:33:56] 10Continuous-Integration-Infrastructure: Upgrade CI QEmu image from Bullseye to Trixie - https://phabricator.wikimedia.org/T443872 (10hashar) 03NEW [07:35:16] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team (Doing 😎), 10Release Pipeline: Upgrade integration-agent-docker instances to trixie - https://phabricator.wikimedia.org/T440670#12525947 (10hashar) Thx! Cumin, there might be a task about it already. For the QEMu image, I filed T443872... [07:59:23] (03PS1) 10Hashar: jjb: upgrade QEmu image from Bullseye to Trixie [integration/config] - 10https://gerrit.wikimedia.org/r/1365867 (https://phabricator.wikimedia.org/T443872) [08:08:58] 10Continuous-Integration-Infrastructure, 13Patch-For-Review: Upgrade CI QEmu image from Bullseye to Trixie - https://phabricator.wikimedia.org/T443872#12525987 (10hashar) The Qemu spawns thumbles fingers waiting to connect. Looking at the logs: ` kex_exchange_identification: read: Connection reset by peer Con... [08:21:40] (03PS2) 10Hashar: jjb: upgrade QEmu image from Bullseye to Trixie [integration/config] - 10https://gerrit.wikimedia.org/r/1365867 (https://phabricator.wikimedia.org/T443872) [08:21:41] (03PS1) 10Hashar: jjb: qemu: give up connecting after some time [integration/config] - 10https://gerrit.wikimedia.org/r/1365900 [08:38:14] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for Shivnagi - https://phabricator.wikimedia.org/T443877 (10Shivnagi) 03NEW [09:45:24] (03open) 10jnuche: housekeeping: base image version update [repos/releng/scap3-dev] - 10https://gitlab.wikimedia.org/repos/releng/scap3-dev/-/merge_requests/48 [09:45:38] (03merge) 10jnuche: housekeeping: base image version update [repos/releng/scap3-dev] - 10https://gitlab.wikimedia.org/repos/releng/scap3-dev/-/merge_requests/48 [10:30:36] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for Shivnagi - https://phabricator.wikimedia.org/T443877#12526440 (10Aklapper) Thanks for your interest! Which project(s) hosted in GitLab do you plan to work on? [10:35:01] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for Shivnagi - https://phabricator.wikimedia.org/T443877#12526460 (10Shivnagi) Hi @Aklapper, I am an Outreachy applicant and I plan to contribute to the Blubber project (https://gitlab.wikimedia.org/repos/releng/blub... [10:53:34] could someone with enough github buttons please fix the typo ('mirorr') in the description of https://github.com/wikimedia/xtools ? [11:02:44] 06Project-Admins: Create project tag for CollabPatrol - https://phabricator.wikimedia.org/T443867#12526549 (10Aklapper) Hi, do you plan to disable issue tracking on Github after creation of a project in Phabricator? [11:11:34] 10Continuous-Integration-Infrastructure, 10Cloud-VPS (Debian Bullseye Deprecation): Re-build integration-cumin.integration.eqiad1.wikimedia.cloud on something newer than bullseye - https://phabricator.wikimedia.org/T433592#12526564 (10Jdforrester-WMF) a:03hashar [11:47:29] 10GitLab (Account Approval), 06Release-Engineering-Team (Doing 😎): Requesting GitLab account activation for Shivnagi - https://phabricator.wikimedia.org/T443877#12526687 (10Aklapper) 05Open→03Resolved a:03Aklapper Great! Account approved. [11:51:04] 06Release-Engineering-Team (Doing 😎), 06WMF-NDA-Requests: Add FSesini-WMF to the WMF-NDA Phab group - https://phabricator.wikimedia.org/T443779#12526707 (10Aklapper) @FSesini-WMF: Unrelated, but can you please also [link your LDAP account to your Phabricator account](https://phabricator.wikimedia.org/setti... [11:55:43] 06Release-Engineering-Team (Doing 😎), 06WMF-NDA-Requests: Add FSesini-WMF to the WMF-NDA Phab group - https://phabricator.wikimedia.org/T443779#12526719 (10FSesini-WMF) @Aklapper thanks for letting me know, I'd missed that step. I've done that now! [12:05:06] FIRING: [2x] PuppetAgentFailure: Puppet agent failure detected on instance deployment-cp-text09 in project deployment-prep - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentFailure [12:56:19] 10GitLab (Project and group requests), 06Release-Engineering-Team (Doing 😎), 10WMF security vulnerability pipeline, 05FY25-26 DE4.4, 06Product Safety and Integrity (Sprint Chrysanthemum (Nov 2 - Nov 20)): GitLab Private Repository Request for: repos/pro... - https://phabricator.wikimedia.org/T443448#12526907 [13:05:17] 06Release-Engineering-Team (Doing 😎), 06WMF-NDA-Requests: WMF-NDA access request for Ammarpad - https://phabricator.wikimedia.org/T443901#12526975 (10Aklapper) 05Open→03Resolved a:03Aklapper Hi, your Phab account is linked to your LDAP account, and your LDAP account is listed on https://ldap.toolforg... [13:46:41] 10Continuous-Integration-Infrastructure, 10Gerrit, 10Projects-Cleanup, 10Wikimedia-GitHub: Extension archival triage - https://phabricator.wikimedia.org/T443861#12527143 (10Aklapper) [13:53:05] FIRING: PuppetAgentStaleLastRun: Last Puppet run was over 24 hours ago on instance deployment-webperf31 in project deployment-prep - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentStaleLastRun [13:53:11] 10Beta-Cluster-Infrastructure: Last Puppet run was over 24 hours ago on instance deployment-webperf31 in project deployment-prep - https://phabricator.wikimedia.org/T443914 (10wmcs-alerts) 03NEW [13:53:57] 10Continuous-Integration-Infrastructure, 13Patch-For-Review: Upgrade CI QEmu image from Bullseye to Trixie - https://phabricator.wikimedia.org/T443872#12527162 (10hashar) 05Open→03Declined I could not figure out how to connect to the `generic`, `genericcloud` or `nocloud` images. The first two have op... [14:34:06] FIRING: PuppetAgentNoResources: No Puppet resources found on instance deployment-webperf31 on project deployment-prep - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentNoResources [15:04:16] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 13Patch-For-Review: Migrate all CI jobs from Bullseye to Bookworm or later and drop Bullseye testing support - https://phabricator.wikimedia.org/T383337#12527457 (10Jdforrester-WMF) [15:04:20] 10Continuous-Integration-Config, 06Release-Engineering-Team, 06SRE, 13Patch-For-Review, 07Puppet: Upgrade operations/puppet CI from Bullseye to Bookworm - https://phabricator.wikimedia.org/T443798#12527456 (10Jdforrester-WMF) [15:12:47] 06Project-Admins: Create project tag for CollabPatrol - https://phabricator.wikimedia.org/T443867#12527523 (10Muffy) Hi ! Yes we plan it, we will disable issue tracking on GitHub as soon as the Phabricator project is created ^^. Thanks ! [15:15:20] (03CR) 10Hashar: Respect environment overrides in browser tests and custom commands (031 comment) [integration/quibble] - 10https://gerrit.wikimedia.org/r/1361209 (https://phabricator.wikimedia.org/T252883) (owner: 10Addshore) [15:23:37] (03open) 10shivnagi: docs: Add section on image layer and cache efficiency [repos/releng/blubber] - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/189 (https://phabricator.wikimedia.org/T268780) [15:33:29] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 13Patch-For-Review: Migrate all CI jobs from Bullseye to Bookworm or later and drop Bullseye testing support - https://phabricator.wikimedia.org/T383337#12527624 (10hashar) [15:33:32] 10Continuous-Integration-Config, 06Release-Engineering-Team, 06SRE, 13Patch-For-Review, 07Puppet: Upgrade operations/puppet CI from Bullseye to Bookworm - https://phabricator.wikimedia.org/T443798#12527623 (10hashar) [15:36:46] James_F: the reparenting out of Bullseye is almost done :) [15:36:58] logstash-filter-verifier can be reparented now that packages are available [15:37:22] for operations-puppet there is a long tail of fix to get pushed in operations/puppet . I'll poke SREs next week to have them reviewed/merged [15:37:32] I am off! [15:44:53] 10GitLab (CI & Job Runners), 06Release-Engineering-Team, 07Essential-Work: Buildkit v0.34.2 released - https://phabricator.wikimedia.org/T443845#12527676 (10dancy) 05Open→03Resolved p:05Triage→03Medium a:03dancy [16:05:26] (03merge) 10dancy: version: 2.18.0 [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/172 [16:13:28] (03open) 10dancy: CI: Cache all layers of the test jobs with `BUILD_CACHE_EXPORT_MODE` [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1417 [16:14:06] (03update) 10dancy: CI: Cache all layers of the test jobs with `BUILD_CACHE_EXPORT_MODE` [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1417 [16:14:20] 10Continuous-Integration-Infrastructure, 10Gerrit, 10Projects-Cleanup, 10Wikimedia-GitHub: Extension archival triage - https://phabricator.wikimedia.org/T443861#12527844 (10Pppery) [16:16:44] (03merge) 10dancy: CI: Cache all layers of the test jobs with `BUILD_CACHE_EXPORT_MODE` [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1417 [16:25:24] (03open) 10dancy: spiderpig: Report the changes of each unfinished job in the event stream [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1418 [16:26:10] (03update) 10dancy: spiderpig: Report the changes of each unfinished job in the event stream [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1418 [16:26:17] (03update) 10dancy: spiderpig: Report the changes of each unfinished job in the event stream [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1418 [16:28:39] (03merge) 10dancy: spiderpig: Report the changes of each unfinished job in the event stream [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1418 [16:39:50] 10Beta-Cluster-Infrastructure, 10CFSSL-PKI, 06Infrastructure-Foundations, 13Patch-For-Review: profile::pki::client does not hash symlinks and does not populate ca-certificates.crt - https://phabricator.wikimedia.org/T436562#12528050 (10dr0ptp4kt) @Southparkfan I'm not in the loop if you're connected with a... [17:07:03] (03open) 10project_16_bot_622d7b6d682aa66907ad1bffedf5b9e6: build(deps): bump github.com/spf13/pflag from 1.0.10 to 1.1.0 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/802 [17:38:42] 06Release-Engineering-Team, 06MediaWiki-API-Platform-Team, 06ServiceOps, 05FY2025-26 KR 5.1, and 2 others: rest-gateway: run system tests via helm test - https://phabricator.wikimedia.org/T424825#12528402 (10JTweed-WMF) Sounds good to me, I'll let @HCoplin-WMF move back if she disagrees! [17:56:18] 06Release-Engineering-Team (Radar), 07Essential-Work, 10Test Kitchen (Experiment Platform Sprint 32): Determine if we can expand TK experiment deployment windows - https://phabricator.wikimedia.org/T440612#12528522 (10JVanderhoop-WMF) Doing a bit of PST -> UTC conversion, and using Monday as an example: * L... [18:14:28] 10Beta-Cluster-Infrastructure, 10Openstack-Magnum, 06tools-infrastructure-team: Rebuild the deployment-wikikube-v127 magnum cluster with the new magnum-cluster-api driver and k8s v1.34 - https://phabricator.wikimedia.org/T437500#12528627 (10Andrew) [18:26:57] 10Beta-Cluster-Infrastructure, 10Openstack-Magnum, 06tools-infrastructure-team: Rebuild the deployment-wikikube-v127 magnum cluster with the new magnum-cluster-api driver and k8s v1.34 - https://phabricator.wikimedia.org/T437500#12528674 (10bd808) a:03bd808 [18:28:48] (03open) 10matmarex: Add make-release helpers check-gpg-sigs.sh and compare-archive.sh [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/283 [18:31:19] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06Release-Engineering-Team (Priority Backlog 📥), 10Openstack-Magnum, 06tools-infrastructure-team: Rebuild the cloud-runner-staging magnum cluster with the new magnum-cluster-api driver - https://phabricator.wikimedia.org/T437498#12528687 (10bd808) [18:31:46] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06Release-Engineering-Team (Priority Backlog 📥), 10Openstack-Magnum, 06tools-infrastructure-team: Rebuild the cloud-runner-staging magnum cluster in the zuul3 project with the new magnum-cluster-api d... - https://phabricator.wikimedia.org/T437498#12528689 [18:32:32] 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team (Priority Backlog 📥), 10Openstack-Magnum, 06tools-infrastructure-team: Rebuild the deployment-wikikube-v127 magnum cluster with the new magnum-cluster-api driver and k8s v1.34 - https://phabricator.wikimedia.org/T437500#12528690 (10bd808) [18:33:55] 10Beta-Cluster-Infrastructure, 10CFSSL-PKI, 06Infrastructure-Foundations, 13Patch-For-Review: profile::pki::client does not hash symlinks and does not populate ca-certificates.crt - https://phabricator.wikimedia.org/T436562#12528697 (10Southparkfan) >>! In T436562#12528050, @dr0ptp4kt wrote: > @Southparkfa... [18:35:20] bd808: +1 for rebuilding the k8s cluster [18:36:29] Southparkfan: I think I can start on that today. Let's see. [18:37:09] (03update) 10matmarex: Add make-release helpers check-gpg-sigs.sh and compare-archive.sh [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/283 [18:37:13] Do you want to move certain services currently hosted in Docker to that cluster? [18:37:26] (03update) 10matmarex: Add make-release helpers check-gpg-sigs.sh and compare-archive.sh [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/283 [18:38:16] Southparkfan: That's my long term hope, yes. Letting us kill off all those other vms and hopefully figure out how to reuse the helm charts from prod. [18:39:04] 10MediaWiki-Releasing, 05MW-1.47-release: Write release announcement for 1.47.0-rc.0 - https://phabricator.wikimedia.org/T441082#12528705 (10MSantos) 05Open→03Resolved [18:39:16] Southparkfan: I have to run for an appointment, but if there are any issues I should watch out for when using the tofu automation I would appreciate an update. [18:41:04] 10Beta-Cluster-Infrastructure, 10Citoid: Update citoid service on beta - https://phabricator.wikimedia.org/T380165#12528708 (10Southparkfan) 05Open→03Resolved a:03Southparkfan @Mvolz, as part of T442482, I have upgraded Citoid to `2026-10-23-150123-production`. Hopefully, this addresses the update co... [18:42:15] bd808: The OpenTofu automation still works fine, feel free to add VMs as needed. [18:42:40] (I have to leave soon too, though) [19:15:48] 10GitLab (CI & Job Runners): Upgrade gitlab-cloud-runners kubernetes - https://phabricator.wikimedia.org/T443969 (10dancy) 03NEW [19:19:06] RESOLVED: PuppetAgentNoResources: No Puppet resources found on instance deployment-webperf31 on project deployment-prep - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentNoResources [19:23:05] RESOLVED: PuppetAgentStaleLastRun: Last Puppet run was over 24 hours ago on instance deployment-webperf31 in project deployment-prep - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentStaleLastRun [19:24:42] !log Upgrading gitlab-cloud-runners (staging) from 1.36.3-do.5 to 1.36.5-do.3 (T443969) [19:24:44] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:24:45] T443969: Upgrade gitlab-cloud-runners kubernetes - https://phabricator.wikimedia.org/T443969 [19:38:05] !log Upgraded gitlab-cloud-runners (staging) from 1.36.3-do.5 to 1.36.5-do.3 (T443969) [19:38:07] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:38:07] T443969: Upgrade gitlab-cloud-runners kubernetes - https://phabricator.wikimedia.org/T443969 [19:39:10] !log Upgrading gitlab-cloud-runners (prod) from 1.36.3-do.5 to 1.36.5-do.3 (T443969) [19:39:13] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:54:04] !log Upgraded gitlab-cloud-runners (prod) from 1.36.3-do.5 to 1.36.5-do.3 (T443969) [19:54:06] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:54:07] T443969: Upgrade gitlab-cloud-runners kubernetes - https://phabricator.wikimedia.org/T443969 [19:55:21] (03open) 10dancy: staging/digitalocean.tfvars: kubernetes_version = "1.36.5-do.3" [auto-deploy] [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/661 (https://phabricator.wikimedia.org/T443969) [19:55:41] (03update) 10dancy: staging/digitalocean.tfvars: kubernetes_version = "1.36.5-do.3" [auto-deploy] [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/661 (https://phabricator.wikimedia.org/T443969) [19:56:33] (03merge) 10dancy: staging/digitalocean.tfvars: kubernetes_version = "1.36.5-do.3" [auto-deploy] [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/661 (https://phabricator.wikimedia.org/T443969) [19:58:51] (03open) 10dancy: production/digitalocean.tfvars: kubernetes_version = "1.36.5-do.3" [auto-deploy] [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/662 (https://phabricator.wikimedia.org/T443969) [20:06:28] (03merge) 10dancy: production/digitalocean.tfvars: kubernetes_version = "1.36.5-do.3" [auto-deploy] [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/662 (https://phabricator.wikimedia.org/T443969) [20:08:41] (03open) 10dancy: spiderpig: Job: Hash whole rows in `Job.unfinished_digests()` [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1419 [20:15:05] (03update) 10dancy: spiderpig: Job: Hash whole rows in `Job.unfinished_digests()` [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1419 [20:18:08] (03merge) 10dancy: spiderpig: Job: Hash whole rows in `Job.unfinished_digests()` [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1419 [20:53:21] 06Release-Engineering-Team (Radar), 07Essential-Work, 10Test Kitchen (Experiment Platform Sprint 32): Determine if we can expand TK experiment deployment windows - https://phabricator.wikimedia.org/T440612#12529000 (10Jdlrobson-WMF) I see... Is there a world where I can manually make the Test Kitchen changes... [20:56:22] (03update) 10mhurd: Add a dashboard for build time saved by the cache proxy [repos/releng/develstats] - 10https://gitlab.wikimedia.org/repos/releng/develstats/-/merge_requests/103 [20:59:40] (03update) 10mhurd: Add a dashboard for moving from Castor to the cache proxy [repos/releng/develstats] - 10https://gitlab.wikimedia.org/repos/releng/develstats/-/merge_requests/103 [20:59:50] (03open) 10dancy: spiderpig: Fetch jobs again only when the event stream reports a change [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1420 [21:00:52] (03update) 10dancy: spiderpig: Fetch jobs again only when the event stream reports a change [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1420 [21:01:40] (03update) 10dancy: istio/main.tf: Bump Istio to version 1.31.3 [auto-deploy] [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/663 (https://phabricator.wikimedia.org/T443969) [21:01:41] (03open) 10dancy: istio/main.tf: Bump Istio to version 1.31.3 [auto-deploy] [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/663 (https://phabricator.wikimedia.org/T443969) [21:02:02] (03merge) 10dancy: spiderpig: Fetch jobs again only when the event stream reports a change [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1420 [21:02:05] (03update) 10dancy: istio/main.tf: Bump Istio to version 1.31.3 [auto-deploy] [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/663 (https://phabricator.wikimedia.org/T443969) [21:03:02] (03merge) 10dancy: istio/main.tf: Bump Istio to version 1.31.3 [auto-deploy] [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/663 (https://phabricator.wikimedia.org/T443969) [21:03:17] (03update) 10mhurd: Add a dashboard for moving from Castor to the cache proxy [repos/releng/develstats] - 10https://gitlab.wikimedia.org/repos/releng/develstats/-/merge_requests/103 [21:08:45] 10Phabricator Translations Extension: Make WMF translations extension pass `bin/i18n validate` - https://phabricator.wikimedia.org/T412651#12529029 (10Pppery) After https://gerrit.wikimedia.org/r/c/phabricator/translations/+/1366738 ` No validation errors found! ` Now all that needs to happen is for that to be... [22:19:27] !log Upgraded gitlab-cloud-runners (staging) from 1.36.5-do.3 to 1.37.0-do.3 (T443969) [22:19:30] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:19:30] T443969: Upgrade gitlab-cloud-runners kubernetes - https://phabricator.wikimedia.org/T443969 [22:34:03] !log Upgraded gitlab-cloud-runners (prod) from 1.36.5-do.3 to 1.37.0-do.3 (T443969) [22:34:05] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:34:05] T443969: Upgrade gitlab-cloud-runners kubernetes - https://phabricator.wikimedia.org/T443969 [22:36:56] (03open) 10dancy: digitalocean.tfvars: kubernetes_version = "1.37.0-do.3" [auto-deploy] [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/664 (https://phabricator.wikimedia.org/T443969) [22:37:20] (03update) 10dancy: digitalocean.tfvars: kubernetes_version = "1.37.0-do.3" [auto-deploy] [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/664 (https://phabricator.wikimedia.org/T443969) [22:38:01] (03merge) 10dancy: digitalocean.tfvars: kubernetes_version = "1.37.0-do.3" [auto-deploy] [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/664 (https://phabricator.wikimedia.org/T443969) [22:51:06] FIRING: PuppetStaleCertificates: Found non-revoked Puppet certificates for 1 deleted instances on deployment-puppetserver-1 - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/PuppetStaleCertificates - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetStaleCertificates [22:53:16] (03merge) 10brennen: Update composer from 2.10.2 to 2.11.0 in all images [repos/releng/dev-images] - 10https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/135 (https://phabricator.wikimedia.org/T440297 https://phabricator.wikimedia.org/T442976) (owner: 10jforrester) [22:55:21] !log Updating development images on contint primary for https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/135 (T440297, T442976) [22:55:25] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:55:26] T440297: Update composer from 2.10.2 to 2.10.3 - https://phabricator.wikimedia.org/T440297 [22:55:26] T442976: Update composer from 2.10.3 to 2.11.0 - https://phabricator.wikimedia.org/T442976 [23:00:15] 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team (Priority Backlog 📥), 10Cloud-VPS (Debian Bullseye Deprecation), 07Epic, 13Patch-For-Review: Migrate deployment-prep away from Debian Bullseye to Bookworm/Trixie - https://phabricator.wikimedia.org/T401839#12529233 (10bd808) 05Open→03In progress [23:00:22] 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team (Priority Backlog 📥), 10Cloud-VPS (Debian Bullseye Deprecation), 07Epic, 13Patch-For-Review: Migrate deployment-prep away from Debian Bullseye to Bookworm/Trixie - https://phabricator.wikimedia.org/T401839#12529235 (10bd808) p:05Triage→03High [23:02:18] 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team (Priority Backlog 📥), 10Openstack-Magnum, 06tools-infrastructure-team: Rebuild the deployment-wikikube-v127 magnum cluster with the new magnum-cluster-api driver and k8s v1.34 - https://phabricator.wikimedia.org/T437500#12529248 (10bd808) [23:05:30] 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team (Priority Backlog 📥), 10Openstack-Magnum, 06tools-infrastructure-team: Rebuild the deployment-wikikube-v127 magnum cluster with the new magnum-cluster-api driver and k8s v1.36 - https://phabricator.wikimedia.org/T437500#12529251 (10bd808) [23:30:44] (03update) 10mhurd: Add a dashboard for moving from Castor to the cache proxy [repos/releng/develstats] - 10https://gitlab.wikimedia.org/repos/releng/develstats/-/merge_requests/103 [23:31:25] (03update) 10mhurd: Add a dashboard for moving from Castor to the cache proxy [repos/releng/develstats] - 10https://gitlab.wikimedia.org/repos/releng/develstats/-/merge_requests/103