[01:58:28] FIRING: InstanceDown: Project deployment-prep instance deployment-cache-text08 is down - https://prometheus-alerts.wmcloud.org/?q=alertname%3DInstanceDown [01:58:33] 10Beta-Cluster-Infrastructure: Project deployment-prep instance deployment-cache-text08 is down - https://phabricator.wikimedia.org/T400775 (10wmcs-alerts) 03NEW [02:03:28] RESOLVED: InstanceDown: Project deployment-prep instance deployment-cache-text08 is down - https://prometheus-alerts.wmcloud.org/?q=alertname%3DInstanceDown [02:03:37] 10Beta-Cluster-Infrastructure: Project deployment-prep instance deployment-cache-text08 is down - https://phabricator.wikimedia.org/T400775#11045598 (10wmcs-alerts) [05:43:53] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: Gitlab switchover (gitlab2002 → gitlab1004) - https://phabricator.wikimedia.org/T400252#11045733 (10Jelto) [05:46:10] GitLab maintenance starting soon 06:00–08:00 UTC, see T400252 for more details. I'll let you know when the maintenance is finished. [05:46:10] T400252: Gitlab switchover (gitlab2002 → gitlab1004) - https://phabricator.wikimedia.org/T400252 [05:47:01] (03merge) 10jelto: switch gitlab production to gitlab1004 [repos/releng/gitlab-settings] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-settings/-/merge_requests/75 (https://phabricator.wikimedia.org/T400252) [05:53:22] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: Gitlab switchover (gitlab2002 → gitlab1004) - https://phabricator.wikimedia.org/T400252#11045748 (10Jelto) [06:36:05] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: Gitlab switchover (gitlab2002 → gitlab1004) - https://phabricator.wikimedia.org/T400252#11045780 (10Jelto) [06:37:24] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: Gitlab switchover (gitlab2002 → gitlab1004) - https://phabricator.wikimedia.org/T400252#11045781 (10ops-monitoring-bot) Cookbook cookbooks.sre.gitlab.failover (Failover of gitlab from gitlab2002.wikimedia.org to gitlab1004.wikimedia.org... [06:37:54] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: Gitlab switchover (gitlab2002 → gitlab1004) - https://phabricator.wikimedia.org/T400252#11045782 (10Jelto) [07:11:00] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: Gitlab switchover (gitlab2002 → gitlab1004) - https://phabricator.wikimedia.org/T400252#11045817 (10Jelto) [07:11:09] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: Gitlab switchover (gitlab2002 → gitlab1004) - https://phabricator.wikimedia.org/T400252#11045818 (10Jelto) [07:48:28] FIRING: [2x] PuppetAgentFailure: Puppet agent failure detected on instance deployment-ms-fe04 in project deployment-prep - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentFailure [08:01:28] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: Gitlab switchover (gitlab2002 → gitlab1004) - https://phabricator.wikimedia.org/T400252#11045870 (10Jelto) [08:03:28] FIRING: [2x] PuppetAgentFailure: Puppet agent failure detected on instance deployment-ms-fe04 in project deployment-prep - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentFailure [08:03:32] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: Gitlab switchover (gitlab2002 → gitlab1004) - https://phabricator.wikimedia.org/T400252#11045873 (10ops-monitoring-bot) Cookbook cookbooks.sre.gitlab.failover (Failover of gitlab from gitlab2002.wikimedia.org to gitlab1004.wikimedia.org... [08:04:28] RESOLVED: [2x] PuppetAgentFailure: Puppet agent failure detected on instance deployment-ms-fe04 in project deployment-prep - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentFailure [08:16:56] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: Gitlab switchover (gitlab2002 → gitlab1004) - https://phabricator.wikimedia.org/T400252#11045935 (10Jelto) [08:17:59] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: Gitlab switchover (gitlab2002 → gitlab1004) - https://phabricator.wikimedia.org/T400252#11045943 (10Jelto) The switchover to `gitlab1004` was successful. There are two missing packages: ` /var/opt/gitlab/gitlab-rails/shared/packages/... [08:40:05] (03PS1) 10Hashar: dockerfiles: remove java8-scala-spark [integration/config] - 10https://gerrit.wikimedia.org/r/1174399 [08:40:52] (03CR) 10Hashar: [C:03+2] "CC @adam.wight@wikimedia.de for information 😊" [integration/config] - 10https://gerrit.wikimedia.org/r/1174399 (owner: 10Hashar) [08:41:12] (03PS2) 10Hashar: dockerfiles: remove java8-scala-spark [integration/config] - 10https://gerrit.wikimedia.org/r/1174399 (https://phabricator.wikimedia.org/T397429) [08:41:56] (03CR) 10Hashar: [C:03+2] "I have amended the commit message to attach the change to T397429" [integration/config] - 10https://gerrit.wikimedia.org/r/1174399 (https://phabricator.wikimedia.org/T397429) (owner: 10Hashar) [08:43:33] (03Merged) 10jenkins-bot: dockerfiles: remove java8-scala-spark [integration/config] - 10https://gerrit.wikimedia.org/r/1174399 (https://phabricator.wikimedia.org/T397429) (owner: 10Hashar) [08:57:27] 10GitLab (Infrastructure), 06collaboration-services: Gitlab switchover (gitlab2002 → gitlab1004) - https://phabricator.wikimedia.org/T400252#11046027 (10Jelto) >>! In T400252#11045935, @Jelto wrote: > The switchover to `gitlab1004` was successful. > > There are two missing packages: > > ` > /var/opt/gitlab/... [08:59:25] 10GitLab (Infrastructure), 06collaboration-services: Gitlab switchover (gitlab2002 → gitlab1004) - https://phabricator.wikimedia.org/T400252#11046033 (10Jelto) [09:19:12] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: Gitlab switchover (gitlab2002 → gitlab1004) - https://phabricator.wikimedia.org/T400252#11046056 (10Jelto) [11:27:43] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: Gitlab switchover (gitlab2002 → gitlab1004) - https://phabricator.wikimedia.org/T400252#11046466 (10Jelto) [12:42:07] (03open) 10jelto: bump CI image to bookworm [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/125 (https://phabricator.wikimedia.org/T384595) [12:42:23] (03update) 10jelto: Draft: bump CI image to bookworm [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/125 (https://phabricator.wikimedia.org/T384595) [12:52:49] (03update) 10jelto: Draft: bump CI image to bookworm [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/125 (https://phabricator.wikimedia.org/T384595) [13:05:33] (03update) 10jelto: Draft: bump CI image to bookworm [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/125 (https://phabricator.wikimedia.org/T384595) [13:10:22] (03update) 10jelto: Draft: bump CI image to bookworm [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/125 (https://phabricator.wikimedia.org/T384595) [13:13:30] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: Gitlab switchover (gitlab2002 → gitlab1004) - https://phabricator.wikimedia.org/T400252#11046794 (10dcaro) Just for the record, I lost ipv4 connectivity for a few minutes twice today to `wikimedia.gitlab.com`, where the packets were bei... [13:19:58] (03update) 10jelto: Draft: bump CI image to bookworm [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/125 (https://phabricator.wikimedia.org/T384595) [13:24:14] (03update) 10jelto: Draft: bump CI image to bookworm [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/125 (https://phabricator.wikimedia.org/T384595) [13:37:07] (03CR) 10Jforrester: "\o/" [integration/config] - 10https://gerrit.wikimedia.org/r/1174399 (https://phabricator.wikimedia.org/T397429) (owner: 10Hashar) [13:39:28] (03update) 10jelto: Draft: bump CI image to bookworm [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/125 (https://phabricator.wikimedia.org/T384595) [14:04:59] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: Gitlab switchover (gitlab2002 → gitlab1004) - https://phabricator.wikimedia.org/T400252#11046950 (10Jelto) Thanks for reporting the issue @dcaro . It looks like the rate-limiting is more active on the eqiad host than on codfw. I'll disa... [14:37:26] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: Gitlab switchover (gitlab2002 → gitlab1004) - https://phabricator.wikimedia.org/T400252#11047112 (10Jelto) [14:56:34] 10Release-Engineering-Team (Doing 😎), 07OKR-Work: Move nightly image build from releases-jenkins to deployment.eqiad.wmnet - https://phabricator.wikimedia.org/T398873#11047209 (10dancy) 05Open→03Resolved This is live. [15:04:13] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for NIGHTFURY609 - https://phabricator.wikimedia.org/T400453#11047253 (10A_smart_kitten) 05Open→03Resolved a:03Jayprakash12345 Looks like your GitLab account got automatically approved ([[https://wikitech.wi... [15:09:38] 10GitLab (Account Approval): Requesting GitLab account activation for Aadhi13 - https://phabricator.wikimedia.org/T400498#11047270 (10A_smart_kitten) 05Open→03Resolved a:03Jayprakash12345 Looks like your GitLab account got automatically approved ([[https://wikitech.wikimedia.org/w/index.php?title=Tool:... [15:12:43] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation - jyothikat22 - https://phabricator.wikimedia.org/T400455#11047283 (10A_smart_kitten) 05Open→03Resolved a:03Jayprakash12345 Looks like your GitLab account got automatically approved ([[https://wikitech.wikim... [15:15:52] 10GitLab (Account Approval): Requesting GitLab account activation - flykrth - https://phabricator.wikimedia.org/T400451#11047296 (10A_smart_kitten) 05Open→03Resolved a:03Jayprakash12345 Looks like your GitLab account got automatically approved ([[https://wikitech.wikimedia.org/w/index.php?title=Tool:Gi... [15:16:15] 10GitLab (Support), 10Release-Engineering-Team (Doing 😎), 07Essential-Work: GitLab Private Repository Request for: research/npov-workstream-research - https://phabricator.wikimedia.org/T399696#11047300 (10Pablo) That works. Thank you, @thcipriani! [15:19:10] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for MalavikaM05 - https://phabricator.wikimedia.org/T400450#11047322 (10A_smart_kitten) 05Open→03Resolved a:03Jayprakash12345 Looks like your GitLab account got automatically approved ([[https://wikitech.wik... [15:22:20] 10GitLab (CI & Job Runners), 06Release-Engineering-Team, 06collaboration-services, 10mwbot-rs: Random GitLab CI failures with ContainersNotReady - https://phabricator.wikimedia.org/T380680#11047331 (10dancy) [16:00:13] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 07Jenkins, 10Wikifunctions, and 3 others: ${{SCRIPT, template="wikimedia.template"}} is not sending a correct template - https://phabricator.wikimedia.org/T400286#11047496 (10vaughnwalters) >>! In T400286#11029021, @Jdforrester-WMF w... [16:02:49] (03open) 10dancy: Bump CI image to bookworm [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/126 (https://phabricator.wikimedia.org/T384595) [16:02:52] (03update) 10dancy: Bump CI image to bookworm [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/126 (https://phabricator.wikimedia.org/T384595) [16:05:24] (03merge) 10dancy: Bump CI image to bookworm [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/126 (https://phabricator.wikimedia.org/T384595) [16:15:09] (03open) 10dancy: Bump CI image to bookworm [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/127 (https://phabricator.wikimedia.org/T384595) [16:15:12] (03update) 10dancy: Bump CI image to bookworm [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/127 (https://phabricator.wikimedia.org/T384595) [16:16:21] (03close) 10dancy: Bump CI image to bookworm [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/127 (https://phabricator.wikimedia.org/T384595) [16:16:32] (03open) 10dancy: Bump CI image to bookworm (followup) [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/128 (https://phabricator.wikimedia.org/T384595) [16:16:35] (03update) 10dancy: Bump CI image to bookworm (followup) [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/128 (https://phabricator.wikimedia.org/T384595) [16:26:05] 10Beta-Cluster-Infrastructure, 10MediaWiki-Releasing, 07Beta-Cluster-reproducible, 05MW-1.43-release, 10MW-Interfaces-Team (MWI-Sprint-15 (2025-07-29 to 2025-08-12)): Duplicate git tags for 1.43.0-rc.0 and 1.43.0 in mediawiki/core.git - https://phabricator.wikimedia.org/T400729#11047558 (10Atieno) [[ ht... [16:27:35] 10Beta-Cluster-Infrastructure, 10MediaWiki-Releasing, 07Beta-Cluster-reproducible, 05MW-1.43-release, 10MW-Interfaces-Team (MWI-Sprint-15 (2025-07-29 to 2025-08-12)): Duplicate git tags for 1.43.0-rc.0 and 1.43.0 in mediawiki/core.git - https://phabricator.wikimedia.org/T400729#11047564 (10Krinkle) ... [16:43:28] (03update) 10dancy: Bump CI image to bookworm (followup) [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/128 (https://phabricator.wikimedia.org/T384595) [16:44:49] (03update) 10dancy: Bump CI image to bookworm (followup) [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/128 (https://phabricator.wikimedia.org/T384595) [16:45:11] (03update) 10dancy: Bump CI image to bookworm (followup) [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/128 (https://phabricator.wikimedia.org/T384595) [16:46:37] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥): Support "other assignee" field for tasks with "Task" type - https://phabricator.wikimedia.org/T396905#11047604 (10brennen) 05Stalled→03Resolved After discussion, calling this good where it's at for now. If anyone would like the field visi... [16:48:13] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥): Support "other assignee" field for tasks with "Task" type - https://phabricator.wikimedia.org/T396905#11047607 (10Aklapper) Thanks! FYI getting a list all Forms which expose a (custom) field: `SELECT CONCAT("https://phabricator.wikimedia.org/m... [16:52:48] (03update) 10dancy: Bump CI image to bookworm (followup) [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/128 (https://phabricator.wikimedia.org/T384595) [16:53:56] (03update) 10dancy: Bump CI image to bookworm (followup) [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/128 (https://phabricator.wikimedia.org/T384595) [17:13:04] (03update) 10dancy: Bump CI image to bookworm (followup) [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/128 (https://phabricator.wikimedia.org/T384595) [17:13:10] (03update) 10dancy: Bump CI image to bookworm (followup) [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/128 (https://phabricator.wikimedia.org/T384595) [17:24:38] (03update) 10dancy: Bump CI image to bookworm (followup) [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/128 (https://phabricator.wikimedia.org/T384595) [17:26:11] (03merge) 10dancy: Bump CI image to bookworm (followup) [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/128 (https://phabricator.wikimedia.org/T384595) [17:26:48] (03close) 10dancy: Draft: bump CI image to bookworm [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/125 (https://phabricator.wikimedia.org/T384595) (owner: 10jelto) [17:39:44] (03open) 10bd808: kubernetes: Grant k8s_{admin, developer, viewer} roles [repos/releng/zuul/tofu-provisioning] - 10https://gitlab.wikimedia.org/repos/releng/zuul/tofu-provisioning/-/merge_requests/46 [17:41:49] (03merge) 10bd808: kubernetes: Grant k8s_{admin, developer, viewer} roles [repos/releng/zuul/tofu-provisioning] - 10https://gitlab.wikimedia.org/repos/releng/zuul/tofu-provisioning/-/merge_requests/46 [17:55:35] 10Gerrit, 10Release-Engineering-Team (Priority Backlog 📥): Folks no longer able to merge gerrit dashboard changes - https://phabricator.wikimedia.org/T400829 (10thcipriani) 03NEW [18:21:57] (03update) 10cdanis: add fault-tolerance [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/108 [18:55:38] (03merge) 10dancy: add fault-tolerance [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/108 (owner: 10cdanis) [18:59:10] (03update) 10dancy: add fault-tolerance [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/108 (owner: 10cdanis) [19:06:45] thanks <3 [19:47:25] 10Continuous-Integration-Infrastructure, 07Jenkins, 06Release-Engineering-Team, 06collaboration-services: dpkg error when deploying Jenkins to releases2003.codfw.wmnet - https://phabricator.wikimedia.org/T400645#11048174 (10Dzahn) I am confused about the latest comment. I had installed jenkins on relea... [19:48:11] 10Continuous-Integration-Infrastructure, 07Jenkins, 06Release-Engineering-Team, 06collaboration-services: dpkg error when deploying Jenkins to releases2003.codfw.wmnet - https://phabricator.wikimedia.org/T400645#11048175 (10Dzahn) 05Declined→03Resolved [19:50:21] 10Continuous-Integration-Infrastructure, 07Jenkins, 06Release-Engineering-Team, 06collaboration-services: dpkg error when deploying Jenkins to releases2003.codfw.wmnet - https://phabricator.wikimedia.org/T400645#11048180 (10bd808) a:03Dzahn [20:16:52] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for Swayam_Agrahari - https://phabricator.wikimedia.org/T400449#11048227 (10A_smart_kitten) 05Open→03Resolved Looks like your GitLab account got automatically approved ([[https://wikitech.wikimedia.org/w/index... [20:18:06] (03PS1) 10Pppery: Zuul: Add Nic Jansma to CI allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/1174547 [20:33:57] (03open) 10dancy: sync: Build "next" image when building regular images [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/925 (https://phabricator.wikimedia.org/T398875) [20:33:58] (03update) 10dancy: sync: Build "next" image when building regular images [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/925 (https://phabricator.wikimedia.org/T398875) [20:36:37] (03update) 10dancy: sync: Build "next" image when building regular images [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/925 (https://phabricator.wikimedia.org/T398875) [20:57:46] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for Chimnayyyy - https://phabricator.wikimedia.org/T400448#11048354 (10A_smart_kitten) 05Open→03Resolved a:03Jayprakash12345 Looks like your GitLab account got automatically approved ([[https://wikitech.wiki... [21:06:04] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation - HridyaKK - https://phabricator.wikimedia.org/T400447#11048383 (10A_smart_kitten) 05Open→03Resolved a:03Jayprakash12345 Looks like your GitLab account got automatically approved ([[https://wikitech.wikimedi... [21:22:20] 10Continuous-Integration-Infrastructure (Zuul upgrade): Provision user in Cloud VPS hosted Kubernetes cluster for use by nodepool - https://phabricator.wikimedia.org/T398367#11048435 (10bd808) 05In progress→03Resolved [21:34:11] (03open) 10dduvall: docs: Refactor README to focus on common usage patterns [repos/releng/blubber] - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/142 [21:34:24] (03close) 10dduvall: docs: Refactor README to focus on common usage patterns [repos/releng/blubber] - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/142 [21:34:49] (03update) 10dduvall: docs: Refactor README to focus on common usage patterns [repos/releng/blubber] - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/140 [21:36:54] (03update) 10dduvall: docs: Refactor README to focus on common usage patterns [repos/releng/blubber] - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/140 [21:41:52] (03approved) 10dduvall: sync: Build "next" image when building regular images [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/925 (https://phabricator.wikimedia.org/T398875) (owner: 10dancy) [21:49:45] 10Continuous-Integration-Infrastructure (Zuul upgrade): Document local dev/testing workflows for OpenTofu gitops automation - https://phabricator.wikimedia.org/T398368#11048489 (10bd808) 05Open→03Resolved a:03bd808 This work was mostly done in these MRs: * https://gitlab.wikimedia.org/repos/releng/zuul... [22:00:30] (03approved) 10dancy: docs: Refactor README to focus on common usage patterns [repos/releng/blubber] - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/140 (owner: 10dduvall) [22:14:39] (03update) 10dancy: AbstractSync: Build "next" image when building regular images [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/925 (https://phabricator.wikimedia.org/T398875) [22:14:45] (03update) 10dancy: AbstractSync: Build "next" image when building regular images [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/925 (https://phabricator.wikimedia.org/T398875) [22:26:15] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06collaboration-services: Provision kubeconfig file with credentials for the nodepool user - https://phabricator.wikimedia.org/T400850 (10bd808) 03NEW [22:26:48] (03merge) 10dancy: AbstractSync: Build "next" image when building regular images [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/925 (https://phabricator.wikimedia.org/T398875) [22:28:54] (03open) 10dancy: Release 4.194.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/926 [22:30:45] (03merge) 10dancy: Release 4.194.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/926 [22:35:58] the nodepool wikitech page redirects to https://wikitech.wikimedia.org/wiki/Obsolete:Nodepool [22:36:35] on the obsolete page it described how nodepool used to be a service that contint-admins could control: [22:36:38] sudo service nodepool status .. etc [22:36:48] but nowadays there is no such service on contint machines. [22:37:09] did it get replaced by something else? [22:38:53] mutante: not for zuulv2, no. It was an optional component for zuulv2. [22:39:07] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06collaboration-services: Provision kubeconfig file with credentials for the nodepool user - https://phabricator.wikimedia.org/T400850#11048593 (10bd808) @Dzahn I would be glad to talk through any questions you have about this component of the necessary c... [22:39:51] thcipriani: do we need nodepool for zuul-new ? [22:40:23] yes, modern zuul has to have nodepool [22:40:45] but it can be configured to work differently than the prior experiment [22:40:52] hmm..ok. then.. it seems like something we used to have but now there is no sign of it in puppet repo [22:41:01] and we need it back [22:41:16] will dig through git lot [22:41:21] yeah, we killed it with fire :) The new one is a Docker container that Dan's code builds [22:41:40] sounds like I need to un-kill it [22:41:54] since Dan said we need it on the new VMs [22:42:20] sure, but it will all be different from the prior deb package full of python [22:42:33] now it's a docker container full of python! ;) [22:43:15] this is what removed it in case it's useful: https://gerrit.wikimedia.org/r/c/operations/puppet/+/473838 but guessing the new puppet will be very different [22:44:40] ok, thank you and sigh [22:45:23] <3 mutante