[00:03:51] (03update) 10catrope: Add a script for checking which security patches can't be backported [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/282 [00:07:54] (03open) 10addshore: feat(toolforge): add authenticated local Wiki Replica proxy [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/794 (https://phabricator.wikimedia.org/T443528) [00:07:56] (03update) 10addshore: Draft: feat(toolforge): add authenticated local Wiki Replica proxy [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/794 (https://phabricator.wikimedia.org/T443528) [00:14:45] (03update) 10catrope: Add a script for checking which security patches can't be backported [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/282 [00:19:50] (03CR) 10Krinkle: [C:03+2] jjb: Update Doxygen from 1.10 to 1.18 [integration/config] - 10https://gerrit.wikimedia.org/r/1362225 (https://phabricator.wikimedia.org/T443533) (owner: 10Krinkle) [00:20:16] 10Continuous-Integration-Config, 10MediaWiki-Documentation, 06Tech-Docs-Team, 13Patch-For-Review: Update Doxygen from 1.10 to 1.18 for MediaWiki docs - https://phabricator.wikimedia.org/T443533#12513871 (10Krinkle) Deployed: https://doc.wikimedia.org/mediawiki-core/master/php/ [00:21:00] (03Merged) 10jenkins-bot: jjb: Update Doxygen from 1.10 to 1.18 [integration/config] - 10https://gerrit.wikimedia.org/r/1362225 (https://phabricator.wikimedia.org/T443533) (owner: 10Krinkle) [00:23:43] Krinkle: Oops, sorry, just seen. Yes, ./jjb-update works for me, I was just using it. [00:25:21] Krinkle: Note that the new Doxygen won’t apply to the old pushed versions , so https://doc.wikimedia.org/mediawiki-core/master/php/ and https://doc.wikimedia.org/mediawiki-core/REL1_46/php/ will be sparkly, but https://doc.wikimedia.org/mediawiki-core/1.46.2/php/ won’t. [00:26:14] Theoretically we could re-trigger the oxygen job for all known tags, but that’d potentially break older versions if we had old syntax that Doxygen now chokes/behaves differently on. [00:34:27] (03open) 10jforrester: Update composer from 2.10.2 to 2.11.0 in all images [repos/releng/dev-images] - 10https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/135 (https://phabricator.wikimedia.org/T440297 https://phabricator.wikimedia.org/T442976) [00:54:26] (03PS1) 10Pppery: Zuul: [mediawiki/extensions/SemanticPageSeries] Mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1362270 (https://phabricator.wikimedia.org/T437245) [01:11:06] (03PS1) 10Pppery: Zuul: [mediawiki/extensions/WikispeechSpeechDataCollector] Mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1362291 (https://phabricator.wikimedia.org/T432801) [01:11:45] (03PS1) 10Pppery: Utils/archive_repo.py: Add missing word [integration/config] - 10https://gerrit.wikimedia.org/r/1362293 [01:19:58] (03approved) 10brennen: Update composer from 2.10.2 to 2.11.0 in all images [repos/releng/dev-images] - 10https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/135 (https://phabricator.wikimedia.org/T440297 https://phabricator.wikimedia.org/T442976) (owner: 10jforrester) [01:59:12] 10Continuous-Integration-Config, 10MediaWiki-Documentation, 06Tech-Docs-Team, 06MediaWiki-Core-Platform-Team (Kanban): Update Doxygen from 1.10 to 1.18 for MediaWiki docs - https://phabricator.wikimedia.org/T443533#12514075 (10Krinkle) 05Open→03Resolved a:03Krinkle [08:33:28] 10Continuous-Integration-Config, 10MediaWiki-Vagrant: Remove CI support for mediawiki/vagrant - https://phabricator.wikimedia.org/T443567 (10hashar) 03NEW [08:34:31] 10Gerrit, 06Collaboration-Services, 07Incident Severity 1, 13Patch-For-Review, 07Sustainability (Incident Followup): Add the the ability to warm caches to Gerrit switchover cookbook - https://phabricator.wikimedia.org/T440177#12514449 (10ABran-WMF) 05In progress→03Open The nightly reindex of the repl... [08:35:59] (03PS1) 10Hashar: Remove mediawiki/vagrant [integration/config] - 10https://gerrit.wikimedia.org/r/1362911 (https://phabricator.wikimedia.org/T443567) [08:48:01] (03PS1) 10Hashar: Migrate mediawiki/ruby/api to ruby 3.3 [integration/config] - 10https://gerrit.wikimedia.org/r/1362921 (https://phabricator.wikimedia.org/T383337) [08:58:04] (03PS2) 10Hashar: Migrate mediawiki/ruby/api to ruby 3.3 [integration/config] - 10https://gerrit.wikimedia.org/r/1362921 (https://phabricator.wikimedia.org/T383337) [08:58:04] (03PS1) 10Hashar: Migrate wmf_styleguide-check to ruby 3.3 [integration/config] - 10https://gerrit.wikimedia.org/r/1362937 (https://phabricator.wikimedia.org/T383337) [09:14:29] 10Gerrit: Gergő Tisza has Gerrit multiple accounts sharing the same email - https://phabricator.wikimedia.org/T443570 (10hashar) 03NEW [09:20:30] (03CR) 10Hashar: [C:03+2] "I am dropping support for ruby 2.7 and unconfiguring mediawiki/vagrant from CI." [integration/config] - 10https://gerrit.wikimedia.org/r/1362911 (https://phabricator.wikimedia.org/T443567) (owner: 10Hashar) [09:21:42] (03Merged) 10jenkins-bot: Remove mediawiki/vagrant [integration/config] - 10https://gerrit.wikimedia.org/r/1362911 (https://phabricator.wikimedia.org/T443567) (owner: 10Hashar) [09:24:26] 10Gerrit: Gergő Tisza has Gerrit multiple accounts sharing the same email - https://phabricator.wikimedia.org/T443570#12514582 (10hashar) 05Open→03Resolved a:03hashar I went to disable the two accounts using: ` gerrit set-account 2280 --inactive gerrit set-account 7018 --inactive ` That hide them from... [09:26:52] (03CR) 10Hashar: [C:03+2] Migrate mediawiki/ruby/api to ruby 3.3 [integration/config] - 10https://gerrit.wikimedia.org/r/1362921 (https://phabricator.wikimedia.org/T383337) (owner: 10Hashar) [09:27:54] (03Merged) 10jenkins-bot: Migrate mediawiki/ruby/api to ruby 3.3 [integration/config] - 10https://gerrit.wikimedia.org/r/1362921 (https://phabricator.wikimedia.org/T383337) (owner: 10Hashar) [09:29:06] (03CR) 10Hashar: [C:03+2] Migrate wmf_styleguide-check to ruby 3.3 [integration/config] - 10https://gerrit.wikimedia.org/r/1362937 (https://phabricator.wikimedia.org/T383337) (owner: 10Hashar) [09:30:36] (03Merged) 10jenkins-bot: Migrate wmf_styleguide-check to ruby 3.3 [integration/config] - 10https://gerrit.wikimedia.org/r/1362937 (https://phabricator.wikimedia.org/T383337) (owner: 10Hashar) [10:30:42] (03PS1) 10QChris: Allow “Gerrit Managers” to import history [extensions/ExtendedInputBox] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1363034 [10:30:43] (03CR) 10QChris: [V:03+2 C:03+2] Allow “Gerrit Managers” to import history [extensions/ExtendedInputBox] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1363034 (owner: 10QChris) [10:33:09] (03PS1) 10QChris: Import done. Revoke import grants [extensions/ExtendedInputBox] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1363036 [10:33:09] (03CR) 10QChris: [V:03+2 C:03+2] Import done. Revoke import grants [extensions/ExtendedInputBox] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1363036 (owner: 10QChris) [10:47:24] 10Gerrit: Gergő Tisza has Gerrit multiple accounts sharing the same email - https://phabricator.wikimedia.org/T443570#12514863 (10Tgr) Thanks for the cleanup! The test account was, unsurprisingly, to test T219316 (long since fixed). I have no idea what the multimedia one was about, some kind of automation? I... [10:55:44] 10Continuous-Integration-Config, 10MediaWiki-Vagrant: Remove CI support for mediawiki/vagrant - https://phabricator.wikimedia.org/T443567#12514896 (10Tgr) CI support was de facto removed in {T398469}, the jobs didn't work since then. [11:22:07] (03PS1) 10Hashar: jjb: remove {name}-ruby2.7-rake, no more used [integration/config] - 10https://gerrit.wikimedia.org/r/1363092 (https://phabricator.wikimedia.org/T383337) [11:22:28] (03CR) 10Hashar: [C:03+2] jjb: remove {name}-ruby2.7-rake, no more used [integration/config] - 10https://gerrit.wikimedia.org/r/1363092 (https://phabricator.wikimedia.org/T383337) (owner: 10Hashar) [11:23:33] (03Merged) 10jenkins-bot: jjb: remove {name}-ruby2.7-rake, no more used [integration/config] - 10https://gerrit.wikimedia.org/r/1363092 (https://phabricator.wikimedia.org/T383337) (owner: 10Hashar) [11:29:49] (03PS1) 10Hashar: jjb: switch operations-puppet-doc to ruby 3.3 [integration/config] - 10https://gerrit.wikimedia.org/r/1363101 (https://phabricator.wikimedia.org/T383337) [12:00:05] FIRING: PuppetAgentFailure: Puppet agent failure detected on instance deployment-cp-text09 in project deployment-prep - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentFailure [12:05:05] FIRING: [2x] PuppetAgentFailure: Puppet agent failure detected on instance deployment-cp-text09 in project deployment-prep - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentFailure [13:33:41] 06Release-Engineering-Team, 06ServiceOps, 05FY2025-26 KR 5.1, 06MediaWiki-Core-Platform-Team (Kanban), and 2 others: rest-gateway: run system tests via helm test - https://phabricator.wikimedia.org/T424825#12515592 (10pmiazga) @JTweed-WMF I don't recall why this is on our Kanban board. Most likely because... [13:36:41] (03CR) 10Hashar: [C:03+2] jjb: switch operations-puppet-doc to ruby 3.3 [integration/config] - 10https://gerrit.wikimedia.org/r/1363101 (https://phabricator.wikimedia.org/T383337) (owner: 10Hashar) [13:37:49] (03Merged) 10jenkins-bot: jjb: switch operations-puppet-doc to ruby 3.3 [integration/config] - 10https://gerrit.wikimedia.org/r/1363101 (https://phabricator.wikimedia.org/T383337) (owner: 10Hashar) [13:37:58] (03PS1) 10Hashar: dockerfiles: remove rake-ruby2.7 [integration/config] - 10https://gerrit.wikimedia.org/r/1363210 (https://phabricator.wikimedia.org/T383337) [13:38:21] (03CR) 10Hashar: [C:03+2] dockerfiles: remove rake-ruby2.7 [integration/config] - 10https://gerrit.wikimedia.org/r/1363210 (https://phabricator.wikimedia.org/T383337) (owner: 10Hashar) [13:38:57] 06Release-Engineering-Team (Doing 😎), 10Catalyst (Luka Ijo Pimeja Jan), 07Essential-Work: CAPI/Patchdemo: Do not reset expiration to default on rebuild - https://phabricator.wikimedia.org/T443601 (10jnuche) 03NEW [13:39:59] (03Merged) 10jenkins-bot: dockerfiles: remove rake-ruby2.7 [integration/config] - 10https://gerrit.wikimedia.org/r/1363210 (https://phabricator.wikimedia.org/T383337) (owner: 10Hashar) [13:40:07] 06Release-Engineering-Team (Doing 😎), 10Catalyst (Luka Ijo Pimeja Jan), 07Essential-Work: CAPI/Patchdemo: Do not reset expiration to default on rebuild - https://phabricator.wikimedia.org/T443601#12515658 (10jnuche) [13:41:58] 06Release-Engineering-Team (Doing 😎), 10Catalyst (Luka Ijo Pimeja Jan), 07Essential-Work: CAPI/Patchdemo: Do not reset expiration to default on rebuild - https://phabricator.wikimedia.org/T443601#12515661 (10jnuche) [13:43:37] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 13Patch-For-Review: Migrate all CI jobs from Bullseye to Bookworm or later and drop Bullseye testing support - https://phabricator.wikimedia.org/T383337#12515683 (10hashar) [13:43:42] (03PS1) 10Hashar: jjb: notify releng when operations-puppet-doc fails [integration/config] - 10https://gerrit.wikimedia.org/r/1363212 [13:43:52] (03CR) 10Hashar: [C:03+2] jjb: notify releng when operations-puppet-doc fails [integration/config] - 10https://gerrit.wikimedia.org/r/1363212 (owner: 10Hashar) [13:45:01] (03Merged) 10jenkins-bot: jjb: notify releng when operations-puppet-doc fails [integration/config] - 10https://gerrit.wikimedia.org/r/1363212 (owner: 10Hashar) [13:48:43] 10GitLab, 06Release-Engineering-Team (Radar), 06Collaboration-Services, 13Patch-For-Review: gitlab behind CDN - https://phabricator.wikimedia.org/T425441#12515721 (10ABran-WMF) [13:49:44] (03PS1) 10Hashar: dockerfiles: reparent ecs from Bullseye to Trixie [integration/config] - 10https://gerrit.wikimedia.org/r/1363219 (https://phabricator.wikimedia.org/T383337) [13:51:26] (03PS1) 10Hashar: jjb: update ecs jobs from Bullseye to Trixie [integration/config] - 10https://gerrit.wikimedia.org/r/1363220 (https://phabricator.wikimedia.org/T383337) [13:51:36] (03CR) 10Hashar: [C:03+2] dockerfiles: reparent ecs from Bullseye to Trixie [integration/config] - 10https://gerrit.wikimedia.org/r/1363219 (https://phabricator.wikimedia.org/T383337) (owner: 10Hashar) [13:52:55] (03Merged) 10jenkins-bot: dockerfiles: reparent ecs from Bullseye to Trixie [integration/config] - 10https://gerrit.wikimedia.org/r/1363219 (https://phabricator.wikimedia.org/T383337) (owner: 10Hashar) [13:53:48] 10Continuous-Integration-Config, 10MediaWiki-Vagrant: Remove CI support for mediawiki/vagrant - https://phabricator.wikimedia.org/T443567#12515760 (10hashar) The test / gate jobs were broken indeed and this formally remove them. The documentation generation still ran though resulting in https://doc.wikimedia.... [13:57:04] !log Regenerating ecs documentation after having upgraded the image from Bullseye to Trixie: `zuul enqueue --trigger gerrit --pipeline postmerge --project operations/software/ecs --change 1280737,6` # T383337 [13:57:08] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [13:57:08] T383337: Migrate all CI jobs from Bullseye to Bookworm or later and drop Bullseye testing support - https://phabricator.wikimedia.org/T383337 [13:59:00] (03CR) 10Hashar: [C:03+2] "Verified after having deployed the jobs against I28bfc3c73ab910171120921506c617740e8b0f6a. The build passed and the doc is still at https:" [integration/config] - 10https://gerrit.wikimedia.org/r/1363220 (https://phabricator.wikimedia.org/T383337) (owner: 10Hashar) [13:59:34] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 13Patch-For-Review: Migrate all CI jobs from Bullseye to Bookworm or later and drop Bullseye testing support - https://phabricator.wikimedia.org/T383337#12515785 (10hashar) [14:00:34] (03Merged) 10jenkins-bot: jjb: update ecs jobs from Bullseye to Trixie [integration/config] - 10https://gerrit.wikimedia.org/r/1363220 (https://phabricator.wikimedia.org/T383337) (owner: 10Hashar) [14:07:59] (03PS1) 10Hashar: dockerfiles: reparent gradle from Bullseye to Trixie [integration/config] - 10https://gerrit.wikimedia.org/r/1363228 (https://phabricator.wikimedia.org/T383337) [14:15:53] (03PS1) 10Hashar: jjb: update gradle job from Bullseye to Trixie [integration/config] - 10https://gerrit.wikimedia.org/r/1363232 (https://phabricator.wikimedia.org/T383337) [14:16:56] (03CR) 10Hashar: [C:03+2] "For info :) Gradle version remains the same. The job is updated with I15ae0eb3c1e75aa899da7117e89712d705e3d99b and afaik it only affects `" [integration/config] - 10https://gerrit.wikimedia.org/r/1363228 (https://phabricator.wikimedia.org/T383337) (owner: 10Hashar) [14:17:31] (03CR) 10Hashar: [C:03+2] jjb: update gradle job from Bullseye to Trixie [integration/config] - 10https://gerrit.wikimedia.org/r/1363232 (https://phabricator.wikimedia.org/T383337) (owner: 10Hashar) [14:18:17] (03Merged) 10jenkins-bot: dockerfiles: reparent gradle from Bullseye to Trixie [integration/config] - 10https://gerrit.wikimedia.org/r/1363228 (https://phabricator.wikimedia.org/T383337) (owner: 10Hashar) [14:19:25] (03PS2) 10Hashar: jjb: update gradle job from Bullseye to Trixie [integration/config] - 10https://gerrit.wikimedia.org/r/1363232 (https://phabricator.wikimedia.org/T383337) [14:19:50] (03CR) 10Hashar: [C:03+2] jjb: update gradle job from Bullseye to Trixie [integration/config] - 10https://gerrit.wikimedia.org/r/1363232 (https://phabricator.wikimedia.org/T383337) (owner: 10Hashar) [14:21:04] (03Merged) 10jenkins-bot: jjb: update gradle job from Bullseye to Trixie [integration/config] - 10https://gerrit.wikimedia.org/r/1363232 (https://phabricator.wikimedia.org/T383337) (owner: 10Hashar) [14:21:43] (03PS1) 10Hashar: ci: update releng/gradle to latest [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/1363239 (https://phabricator.wikimedia.org/T383337) [14:42:16] (03open) 10dpogorzelski: Add repos/data-engineering/keda to the trusted repos [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/197 [14:50:04] 06Release-Engineering-Team, 06Collaboration-Services, 06NFDI-Germany, 10Wikidata, and 3 others: Github actions 403 errors cloning from Gerrit - https://phabricator.wikimedia.org/T441387#12516124 (10karapayneWMDE) [15:35:36] 10Gerrit, 10MediaWiki-extensions-Extended-Inputbox: Create Git Repo for Extended-Inputbox - https://phabricator.wikimedia.org/T443040#12516638 (10Celian) 05Stalled→03Resolved a:03Celian [15:43:38] (03merge) 10dancy: Add repos/data-engineering/keda to the trusted repos [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/197 (owner: 10dpogorzelski) [16:45:56] 10GerritBot: Gerritbot fails to add a comment if Phabricator failed - https://phabricator.wikimedia.org/T443652 (10Urbanecm_WMF) 03NEW [16:48:23] 10Gerrit, 06Release-Engineering-Team (Priority Backlog 📥), 06MediaWiki-Gerrit-Group-Requests, 07Essential-Work: wmf-deployment members can push directly to deployment branches - https://phabricator.wikimedia.org/T439548#12517443 (10thcipriani) 05Open→03Resolved This merged, thanks to @dancy's revie... [16:51:27] 06Release-Engineering-Team (Doing 😎), 06WMF-NDA-Requests: Requesting access to WMF-NDA project for Rmaung - https://phabricator.wikimedia.org/T443622#12517482 (10Aklapper) 05Open→03Resolved a:03Aklapper Added after verification [16:54:25] 10GitLab (Project and group requests), 06Release-Engineering-Team (Doing 😎), 10WMF security vulnerability pipeline, 05FY25-26 DE4.4, 06Product Safety and Integrity (Sprint Aster (Oct 12 - Oct 30)): GitLab Private Repository Request for: repos/product-saf... - https://phabricator.wikimedia.org/T443448#12517511 [16:54:55] 10GitLab (Project and group requests), 06Release-Engineering-Team (Doing 😎): GitLab Private Repository Request for: repos/security/security-patches - https://phabricator.wikimedia.org/T443510#12517537 (10dduvall) 05Open→03In progress p:05Triage→03Medium a:03dduvall [16:55:16] 10GitLab (Project and group requests), 06Release-Engineering-Team (Doing 😎), 10WMF security vulnerability pipeline, 05FY25-26 DE4.4, 06Product Safety and Integrity (Sprint Aster (Oct 12 - Oct 30)): GitLab Private Repository Request for: repos/product-saf... - https://phabricator.wikimedia.org/T443448#12517541 [17:09:45] (03open) 10project_16_bot_622d7b6d682aa66907ad1bffedf5b9e6: build(deps): bump github.com/itchyny/gojq from 0.12.19 to 0.12.20 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/795 [17:09:45] (03update) 10project_16_bot_622d7b6d682aa66907ad1bffedf5b9e6: build(deps): bump github.com/itchyny/gojq from 0.12.19 to 0.12.20 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/795 [17:10:23] (03open) 10project_16_bot_622d7b6d682aa66907ad1bffedf5b9e6: build(deps): bump golang.org/x/net from 0.60.0 to 0.62.0 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/796 [17:10:24] (03update) 10project_16_bot_622d7b6d682aa66907ad1bffedf5b9e6: build(deps): bump golang.org/x/net from 0.60.0 to 0.62.0 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/796 [17:10:25] (03update) 10project_16_bot_622d7b6d682aa66907ad1bffedf5b9e6: build(deps): bump golang.org/x/net from 0.60.0 to 0.62.0 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/796 [17:29:44] 10GitLab (Project and group requests), 06Release-Engineering-Team (Doing 😎), 10WMF security vulnerability pipeline, 05FY25-26 DE4.4, 06Product Safety and Integrity (Sprint Aster (Oct 12 - Oct 30)): GitLab Private Repository Request for: repos/product-saf... - https://phabricator.wikimedia.org/T443448#12517967 [17:31:55] 10Continuous-Integration-Config, 06Growth-Team, 10GrowthExperiments, 07ci-test-error (WMF-deployed Build Failure): phpbench job for GrowthExperiments fails due to missing dependency - https://phabricator.wikimedia.org/T329280#12517997 (10Urbanecm_WMF) 05Open→03Resolved a:03Jdforrester-WMF Effecti... [17:32:00] 10GitLab (Project and group requests), 06Release-Engineering-Team (Doing 😎): GitLab Private Repository Request for: repos/security/security-patches - https://phabricator.wikimedia.org/T443510#12518001 (10dduvall) 05In progress→03Resolved @Rsilvola done! Please review the [[ https://gitlab.wikimedia.org... [17:40:01] 06Release-Engineering-Team (Priority Backlog 📥), 10dev-images, 10Catalyst (Luka Ijo Pimeja Jan), 07Essential-Work: Upgrade apache2 images in Catalyst to Trixie - https://phabricator.wikimedia.org/T443286#12518105 (10dduvall) p:05Triage→03Medium [17:44:42] 06Release-Engineering-Team (Radar), 10Scap, 10ChangeProp, 10Citoid, and 22 others: Migrate node-based services in production to node26 - https://phabricator.wikimedia.org/T437512#12518207 (10dduvall) [17:45:52] (03PS1) 10Vaughn Walters: jjb: [catalyst-daily-CommunityConfiguration] Add CommunityConfiguration job [integration/config] - 10https://gerrit.wikimedia.org/r/1363351 (https://phabricator.wikimedia.org/T432907) [17:45:52] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team (Radar), 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, and 3 others: CI build slow and timing out sometimes - https://phabricator.wikimedia.org/T434024#12518213 (10dduvall) [17:46:06] (03CR) 10Vaughn Walters: [C:03+2] "Tests green at Tests green at https://integration.wikimedia.org/ci/view/selenium-daily/job/catalyst-daily-CommunityConfiguration/1/console" [integration/config] - 10https://gerrit.wikimedia.org/r/1363351 (https://phabricator.wikimedia.org/T432907) (owner: 10Vaughn Walters) [17:47:02] (03Merged) 10jenkins-bot: jjb: [catalyst-daily-CommunityConfiguration] Add CommunityConfiguration job [integration/config] - 10https://gerrit.wikimedia.org/r/1363351 (https://phabricator.wikimedia.org/T432907) (owner: 10Vaughn Walters) [18:01:02] (03PS1) 10Pppery: Zuul: [mediawiki/extensions/AutoCreatePage] Mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1363377 (https://phabricator.wikimedia.org/T440822) [18:05:35] 10Gerrit, 06Release-Engineering-Team, 06Security-Team, 07SecTeam-Processed, 07Security: Prevent Gerrit changes with top-level .patch files from being pushed - https://phabricator.wikimedia.org/T391269#12518403 (10hashar) 05Open→03Declined I am declining this one for now pending outcome of discuss... [18:07:21] 10Gerrit, 06Release-Engineering-Team: Timeout when trying to connect to gerrit over ssh - https://phabricator.wikimedia.org/T385019#12518421 (10hashar) 05Open→03Resolved a:03hashar Assuming this one got resolved and was an issue on @daniel side :) [18:09:44] 10Gerrit, 06Release-Engineering-Team, 07SecTeam-Processed, 07Security: Prevent Gerrit changes with top-level .patch files from being pushed - https://phabricator.wikimedia.org/T391269#12518435 (10sbassett) [18:12:52] 06Project-Admins, 06Release-Engineering-Team, 05MediaWiki-backport-deployments: Archive #MediaWiki-backport-deployments project - https://phabricator.wikimedia.org/T391779#12518441 (10Aklapper) [18:33:08] 10Continuous-Integration-Config, 06Release-Engineering-Team, 10BlueSpice: BlueSpice/HalloWelt force merging to branches and messing up mediawiki/extensions.git - https://phabricator.wikimedia.org/T431845#12518530 (10dduvall) I've removed force-push rights from `bluespice-admin` in the following projects: ht... [18:37:01] 10Continuous-Integration-Config, 06Release-Engineering-Team (Doing 😎), 10BlueSpice: BlueSpice/HalloWelt force merging to branches and messing up mediawiki/extensions.git - https://phabricator.wikimedia.org/T431845#12518554 (10dduvall) p:05Unbreak!→03High [18:45:44] sometimes i wonder why gerritlabs asks for confirmation when submitting patches for review but not when merging them [19:02:10] 10Beta-Cluster-Infrastructure, 10Cloud-VPS (Debian Bullseye Deprecation): Replace deployment-sessionstore06 (Bullseye deprecation) - https://phabricator.wikimedia.org/T436471#12518683 (10Southparkfan) 05Open→03In progress a:03Southparkfan [19:05:05] (03PS2) 10Mhurd: zuul: skip Castor in UploadWizard's integration-cache-proxy trial [integration/config] - 10https://gerrit.wikimedia.org/r/1357928 (https://phabricator.wikimedia.org/T439116) [19:07:30] (03CR) 10Ahmon Dancy: [C:03+1] "This looks safe to merge right now. @mhurd@wikimedia.org Agreed?" [integration/config] - 10https://gerrit.wikimedia.org/r/1357928 (https://phabricator.wikimedia.org/T439116) (owner: 10Mhurd) [19:08:10] (03CR) 10Ahmon Dancy: [C:03+1] "(once rebased as a standalone commit, that is)" [integration/config] - 10https://gerrit.wikimedia.org/r/1357928 (https://phabricator.wikimedia.org/T439116) (owner: 10Mhurd) [19:18:42] (03PS3) 10Mhurd: zuul: skip Castor in UploadWizard's integration-cache-proxy trial [integration/config] - 10https://gerrit.wikimedia.org/r/1357928 (https://phabricator.wikimedia.org/T439116) [19:22:07] (03CR) 10Ahmon Dancy: [C:03+2] zuul: skip Castor in UploadWizard's integration-cache-proxy trial [integration/config] - 10https://gerrit.wikimedia.org/r/1357928 (https://phabricator.wikimedia.org/T439116) (owner: 10Mhurd) [19:22:39] !log taking integration-agent-docker-107[0-5] offline for migration to trixie based agents (T440670) [19:22:42] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:22:42] T440670: Upgrade integration-agent-docker instances to trixie - https://phabricator.wikimedia.org/T440670 [19:23:25] (03Merged) 10jenkins-bot: zuul: skip Castor in UploadWizard's integration-cache-proxy trial [integration/config] - 10https://gerrit.wikimedia.org/r/1357928 (https://phabricator.wikimedia.org/T439116) (owner: 10Mhurd) [19:23:40] (03CR) 10Mhurd: "Yes, rebased as standalone, agreed." [integration/config] - 10https://gerrit.wikimedia.org/r/1357928 (https://phabricator.wikimedia.org/T439116) (owner: 10Mhurd) [19:23:52] 06Release-Engineering-Team (Doing 😎), 06WMF-NDA-Requests: WMF-NDA group access for gaiarajan - https://phabricator.wikimedia.org/T443679#12518779 (10Aklapper) 05Open→03Resolved a:03Aklapper Added after verification. [19:24:18] !log provisioning integration-agent-docker-109[6-9] trixie based agents (T440670) [19:24:21] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:24:37] !log Reloading Zuul to deploy https://gerrit.wikimedia.org/r/c/integration/config/+/1357928 [19:24:38] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:25:04] (03PS4) 10Mhurd: castor: skip the load and save when CASTOR_DISABLED=1 [integration/config] - 10https://gerrit.wikimedia.org/r/1357927 (https://phabricator.wikimedia.org/T439116) [19:27:27] !log ./jjb-update mediawiki-node26 quibble-vendor-mysql-php83 (testing T439116) [19:27:29] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:27:29] T439116: Test the cache proxy on integration VPS - https://phabricator.wikimedia.org/T439116 [19:31:25] (03open) 10dancy: logstash_poller: Log one line per error message in `scap poll-logstash` [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1404 [19:32:20] (03update) 10dancy: logstash_poller: Log one line per error message in `scap poll-logstash` [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1404 [19:32:23] (03update) 10dancy: logstash_poller: Log one line per error message in `scap poll-logstash` [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1404 [19:35:46] gerrit1003 - systemctl restart gerrit to deploy gerrit:1308081 to cap index query results to 5000 (T431398) [19:36:54] (03CR) 10Ahmon Dancy: [C:03+2] castor: skip the load and save when CASTOR_DISABLED=1 [integration/config] - 10https://gerrit.wikimedia.org/r/1357927 (https://phabricator.wikimedia.org/T439116) (owner: 10Mhurd) [19:37:02] (03merge) 10dancy: logstash_poller: Log one line per error message in `scap poll-logstash` [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1404 [19:37:59] (03Merged) 10jenkins-bot: castor: skip the load and save when CASTOR_DISABLED=1 [integration/config] - 10https://gerrit.wikimedia.org/r/1357927 (https://phabricator.wikimedia.org/T439116) (owner: 10Mhurd) [19:40:01] (03open) 10dancy: spiderpig: Rename _get_logs() to _read_logstash_errors() [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1405 [19:40:24] (03update) 10dancy: spiderpig: Rename _get_logs() to _read_logstash_errors() [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1405 [19:41:04] 10GitLab (Project and group requests), 06Release-Engineering-Team (Doing 😎): GitLab Private Repository Request for: repos/security/security-patches - https://phabricator.wikimedia.org/T443510#12518838 (10Rsilvola) Thank you! [19:43:31] (03merge) 10dancy: spiderpig: Rename _get_logs() to _read_logstash_errors() [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1405 [19:45:49] 10Phabricator, 06Release-Engineering-Team (Priority Backlog 📥): Remove use of PhutilURI::getQueryParams() and PhutilURI::setQueryParams() - https://phabricator.wikimedia.org/T443688 (10Aklapper) 03NEW p:05Triage→03Medium [19:46:48] (03open) 10dancy: spiderpig: web: Fix two bugs in Backport and Train [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1406 [19:46:56] (03update) 10dancy: spiderpig: web: Fix two bugs in Backport and Train [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1406 [19:50:04] (03merge) 10dancy: spiderpig: web: Fix two bugs in Backport and Train [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1406 [19:52:54] (03open) 10dancy: spiderpig: web: Correct the web types and link them to their Python sources [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1407 [19:53:22] (03update) 10dancy: spiderpig: web: Correct the web types and link them to their Python sources [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1407 [19:54:57] (03merge) 10dancy: spiderpig: web: Correct the web types and link them to their Python sources [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1407 [19:55:38] 10Phabricator, 06Release-Engineering-Team (Doing 😎), 07PHP 8.5 support: Null array key error in PhabricatorProjectReportsController::renderDateControls() - https://phabricator.wikimedia.org/T443689 (10Aklapper) 03NEW [19:58:16] (03open) 10dancy: spiderpig: web: Convert api.js and state.js to TypeScript [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1408 [19:59:11] 10Phabricator, 06Release-Engineering-Team (Doing 😎), 07PHP 8.5 support: Null array key error in PhabricatorProjectReportsController::renderDateControls() - https://phabricator.wikimedia.org/T443689#12518928 (10Aklapper) [20:01:09] !log added new integration-agent-docker-{1097,1098,1099,1100,1101} trixie based agents to jenkins (T440670) [20:01:11] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:01:12] T440670: Upgrade integration-agent-docker instances to trixie - https://phabricator.wikimedia.org/T440670 [20:02:50] (03open) 10aklapper: PhabricatorProjectReportsController: Avoid null array key for period [repos/phabricator/phabricator] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/merge_requests/132 (https://phabricator.wikimedia.org/T443689) [20:03:28] !log removing integration-agent-docker-{1071,1072,1073,1074,1075} agents from jenkins and deleting instances (T440670) [20:03:31] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:06:44] 06Project-Admins, 06Release-Engineering-Team (Doing 😎): Create project tag for Extended-Inputbox - https://phabricator.wikimedia.org/T442416#12518972 (10Celian) Hello, can we rename the tag in #mediawiki-extensions-extendedinputbox without a hypen ? (We change the name to line up with the Gerrit name) [20:16:55] (03open) 10aklapper: Remove use of deprecated PhutilURI::setQueryParams() [repos/phabricator/phabricator] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/merge_requests/133 (https://phabricator.wikimedia.org/T443688) [20:22:05] FIRING: PuppetAgentNoResources: No Puppet resources found on instance deployment-sessionstore07 on project deployment-prep - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentNoResources [20:22:11] 10Beta-Cluster-Infrastructure: No Puppet resources found on instance deployment-sessionstore07 on project deployment-prep - https://phabricator.wikimedia.org/T443697 (10wmcs-alerts) 03NEW [20:32:24] !log Preparing Cassandra for sessionstore switchover - T436471 [20:32:27] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:32:27] T436471: Replace deployment-sessionstore06 (Bullseye deprecation) - https://phabricator.wikimedia.org/T436471 [20:33:24] (03open) 10aklapper: Remove use of deprecated PhutilURI::getQueryParams() [repos/phabricator/phabricator] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/merge_requests/134 (https://phabricator.wikimedia.org/T443688) [20:47:05] RESOLVED: PuppetAgentNoResources: No Puppet resources found on instance deployment-sessionstore07 on project deployment-prep - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentNoResources [20:51:53] 10Phabricator, 10Phabricator Antivandalism Extension, 06Release-Engineering-Team (Doing 😎): PhabricatorAntiVandalismConfigOptions must implement getApplicationClassName() - https://phabricator.wikimedia.org/T443706 (10Aklapper) 03NEW p:05Triage→03Low [20:56:20] 10Phabricator (phabricator-next), 10Phabricator Antivandalism Extension, 06Release-Engineering-Team (Doing 😎): PhabricatorAntiVandalismConfigOptions must implement getApplicationClassName() - https://phabricator.wikimedia.org/T443706#12519343 (10Aklapper) Fixed by https://gerrit.wikimedia.org/r/plugins/gitil... [20:57:00] 10Phabricator (phabricator-next), 10Phabricator Antivandalism Extension, 06Release-Engineering-Team (Doing 😎): PhabricatorAntiVandalismConfigOptions must implement getApplicationClassName() - https://phabricator.wikimedia.org/T443706#12519346 (10Aklapper) [20:57:02] 10Phabricator (phabricator-next), 06Release-Engineering-Team, 06Collaboration-Services: Deploy Phab/Phorge 2026-10-20 - https://phabricator.wikimedia.org/T441842#12519347 (10Aklapper) [20:57:08] 10Phabricator (phabricator-next), 10Phabricator Antivandalism Extension, 06Release-Engineering-Team (Doing 😎): PhabricatorAntiVandalismConfigOptions must implement getApplicationClassName() - https://phabricator.wikimedia.org/T443706#12519348 (10Aklapper) 05Open→03Stalled [21:03:41] 06Release-Engineering-Team (Radar), 10Scap, 10Citoid, 10CXServer, and 21 others: Migrate node-based services in production to node26 - https://phabricator.wikimedia.org/T437512#12519364 (10Jdforrester-WMF) [21:23:42] 10Beta-Cluster-Infrastructure, 10Cloud-VPS (Debian Bullseye Deprecation): Replace deployment-sessionstore06 (Bullseye deprecation) - https://phabricator.wikimedia.org/T436471#12519535 (10Southparkfan) The plan: - [x] Spin up `deployment-sessionstore07` - [x] In `deployment-sessionstore` prefix hiera: set `prof... [21:45:24] !log Decommission deployment-sessionstore06 - T436471 [21:45:26] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:45:26] T436471: Replace deployment-sessionstore06 (Bullseye deprecation) - https://phabricator.wikimedia.org/T436471 [21:49:42] 10Beta-Cluster-Infrastructure, 10Cloud-VPS (Debian Bullseye Deprecation), 13Patch-For-Review: Replace deployment-sessionstore06 (Bullseye deprecation) - https://phabricator.wikimedia.org/T436471#12519637 (10Southparkfan) 05In progress→03Resolved [21:58:57] 10Beta-Cluster-Infrastructure, 06tools-infrastructure-team: Secure deployment-prep against php2shell and restore access - https://phabricator.wikimedia.org/T442482#12519708 (10Southparkfan) [21:59:13] !log Upgrade change-propragation container to use new image with node26 (T442195) - T442482 [21:59:17] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:59:17] T442195: Update change-prop to node22+ - https://phabricator.wikimedia.org/T442195 [21:59:17] T442482: Secure deployment-prep against php2shell and restore access - https://phabricator.wikimedia.org/T442482 [22:01:02] thank you! [22:04:16] apergos: Thank YOU for the awesome work :) [22:06:25] summoning James F who did most of it :-) [22:08:40] cpjobqueue is producing lots of error messages, but I wouldn't be surprised if it's a pre-upgrade issue [22:10:01] ` [22:10:18] `{"name":"change-propagation","hostname":"6fbb4d78578c","pid":1,"level":"ERROR","err":{"message":"Broker: Unknown topic or partition","name":"changeprop","stack":"Error: Broker: Unknown topic or partition\n at LibrdKafkaError. (......) ","rule_name":"low_traffic_jobs","executor":"RetryExecutor","levelPath":"error/consumer"},"msg":"Broker: Unknown topic or partition","time":"2026-11-04T21:55:56.709Z","v":0}` [22:12:48] unfortunately I don't know. I'm pretty new to the changeprop train so I don't know the baseline behaviour [22:20:24] Yeah, I don't know either. Interestingly, it's only producing errors for `low_traffic_jobs`. But I can see `eqiad.mediawiki.job` topics on kafka-main-5, so... [22:32:36] !log taking integration-agent-docker-{1076,1077,1078,1079,1080,1081} offline to be replaced by trixie agents (T440670) [22:32:39] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:32:39] T440670: Upgrade integration-agent-docker instances to trixie - https://phabricator.wikimedia.org/T440670 [22:40:56] !log creating new integration-agent-docker-{1102,1103,1104,1105,1106,1107} trixie based instances/agents (T440670) [22:40:59] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:40:59] T440670: Upgrade integration-agent-docker instances to trixie - https://phabricator.wikimedia.org/T440670 [22:51:06] FIRING: PuppetStaleCertificates: Found non-revoked Puppet certificates for 1 deleted instances on deployment-puppetserver-1 - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/PuppetStaleCertificates - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetStaleCertificates [22:57:07] 10Beta-Cluster-Infrastructure, 10Cloud-VPS (Debian Bullseye Deprecation): Replace deployment-etcd05 (Bullseye deprecation) - https://phabricator.wikimedia.org/T436473#12519969 (10Southparkfan) 05Open→03In progress a:03Southparkfan [22:57:14] !log registered new integration-agent-docker-{1102,1103,1104,1105,1106,1107} trixie based agents with jenkins (T440670) [22:57:17] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:57:17] T440670: Upgrade integration-agent-docker instances to trixie - https://phabricator.wikimedia.org/T440670 [22:59:23] !log deleting integration-agent-docker-{1076,1077,1078,1079,1081,1082} agents (T440670) [22:59:25] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:02:37] !log taking integration-agent-docker-{1083,1084,1085,1086,1087,1088} offline to be replaced by trixie agents (T440670) [23:02:39] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:02:40] T440670: Upgrade integration-agent-docker instances to trixie - https://phabricator.wikimedia.org/T440670 [23:03:35] 10Beta-Cluster-Infrastructure, 10Cloud-VPS (Debian Bullseye Deprecation): Replace deployment-echostore02 (Bullseye deprecation) - https://phabricator.wikimedia.org/T436479#12519986 (10Southparkfan) 05Open→03In progress a:03Southparkfan [23:04:09] !log creating new integration-agent-docker-{1108,1109,1110,1111,1112,1113} trixie based instances/agents (T440670) [23:04:12] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:27:07] !log registered new integration-agent-docker-{1108,1109,1110,1111,1112,1113} trixie based agents with jenkins (T440670) [23:27:10] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:27:10] T440670: Upgrade integration-agent-docker instances to trixie - https://phabricator.wikimedia.org/T440670 [23:27:49] !log deleting integration-agent-docker-{1083,1084,1085,1086,1087,1088} agents and instances (T440670) [23:27:52] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:31:39] !log taking integration-agent-docker-{1089,1090,1091,1092,1093} offline to be replaced by trixie agents (T440670) [23:31:42] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:33:37] !log creating new integration-agent-docker-{1114,1115,1116,1117,1118} trixie based instances/agents (T440670) [23:33:40] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:33:40] T440670: Upgrade integration-agent-docker instances to trixie - https://phabricator.wikimedia.org/T440670 [23:55:00] !log registered new integration-agent-docker-{1114,1115,1116,1117,1118} trixie based agents with jenkins (T440670) [23:55:03] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:55:03] T440670: Upgrade integration-agent-docker instances to trixie - https://phabricator.wikimedia.org/T440670 [23:55:21] !log deleting integration-agent-docker-{1089,1090,1091,1092,1093} agents and instances (T440670) [23:55:24] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL