[00:04:34] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 07Essential-Work, 05Release, 05Train Deployments: 1.44.0-wmf.27 deployment blockers - https://phabricator.wikimedia.org/T386222#10784804 (10thcipriani) 05Openβ†’03Resolved [00:20:42] (03update) 10thcipriani: spiderpig: Send HTTP access log to syslog if use_syslog enabled [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/780 (owner: 10dancy) [00:23:47] (03approved) 10thcipriani: spiderpig: Send HTTP access log to syslog if use_syslog enabled [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/780 (owner: 10dancy) [00:23:53] (03merge) 10thcipriani: spiderpig: Send HTTP access log to syslog if use_syslog enabled [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/780 (owner: 10dancy) [00:43:14] 10Release-Engineering-Team (Yak Shaving πŸƒπŸͺ’), 10Scap (SpiderPig πŸ•ΈοΈ), 07OKR-Work: Add browser notification when deployment is awaiting user interaction - https://phabricator.wikimedia.org/T392487#10784813 (10thcipriani) [00:43:44] 10GitLab (Project Migration), 06Release-Engineering-Team, 07Essential-Work: Create new GitLab project group: wmgmc - https://phabricator.wikimedia.org/T387060#10784814 (10thcipriani) [00:44:05] 10Release-Engineering-Team (Yak Shaving πŸƒπŸͺ’), 10Scap (SpiderPig πŸ•ΈοΈ), 07OKR-Work: Support pasting Gerrit URLs into SpiderPig - https://phabricator.wikimedia.org/T392591#10784818 (10thcipriani) [01:28:47] 06Release-Engineering-Team, 10Scap, 10Dumps-Generation: scap needs to be k8s-cluster aware - https://phabricator.wikimedia.org/T388761#10784839 (10Scott_French) Following up, I did get a chance to sketch this out, and indeed (1) it's not all that complicated in practice but (2) it does run head-first into th... [05:22:42] 10MediaWiki-Releasing, 10MediaWiki-extensions-ContactPage: Bundle Extension:ContactPage with MediaWiki core - https://phabricator.wikimedia.org/T392444#10784919 (10Pppery) 05Openβ†’03Declined Clear consensus against doing this. Closing accordingly. [05:57:29] 10Gerrit, 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review, 07Wikimedia-Incident: Investigate out of date refs following gerrit switchover - https://phabricator.wikimedia.org/T393034#10784948 (10Marostegui) >>! In T393034#10783498, @thcipriani wrote: >>>! In T393034#10783259, @Mat... [06:46:19] 10Gerrit, 10Release-Engineering-Team (Doing 😎), 10Bitu, 06collaboration-services, and 2 others: Confusion over which interface sets ssh keys for use with Gerrit (hint: not IDM) - https://phabricator.wikimedia.org/T393092#10784988 (10bd808) [06:46:36] 10Gerrit, 10Release-Engineering-Team (Doing 😎), 10Bitu, 06collaboration-services, and 2 others: Confusion over which interface sets ssh keys for use with Gerrit (hint: not IDM) - https://phabricator.wikimedia.org/T393092#10784990 (10bd808) 05Resolvedβ†’03Invalid [06:50:25] (03PS2) 10Jforrester: Zuul: [mediawiki/extensions/WikimediaMessages] Add cldr as phan dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1140208 (https://phabricator.wikimedia.org/T391230) (owner: 10Sbisson) [06:50:27] (03CR) 10Jforrester: [C:03+2] Zuul: [mediawiki/extensions/WikimediaMessages] Add cldr as phan dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1140208 (https://phabricator.wikimedia.org/T391230) (owner: 10Sbisson) [06:51:52] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/WikimediaMessages] Add cldr as phan dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1140208 (https://phabricator.wikimedia.org/T391230) (owner: 10Sbisson) [06:52:04] !log Zuul: [mediawiki/extensions/WikimediaMessages] Add cldr as phan dependency, for T391230 [06:52:06] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [06:52:06] T391230: Unified Dashboard: Support country-level filtering under custom suggestions view - https://phabricator.wikimedia.org/T391230 [06:55:05] (03update) 10bd808: SpiderPig: auto select first backport search match [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/731 (https://phabricator.wikimedia.org/T392508) [06:56:23] (03PS1) 10Jforrester: Zuul: [mediawiki/extensions/WikimediaMessages] Add cldr as full CI dependency too [integration/config] - 10https://gerrit.wikimedia.org/r/1140568 (https://phabricator.wikimedia.org/T391230) [06:57:08] (03merge) 10bd808: SpiderPig: auto select first backport search match [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/731 (https://phabricator.wikimedia.org/T392508) [06:57:35] (03CR) 10CI reject: [V:04-1] Zuul: [mediawiki/extensions/WikimediaMessages] Add cldr as full CI dependency too [integration/config] - 10https://gerrit.wikimedia.org/r/1140568 (https://phabricator.wikimedia.org/T391230) (owner: 10Jforrester) [06:58:20] (03PS2) 10Jforrester: Zuul: [mediawiki/extensions/WikimediaMessages] Add cldr as full CI dep too [integration/config] - 10https://gerrit.wikimedia.org/r/1140568 (https://phabricator.wikimedia.org/T391230) [06:58:35] (03CR) 10Jforrester: [C:03+2] Zuul: [mediawiki/extensions/WikimediaMessages] Add cldr as full CI dep too [integration/config] - 10https://gerrit.wikimedia.org/r/1140568 (https://phabricator.wikimedia.org/T391230) (owner: 10Jforrester) [06:59:50] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/WikimediaMessages] Add cldr as full CI dep too [integration/config] - 10https://gerrit.wikimedia.org/r/1140568 (https://phabricator.wikimedia.org/T391230) (owner: 10Jforrester) [07:00:08] !log Zuul: [mediawiki/extensions/WikimediaMessages] Add cldr as full CI dep too, for T391230 [07:00:10] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [07:00:11] T391230: Unified Dashboard: Support country-level filtering under custom suggestions view - https://phabricator.wikimedia.org/T391230 [07:16:15] (03open) 10addshore: A few small things on Hackathon flight [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/610 [07:23:45] (03PS3) 10Hashar: zuul: add aeywoo[at]proton.me to whitelist [integration/config] - 10https://gerrit.wikimedia.org/r/1140501 (owner: 10RhinosF1) [07:23:56] (03CR) 10Hashar: [C:03+2] zuul: add aeywoo[at]proton.me to whitelist [integration/config] - 10https://gerrit.wikimedia.org/r/1140501 (owner: 10RhinosF1) [07:25:31] (03Merged) 10jenkins-bot: zuul: add aeywoo[at]proton.me to whitelist [integration/config] - 10https://gerrit.wikimedia.org/r/1140501 (owner: 10RhinosF1) [07:28:18] Thanks hasharZzz [07:28:34] RhinosF1: you are welcome! :) [07:28:41] and thank you for all those patches [07:33:53] (03open) 10addshore: docker images: Bump postgres to 13.20 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/611 [07:34:56] (03open) 10addshore: Bump docker node20-test-browser:20.19.1 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/612 [07:35:41] (03open) 10addshore: Bump docker shellbox images [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/613 [07:40:02] (03open) 10addshore: go bump spf13 packages [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/614 [07:41:08] (03PS2) 10Hashar: helm-linter: Remove duplicate update-alternatives for helm3 [integration/config] - 10https://gerrit.wikimedia.org/r/1140168 (https://phabricator.wikimedia.org/T387548) (owner: 10Jelto) [07:41:41] (03CR) 10Hashar: [C:03+2] "I have amended the commit message to add the output of `update-alternatives --query helm` ;)" [integration/config] - 10https://gerrit.wikimedia.org/r/1140168 (https://phabricator.wikimedia.org/T387548) (owner: 10Jelto) [07:43:09] (03Merged) 10jenkins-bot: helm-linter: Remove duplicate update-alternatives for helm3 [integration/config] - 10https://gerrit.wikimedia.org/r/1140168 (https://phabricator.wikimedia.org/T387548) (owner: 10Jelto) [07:47:43] 10GitLab (Infrastructure), 10Release-Engineering-Team (Radar), 10ChangeProp, 06cloud-services-team, and 10 others: Figure out a plan to move forward with regarding Redis License changes - https://phabricator.wikimedia.org/T360596#10785130 (10hashar) Redis is now available under the AGPLv3. That was announc... [07:49:40] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 07Essential-Work, 05Release, 05Train Deployments: 1.45.0-wmf.1 deployment blockers - https://phabricator.wikimedia.org/T386223#10785145 (10Jdlrobson-WMF) [07:50:49] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 07Essential-Work, 05Release, 05Train Deployments: 1.45.0-wmf.1 deployment blockers - https://phabricator.wikimedia.org/T386223#10785149 (10Jdlrobson-WMF) Seems to relate to https://gerrit.wikimedia.org/g/mediawiki/extensions/ContentTranslation/+/275d8ce850c7... [08:05:42] 10GitLab (Infrastructure), 10Release-Engineering-Team (Radar), 10ChangeProp, 06cloud-services-team, and 10 others: Figure out a plan to move forward with regarding Redis License changes - https://phabricator.wikimedia.org/T360596#10785204 (10akosiaris) >>! In T360596#10785130, @hashar wrote: > Redis is now... [08:10:00] (03PS1) 10Jforrester: Zuul: [mediawiki/extensions/WikiLambda] Make Catalyst voting again [integration/config] - 10https://gerrit.wikimedia.org/r/1140636 [08:11:06] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 07Essential-Work, 05Release, 05Train Deployments: 1.45.0-wmf.1 deployment blockers - https://phabricator.wikimedia.org/T386223#10785213 (10Jdforrester-WMF) Note that despite the label of this task, this will be 1.44.0-wmf.28. [08:28:21] (03open) 10addshore: Allow global flags again in docker command [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/615 [08:28:53] (03close) 10addshore: Go dep bumps 2025-03-01 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/606 [08:37:17] (03PS1) 10Hashar: jjb: update helm-lint job to latest helm image [integration/config] - 10https://gerrit.wikimedia.org/r/1140641 (https://phabricator.wikimedia.org/T387548) [08:37:50] James_F: hi, aren't you at the hackathon? [08:38:16] I feel like switching quibble jobs from 1.13.0 to 1.14.0 whichi s https://gerrit.wikimedia.org/r/c/integration/config/+/1140215 [08:38:26] (03approved) 10addshore: Allow global flags again in docker command [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/615 [08:38:30] (03merge) 10addshore: Allow global flags again in docker command [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/615 [08:39:05] that would invalidate the success cache, which I don't think is an issue :) [08:41:55] hashar: I am. [08:42:00] (03CR) 10Hashar: [C:03+2] jjb: update helm-lint job to latest helm image [integration/config] - 10https://gerrit.wikimedia.org/r/1140641 (https://phabricator.wikimedia.org/T387548) (owner: 10Hashar) [08:42:03] hashar: Go for it. [08:42:11] (03CR) 10Hashar: [C:03+2] "I have updated the jobs with https://gerrit.wikimedia.org/r/c/integration/config/+/1140641 :)" [integration/config] - 10https://gerrit.wikimedia.org/r/1140168 (https://phabricator.wikimedia.org/T387548) (owner: 10Jelto) [08:42:19] James_F: <3 [08:43:18] (03Merged) 10jenkins-bot: jjb: update helm-lint job to latest helm image [integration/config] - 10https://gerrit.wikimedia.org/r/1140641 (https://phabricator.wikimedia.org/T387548) (owner: 10Hashar) [08:43:32] !log Updating Quibble jobs to 1.14.0 | https://gerrit.wikimedia.org/r/c/integration/config/+/1140215 | T378797 T384927 T386691 [08:43:36] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [08:43:37] T378797: [SPIKE] Use PHPUnit test results cache timing data to distribute tests in parallel runs - https://phabricator.wikimedia.org/T378797 [08:43:37] T384927: Download combined phpunit.results.cache timing data and use it to create balanced split_groups - https://phabricator.wikimedia.org/T384927 [08:43:37] T386691: How to e2e/integration test simple Item search - https://phabricator.wikimedia.org/T386691 [08:43:44] 10Gerrit, 10Release-Engineering-Team (Doing 😎), 10Bitu, 06collaboration-services, and 2 others: Confusion over which interface sets ssh keys for use with Gerrit (hint: not IDM) - https://phabricator.wikimedia.org/T393092#10785277 (10matmarex) I added a note to the help page to clarify that the Gerrit a... [08:44:16] I `./jjb-update` for that [09:21:46] (03CR) 10Hashar: [C:03+2] "I have let @jforrester@wikimedia.org know about the update, so the Hackathon users could know." [integration/config] - 10https://gerrit.wikimedia.org/r/1140215 (https://phabricator.wikimedia.org/T378797) (owner: 10Hashar) [09:23:03] (03Merged) 10jenkins-bot: jjb: switch jobs to Quibble 1.14.0 [integration/config] - 10https://gerrit.wikimedia.org/r/1140215 (https://phabricator.wikimedia.org/T378797) (owner: 10Hashar) [09:24:37] (03CR) 10Hashar: [C:03+1] Zuul: [mediawiki/extensions/WikiLambda] Make Catalyst voting again [integration/config] - 10https://gerrit.wikimedia.org/r/1140636 (owner: 10Jforrester) [09:55:59] (03update) 10addshore: A few small things on Hackathon flight [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/610 [09:56:13] (03merge) 10addshore: A few small things on Hackathon flight [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/610 [09:56:33] (03merge) 10addshore: docker images: Bump postgres to 13.20 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/611 [09:56:45] (03merge) 10addshore: Bump docker node20-test-browser:20.19.1 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/612 [09:56:55] (03merge) 10addshore: Bump docker shellbox images [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/613 [11:48:13] (03open) 10addshore: Improved global command validation when using `--help` [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/616 [11:54:47] (03open) 10addshore: docker: Suggest security_opt in custom files [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/617 [12:07:07] (03PS1) 10MusikAnimal: Zuul: add BetaFeatures as CodeEditor phan dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1140685 (https://phabricator.wikimedia.org/T373711) [12:08:37] (03PS2) 10MusikAnimal: Zuul: add BetaFeatures as CodeEditor phan dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1140685 (https://phabricator.wikimedia.org/T373711) [12:09:38] (03PS3) 10Jforrester: Zuul: [mediawiki/extensions/CodeEditor] Add BetaFeatures phan dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1140685 (https://phabricator.wikimedia.org/T373711) (owner: 10MusikAnimal) [12:10:39] (03CR) 10Jforrester: [C:03+2] Zuul: [mediawiki/extensions/WikiLambda] Make Catalyst voting again [integration/config] - 10https://gerrit.wikimedia.org/r/1140636 (owner: 10Jforrester) [12:12:18] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/WikiLambda] Make Catalyst voting again [integration/config] - 10https://gerrit.wikimedia.org/r/1140636 (owner: 10Jforrester) [12:16:13] apparently gerritlab requires write access to the repo? [12:16:25] (03open) 10taavi: make-release: Ignore zip files as well [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/169 [12:18:45] !log Zuul: [mediawiki/extensions/WikiLambda] Make Catalyst voting again [12:18:46] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [12:19:41] (03approved) 10jforrester: make-release: Ignore zip files as well [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/169 (owner: 10taavi) [12:27:57] (03CR) 10Jforrester: [C:03+2] Zuul: [mediawiki/extensions/CodeEditor] Add BetaFeatures phan dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1140685 (https://phabricator.wikimedia.org/T373711) (owner: 10MusikAnimal) [12:29:14] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/CodeEditor] Add BetaFeatures phan dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1140685 (https://phabricator.wikimedia.org/T373711) (owner: 10MusikAnimal) [12:30:13] !log Zuul: [mediawiki/extensions/CodeEditor] Add BetaFeatures phan dependency, for T373711 [12:30:15] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [12:30:16] T373711: Add support for Scribunto, JavaScript, CSS, JSON and Vue to CodeMirror 6 - https://phabricator.wikimedia.org/T373711 [12:35:45] (03merge) 10addshore: docker: Suggest security_opt in custom files [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/617 [12:37:44] (03PS1) 10Jforrester: Docker: Provide CI base image for Debian Trixie [integration/config] - 10https://gerrit.wikimedia.org/r/1140696 [12:38:09] (03update) 10addshore: docker: Improve handling and docs around custom name [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/618 [12:38:10] (03open) 10addshore: docker: Improve handling and docs around custom name [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/618 [12:58:27] (03merge) 10addshore: Improved global command validation when using `--help` [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/616 [13:05:19] (03merge) 10addshore: docker: Improve handling and docs around custom name [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/618 [13:26:13] 10Release-Engineering-Team (Yak Shaving πŸƒπŸͺ’), 10Scap (SpiderPig πŸ•ΈοΈ): Automatically select first backport search match - https://phabricator.wikimedia.org/T392508#10786070 (10bd808) > @brennen naturally wanted to use spaces or commas to put in a list of backport numbers and have them all selected on match. This... [14:18:03] (03update) 10daimona: Check maxWarnings when disabling eslint rules [repos/ci-tools/libup] - 10https://gitlab.wikimedia.org/repos/ci-tools/libup/-/merge_requests/72 (https://phabricator.wikimedia.org/T263922) [14:23:16] 10Continuous-Integration-Infrastructure, 06collaboration-services, 13Patch-For-Review: CI is overwhelmed and lots of jobs are failing randomly (2025-04-29) - https://phabricator.wikimedia.org/T392963#10786226 (10hashar) > Is there a task to track this (either making jenkins aware of containers or switchi... [14:24:49] (03update) 10taavi: runner: Handle repos with eslint maxWarnings: 0 [repos/ci-tools/libup] - 10https://gitlab.wikimedia.org/repos/ci-tools/libup/-/merge_requests/36 [14:38:33] (03open) 10dancy: Release 4.159.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/786 [14:41:00] (03merge) 10dancy: Release 4.159.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/786 [14:49:33] 10Release-Engineering-Team (Yak Shaving πŸƒπŸͺ’), 10Scap (SpiderPig πŸ•ΈοΈ): Dismiss interaction notification if someone responded to the interaction - https://phabricator.wikimedia.org/T393026#10786303 (10dancy) 05Openβ†’03Resolved Deployed via scap 4.159.0 [14:55:26] (03update) 10jforrester: ESLint: Replace wikimedia/client-es5 with wikimedia/client [repos/ci-tools/libup] - 10https://gitlab.wikimedia.org/repos/ci-tools/libup/-/merge_requests/70 (owner: 10esanders) [14:59:28] (03update) 10jforrester: ESLint: Replace wikimedia/client-es5 with wikimedia/client [repos/ci-tools/libup] - 10https://gitlab.wikimedia.org/repos/ci-tools/libup/-/merge_requests/70 (owner: 10esanders) [15:01:31] 10Release-Engineering-Team (Yak Shaving πŸƒπŸͺ’), 10Scap (SpiderPig πŸ•ΈοΈ): Automatically select first backport search match - https://phabricator.wikimedia.org/T392508#10786361 (10dancy) Deployed via scap 4.159.0 [15:03:22] (03update) 10jforrester: ESLint: Replace wikimedia/client-es5 with wikimedia/client [repos/ci-tools/libup] - 10https://gitlab.wikimedia.org/repos/ci-tools/libup/-/merge_requests/70 (owner: 10esanders) [15:04:21] (03update) 10taavi: runner: Handle repos with eslint maxWarnings: 0 [repos/ci-tools/libup] - 10https://gitlab.wikimedia.org/repos/ci-tools/libup/-/merge_requests/36 [15:08:06] (03update) 10taavi: runner: Handle repos with eslint maxWarnings: 0 [repos/ci-tools/libup] - 10https://gitlab.wikimedia.org/repos/ci-tools/libup/-/merge_requests/36 [15:11:06] (03approved) 10jforrester: ESLint: Replace wikimedia/client-es5 with wikimedia/client [repos/ci-tools/libup] - 10https://gitlab.wikimedia.org/repos/ci-tools/libup/-/merge_requests/70 (owner: 10esanders) [15:11:29] (03close) 10jforrester: Check maxWarnings when disabling eslint rules [repos/ci-tools/libup] - 10https://gitlab.wikimedia.org/repos/ci-tools/libup/-/merge_requests/72 (https://phabricator.wikimedia.org/T263922) (owner: 10daimona) [15:13:59] (03update) 10taavi: runner: Handle repos with eslint maxWarnings: 0 [repos/ci-tools/libup] - 10https://gitlab.wikimedia.org/repos/ci-tools/libup/-/merge_requests/36 (https://phabricator.wikimedia.org/T263922) [15:15:04] (03update) 10taavi: ESLint: Replace wikimedia/client-es5 with wikimedia/client [repos/ci-tools/libup] - 10https://gitlab.wikimedia.org/repos/ci-tools/libup/-/merge_requests/70 (owner: 10esanders) [15:15:11] (03merge) 10taavi: ESLint: Replace wikimedia/client-es5 with wikimedia/client [repos/ci-tools/libup] - 10https://gitlab.wikimedia.org/repos/ci-tools/libup/-/merge_requests/70 (owner: 10esanders) [15:16:45] (03update) 10taavi: runner: Handle repos with eslint maxWarnings: 0 [repos/ci-tools/libup] - 10https://gitlab.wikimedia.org/repos/ci-tools/libup/-/merge_requests/36 (https://phabricator.wikimedia.org/T263922) [15:17:25] 10Phabricator, 10Release-Engineering-Team (Priority Backlog πŸ“₯): Update to Phorge upstream 2025.18 release - https://phabricator.wikimedia.org/T386558#10786420 (10Aklapper) [15:18:32] (03merge) 10taavi: runner: Handle repos with eslint maxWarnings: 0 [repos/ci-tools/libup] - 10https://gitlab.wikimedia.org/repos/ci-tools/libup/-/merge_requests/36 (https://phabricator.wikimedia.org/T263922) [15:18:36] 10Phabricator, 10Release-Engineering-Team (Priority Backlog πŸ“₯): Update to Phorge upstream 2025.18 release - https://phabricator.wikimedia.org/T386558#10786421 (10Aklapper) [15:25:03] maintenance-disconnect-full-disks build 698140 integration-agent-docker-1060 (/: 26%, /srv: 99%, /var/lib/docker: 60%): OFFLINE due to disk space [15:26:37] 10GitLab, 06collaboration-services: 422 error logging into gitlab during the hackathon - https://phabricator.wikimedia.org/T393191 (10Reedy) 03NEW [15:30:03] maintenance-disconnect-full-disks build 698141 integration-agent-docker-1060 (/: 26%, /srv: 6%, /var/lib/docker: 56%): RECOVERY disk space OK [15:38:41] (03open) 10dancy: Backport.vue: Rename $autocomplete variable [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/787 (https://phabricator.wikimedia.org/T392508) [15:38:42] (03update) 10dancy: Backport.vue: Rename $autocomplete variable [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/787 (https://phabricator.wikimedia.org/T392508) [15:39:33] (03update) 10dancy: Backport.vue: Rename $autocomplete variable [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/787 (https://phabricator.wikimedia.org/T392508) [15:44:52] (03update) 10dancy: Backport.vue: Rename $autocomplete variable [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/787 (https://phabricator.wikimedia.org/T392508) [15:46:38] 10GitLab, 06collaboration-services: 422 error logging into gitlab during the hackathon - https://phabricator.wikimedia.org/T393191#10786538 (10Aklapper) Cannot reproduce. I'm on the `Wikimedia` wifi at the Hackathon and I just logged into our GitLab without a problem, and 2FA also worked without a problem. [15:52:17] 10GitLab, 06collaboration-services: 422 error logging into gitlab during the hackathon - https://phabricator.wikimedia.org/T393191#10786557 (10thcipriani) FWIW, I'm not seeing much of a spike in 4xx responses per grafana: https://grafana.wikimedia.org/d/R_1IvBZnz/gitlab-omnibus-overview?orgId=1&refresh=1m&view... [16:21:11] 10Continuous-Integration-Infrastructure: gate-and-submit backlogged due to waiting for castor-save-workspace-cache - https://phabricator.wikimedia.org/T353925#10786671 (10Daimona) This is still happening, and often a major bottleneck when there are many changes being submitted, with waits in the order of minutes. [16:31:51] 10Continuous-Integration-Infrastructure, 10Testing Support, 10ci-test-error (WMF-deployed Build Failure), 10MW-1.44-notes (1.44.0-wmf.23; 2025-04-01), 13Patch-For-Review: Selenium timeouts can cause the job to remain stuck until the build times out - https://phabricator.wikimedia.org/T389536#10786712 (10D... [16:32:55] 10Continuous-Integration-Infrastructure, 07Developer Productivity: Provide recheck option for only failed jobs - https://phabricator.wikimedia.org/T392941#10786716 (10Jdlrobson-WMF) Example of a patch which is blocking the train and taking a long time (and lots of resources) to get committed due to flakey test... [16:38:21] (03open) 10dancy: Backport.vue: Remove unnecessary {{ item.text }} [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/788 (https://phabricator.wikimedia.org/T392508) [16:38:23] (03update) 10dancy: Backport.vue: Remove unnecessary {{ item.text }} [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/788 (https://phabricator.wikimedia.org/T392508) [16:49:28] Limited time offer: anyone around who would be so kind to SSH into integration-agent-docker-1057 and see what https://integration.wikimedia.org/ci/job/wmf-quibble-selenium-php81/12434/console is doing? [16:50:20] i.e., checking the process list and seeing if we have the same chrome crash as in T389536#10675707 [16:50:21] T389536: Selenium timeouts can cause the job to remain stuck until the build times out - https://phabricator.wikimedia.org/T389536 [16:52:37] (03update) 10brennen: Draft: SpiderPig log view [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/767 (https://phabricator.wikimedia.org/T391005) [16:53:45] Uh, limited time offer has ended. Somehow it recovered on its own. [17:06:26] 10Continuous-Integration-Infrastructure, 10Testing Support, 10ci-test-error (WMF-deployed Build Failure), 10MW-1.44-notes (1.44.0-wmf.23; 2025-04-01), 13Patch-For-Review: Selenium timeouts can cause the job to remain stuck until the build times out - https://phabricator.wikimedia.org/T389536#10786832 (10D... [17:09:32] (03open) 10jforrester: Move eslint client-es5 fixes to its own, later method [repos/ci-tools/libup] - 10https://gitlab.wikimedia.org/repos/ci-tools/libup/-/merge_requests/73 [17:12:00] (03approved) 10taavi: Move eslint client-es5 fixes to its own, later method [repos/ci-tools/libup] - 10https://gitlab.wikimedia.org/repos/ci-tools/libup/-/merge_requests/73 (owner: 10jforrester) [17:12:03] (03merge) 10taavi: Move eslint client-es5 fixes to its own, later method [repos/ci-tools/libup] - 10https://gitlab.wikimedia.org/repos/ci-tools/libup/-/merge_requests/73 (owner: 10jforrester) [17:17:15] (03update) 10brennen: Draft: SpiderPig log view [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/767 (https://phabricator.wikimedia.org/T391005) [17:17:15] 10Continuous-Integration-Infrastructure: Fail more gracefully when a job is out of resources - https://phabricator.wikimedia.org/T393201 (10Daimona) 03NEW [17:32:06] (03PS1) 10Jforrester: Zuul: [mediawiki/extensions/WikimediaMessages] Add Echo as a phan dep [integration/config] - 10https://gerrit.wikimedia.org/r/1140769 [17:32:45] (03CR) 10Jforrester: [C:03+2] Zuul: [mediawiki/extensions/WikimediaMessages] Add Echo as a phan dep [integration/config] - 10https://gerrit.wikimedia.org/r/1140769 (owner: 10Jforrester) [17:39:08] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/WikimediaMessages] Add Echo as a phan dep [integration/config] - 10https://gerrit.wikimedia.org/r/1140769 (owner: 10Jforrester) [17:39:40] !log Zuul: [mediawiki/extensions/WikimediaMessages] Add Echo as a phan dep [17:39:41] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:41:44] 10GitLab, 06collaboration-services: 422 error logging into gitlab during the hackathon - https://phabricator.wikimedia.org/T393191#10786955 (10Dzahn) First thoughts by some might be related to throttling (fyi, for those with access, this has the details where we exempted the Hackathon network T382309). But th... [17:45:59] 10GitLab, 06collaboration-services: 422 error logging into gitlab during the hackathon - https://phabricator.wikimedia.org/T393191#10786962 (10Reedy) I had an issue earlier, so did another person… I’m not sure if either was after the wifi reset [17:50:24] 10GitLab, 06collaboration-services: 422 error logging into gitlab during the hackathon - https://phabricator.wikimedia.org/T393191#10786965 (10Dzahn) I can confirm though there are multiple hits in the gitlab access_log that match a 422 and an IP from the Hackathon network. [17:54:59] 10GitLab, 06collaboration-services: 422 error logging into gitlab during the hackathon - https://phabricator.wikimedia.org/T393191#10786967 (10Dzahn) Here are the timestamps. There is block between 09:31 and 09:43, another 10:04 to 10:24, 2 lone events at 15:34 and then it stopped. Would this correlate with lo... [18:02:16] 10GitLab, 06collaboration-services: 422 error logging into gitlab during the hackathon - https://phabricator.wikimedia.org/T393191#10786981 (10Dzahn) The url.path is almost always `/users/auth/openid_connect/callback?...` with the exception of the last 2 which were for `import/github.json`. The referer is mo... [18:12:48] 10GitLab, 06collaboration-services: 422 error logging into gitlab during the hackathon - https://phabricator.wikimedia.org/T393191#10787024 (10Dzahn) It has also been confirmed that our CAS (idp) has mostly been idling. https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&refresh=5m&var-server=idp1... [18:28:18] (03update) 10brennen: Draft: SpiderPig log view [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/767 (https://phabricator.wikimedia.org/T391005) [19:10:03] (03update) 10thcipriani: Draft: SpiderPig log view [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/767 (https://phabricator.wikimedia.org/T391005) (owner: 10brennen) [21:22:14] 10Continuous-Integration-Config: Make sure we are using npm cache in selenium jobs - https://phabricator.wikimedia.org/T393218 (10Daimona) 03NEW [21:26:06] 10Continuous-Integration-Config: Reduce impact of postbuild steps on job execution time - https://phabricator.wikimedia.org/T393220 (10Daimona) 03NEW [21:37:16] (03PS3) 10Awight: Switch to explicit numbering of footnote markers [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1109034 (https://phabricator.wikimedia.org/T382310) [21:37:17] 10Continuous-Integration-Infrastructure, 10CampaignEvents, 06Campaigns-Product-Team, 07ci-test-error: api-testing CI failure: Timeout of 5000ms exceeded in "before all" hook in "POST /campaignevents/v0/event_registration" - https://phabricator.wikimedia.org/T389863#10787482 (10Daimona) a:05Daimonaβ†’03Non... [22:41:55] (03update) 10thcipriani: Draft: SpiderPig log view [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/767 (https://phabricator.wikimedia.org/T391005) (owner: 10brennen)