[00:22:46] (03update) 10swfrench: kubernetes: support image "kind" and deprecate the debug boolean [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/702 (https://phabricator.wikimedia.org/T389499) [00:27:31] (03update) 10swfrench: kubernetes: support image "kind" and deprecate the debug boolean [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/702 (https://phabricator.wikimedia.org/T389499) [00:48:23] (03approved) 10dancy: kubernetes: support image "kind" and deprecate the debug boolean [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/702 (https://phabricator.wikimedia.org/T389499) (owner: 10swfrench) [01:23:09] (03update) 10swfrench: kubernetes: support image "kind" and deprecate the debug boolean [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/702 (https://phabricator.wikimedia.org/T389499) [01:24:20] (03update) 10swfrench: kubernetes: support image "kind" and deprecate the debug boolean [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/702 (https://phabricator.wikimedia.org/T389499) [03:00:47] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-CentralAuth, 10WMF-JobQueue, 05Account-Vanishing, and 4 others: "JobQueueError: Could not enqueue jobs" when trying to request a global vanish on Beta Cluster - https://phabricator.wikimedia.org/T389585#10685457 (10Djackson-ctr) QA is completed, (//Glob... [09:04:58] 10GitLab: Gitlab MFA reset for Esuvat Bomani - https://phabricator.wikimedia.org/T383123#10685656 (10Aklapper) 05Stalled→03Resolved >>! In T383123#10437545, @MBywater-WMF wrote: > Esuvat confirmed they were able to login using SSH codes, so this task can be closed. Feel free to close tasks in that case... [09:09:43] 10GitLab: Gitlab MFA reset for Esuvat Bomani - https://phabricator.wikimedia.org/T383123#10685683 (10Aklapper) >>! In T383123#10439981, @EBomani-WMF wrote: > Additionally, Is there a way to connect my EBomani and EBomani-WMF accounts > on Phabricator? @EBomani: Please go to https://phabricator.wikimedia.org... [10:19:04] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥), 10Wikimedia-Phabricator-Extensions: Remove custom "Expert Mode" - https://phabricator.wikimedia.org/T351289#10685889 (10Aklapper) > "Never open a gate you don't know why it was shut" feels like it applies. If I'm supposed to maintain a place and... [10:23:13] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥), 10Wikimedia-Phabricator-Extensions: Remove custom "Expert Mode" - https://phabricator.wikimedia.org/T351289#10685895 (10Peachey88) I wonder if we should write up the various features it gave and raise upstream for potential inclusion of them in... [11:31:30] 10GitLab (Pipeline Services Migration🐤), 06collaboration-services, 13Patch-For-Review: move os-reports.wikimedia.org to kubernetes - https://phabricator.wikimedia.org/T350794#10686062 (10Jelto) I added Puppetdb to the external-services configuration, so access can be managed using Kubernetes services and pro... [12:13:12] 10Scap, 13Patch-For-Review: SecurityPatchBot: Failing due to malformed patch - https://phabricator.wikimedia.org/T390144#10686260 (10jnuche) 05Open→03Resolved a:03jnuche [14:04:44] 10Deployments, 06serviceops, 10Shellbox, 10Wikibase-Quality-Constraints, and 5 others: Burst of GuzzleHttp Exception for http://localhost:6025/call/constraint-regex-checker - https://phabricator.wikimedia.org/T371633#10686491 (10AudreyPenven_WMDE) a:03AudreyPenven_WMDE [14:16:54] 10Deployments, 06serviceops, 10Shellbox, 10Wikibase-Quality-Constraints, and 5 others: Burst of GuzzleHttp Exception for http://localhost:6025/call/constraint-regex-checker - https://phabricator.wikimedia.org/T371633#10686526 (10AudreyPenven_WMDE) a:05AudreyPenven_WMDE→03None [14:21:51] (03merge) 10dancy: kubernetes: support image "kind" and deprecate the debug boolean [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/702 (https://phabricator.wikimedia.org/T389499) (owner: 10swfrench) [14:31:29] Yippee, build fixed! [14:31:29] Project beta-update-databases-eqiad build #83586: 09FIXED in 11 min: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/83586/ [14:36:40] 10Continuous-Integration-Infrastructure, 10Testing Support, 10ci-test-error (WMF-deployed Build Failure), 10MW-1.44-notes (1.44.0-wmf.23; 2025-04-01), and 2 others: Selenium timeouts can cause the job to remain stuck until the build times out - https://phabricator.wikimedia.org/T389536#10686605 (10mszabo)... [14:50:00] (03open) 10mszabo: releases: bump wdio-mediawiki to 2.7.1 [repos/ci-tools/libup-config] - 10https://gitlab.wikimedia.org/repos/ci-tools/libup-config/-/merge_requests/68 (https://phabricator.wikimedia.org/T389536) [14:58:38] 10Scap, 06serviceops: Migrate scap's maintenance script invocations to PHP 8.1 - https://phabricator.wikimedia.org/T390225#10686630 (10dancy) >>! In T390225#10684280, @Scott_French wrote: > @dduvall - When you get a chance, could you confirm that my understanding here is correct? Confirming on Dan's behalf.... [15:01:19] 10Continuous-Integration-Infrastructure, 10Testing Support, 10ci-test-error (WMF-deployed Build Failure), 10MW-1.44-notes (1.44.0-wmf.23; 2025-04-01), and 2 others: Selenium timeouts can cause the job to remain stuck until the build times out - https://phabricator.wikimedia.org/T389536#10686638 (10Daimona)... [15:05:26] (03update) 10mszabo: releases: bump wdio-mediawiki to 2.7.1 [repos/ci-tools/libup-config] - 10https://gitlab.wikimedia.org/repos/ci-tools/libup-config/-/merge_requests/68 (https://phabricator.wikimedia.org/T389536) [15:05:47] that's an IRC message for adding a reviewer [15:05:49] seems kinda excessive [15:39:33] (03PS1) 10Santiago Faci: Zuul: [mediawiki/extensions/MetricsPlatform]: Updated CI dependencies [integration/config] - 10https://gerrit.wikimedia.org/r/1131988 (https://phabricator.wikimedia.org/T390303) [16:17:56] (03approved) 10jforrester: releases: bump wdio-mediawiki to 2.7.1 [repos/ci-tools/libup-config] - 10https://gitlab.wikimedia.org/repos/ci-tools/libup-config/-/merge_requests/68 (https://phabricator.wikimedia.org/T389536) (owner: 10mszabo) [16:18:00] (03merge) 10jforrester: releases: bump wdio-mediawiki to 2.7.1 [repos/ci-tools/libup-config] - 10https://gitlab.wikimedia.org/repos/ci-tools/libup-config/-/merge_requests/68 (https://phabricator.wikimedia.org/T389536) (owner: 10mszabo) [16:30:35] Yippee, build fixed! [16:30:35] Project mediawiki-core-doxygen build #9269: 09FIXED in 12 min: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen/9269/ [16:35:33] (03open) 10dancy: spiderpig: Show job duration information [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/726 (https://phabricator.wikimedia.org/T383851) [16:35:35] (03update) 10dancy: spiderpig: Show job duration information [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/726 (https://phabricator.wikimedia.org/T383851) [16:37:23] (03update) 10dancy: spiderpig: Show job duration information [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/726 (https://phabricator.wikimedia.org/T383851) [16:39:49] (03merge) 10dancy: spiderpig: Show job duration information [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/726 (https://phabricator.wikimedia.org/T383851) [16:40:23] 10Scap (SpiderPig 🕸️): SpiderPig UI: address user interface feedback - https://phabricator.wikimedia.org/T383835#10686949 (10dancy) [16:40:53] 10Scap (SpiderPig 🕸️), 13Patch-For-Review: SpiderPig UI: move job duration time - https://phabricator.wikimedia.org/T383851#10686950 (10dancy) 05Open→03Resolved [16:48:09] mszabo: The events sent by gitlab are not very granular. I'm pretty sue we just get a "action: update" event for everything that touches the MR. https://docs.gitlab.com/user/project/integrations/webhook_events/#merge-request-events [16:48:30] *sure [16:48:39] ack, thanks :) [16:50:05] https://gitlab.wikimedia.org/toolforge-repos/wikibugs2/-/blob/main/src/wikibugs2/gitlab.py?ref_type=heads#L178 would be the place to filter things out if we could tell the difference. [16:50:12] yeah, i think there's a `changes` key in there but... [16:51:12] It's been a minute since I actually looked at the json blobs for this stuff :) [16:51:17] yeah, same. [16:57:37] argh: walked to the coffeeshop without my yubikey. [16:58:02] 10GitLab (Integrations): [gitlab-webhooks] setup log rotation for /data/project/gitlab-webhooks/logs/events.log - https://phabricator.wikimedia.org/T390311 (10bd808) 03NEW [17:01:50] does look like it has for example current and previous reviewers. [17:03:16] i guess in the interests of less botspam, we could drop "update" events altogether, or figure out which ones we think ar e significant. [17:04:02] I'm getting the events for that particular MR from the logs so I can stare at them later [17:04:47] (03open) 10thcipriani: DNM (obvs): a dubious progress bar example! [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/727 [17:04:48] (03update) 10thcipriani: DNM (obvs): a dubious progress bar example! [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/727 [17:05:10] 10GitLab (Integrations): [gitlab-webhooks] setup log rotation for /data/project/gitlab-webhooks/logs/events.log - https://phabricator.wikimedia.org/T390311#10687052 (10brennen) [17:06:44] (03close) 10thcipriani: DNM (obvs): a dubious progress bar example! [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/727 [17:11:16] It does look like there is enough state data that we could try to ignore things that only set/remove a reviewer. https://phabricator.wikimedia.org/P74497$125-146 [17:21:47] T390313 [17:21:48] T390313: Filter out changes that only set/removed the reviewer - https://phabricator.wikimedia.org/T390313 [17:27:25] 10GitLab (Integrations): [gitlab-webhooks] setup log rotation for /data/project/gitlab-webhooks/logs/events.log - https://phabricator.wikimedia.org/T390311#10687107 (10bd808) In #tool-gitlab-account-approval I do log rotation with a daily scheduled job. `lang=yaml,name=jobs.yaml - name: logrotate command: "log... [17:33:15] awesome, thank you for filing that! [18:40:47] 10Gerrit, 06Release-Engineering-Team, 06collaboration-services, 07User-notice: Enable browser notifications system in Gerrit - https://phabricator.wikimedia.org/T389327#10687277 (10Paladox) https://phabricator.wikimedia.org/F58932052 has nothing to do with this. It's to do with the wikimedia checkers p... [18:54:32] 06Release-Engineering-Team, 10MW-on-K8s, 06serviceops, 13Patch-For-Review: Refactor scap's kubernetes DeploymentsConfig to support selection of image kinds - https://phabricator.wikimedia.org/T389499#10687323 (10Scott_French) [18:59:47] 06Release-Engineering-Team, 10MW-on-K8s, 06serviceops, 13Patch-For-Review: Refactor scap's kubernetes DeploymentsConfig to support selection of image kinds - https://phabricator.wikimedia.org/T389499#10687338 (10Scott_French) Alright, once a scap release is deployed that picks up https://gitlab.wikimedia.o... [20:45:13] relengers, is anyone around to help sync some patches to wmf.22 and mediawki-config? [21:05:47] (03PS10) 10Krinkle: zuul: Remove uncontrolled recursion of extra-dependencies in MW jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1130957 (https://phabricator.wikimedia.org/T389998) [21:05:51] (03CR) 10Krinkle: [C:03+2] zuul: Remove uncontrolled recursion of extra-dependencies in MW jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1130957 (https://phabricator.wikimedia.org/T389998) (owner: 10Krinkle) [21:07:11] (03Merged) 10jenkins-bot: zuul: Remove uncontrolled recursion of extra-dependencies in MW jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1130957 (https://phabricator.wikimedia.org/T389998) (owner: 10Krinkle) [21:08:29] kostajh: did you find the help you needed? [21:08:59] bd808: I think sbassett said he could do it, but would be great if someone else could take this on [21:09:10] as sbassett and i are both on other things [21:10:48] somebody who knows what is changing and why needs to verify the deploy. Typing `scap backport` is the easy part. [21:12:28] sbassett will do it [21:15:13] !log Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1130957 [21:15:13] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:15:35] scott's asking about the deploy in -opertations and on slack. hopefully he'll get the answer he needs soon [21:16:01] i think this is ok. [21:35:30] (03CR) 10Gergő Tisza: "I think this broke CI for a bunch of extensions, e.g. https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CentralAuth/+/1132029 (which e" [integration/config] - 10https://gerrit.wikimedia.org/r/1130957 (https://phabricator.wikimedia.org/T389998) (owner: 10Krinkle) [21:35:50] Krinkle: ^ [21:35:56] ack, looking at it [21:36:07] could we revert https://gerrit.wikimedia.org/r/1130957 please until after we do the backport? [21:37:09] friday backports? [21:37:09] OK [21:37:24] (03PS1) 10Krinkle: Revert "zuul: Remove uncontrolled recursion of extra-dependencies in MW jobs" [integration/config] - 10https://gerrit.wikimedia.org/r/1132035 [21:37:30] (03CR) 10Krinkle: [C:03+2] Revert "zuul: Remove uncontrolled recursion of extra-dependencies in MW jobs" [integration/config] - 10https://gerrit.wikimedia.org/r/1132035 (owner: 10Krinkle) [21:38:43] (03Merged) 10jenkins-bot: Revert "zuul: Remove uncontrolled recursion of extra-dependencies in MW jobs" [integration/config] - 10https://gerrit.wikimedia.org/r/1132035 (owner: 10Krinkle) [21:38:50] (03CR) 10Krinkle: [C:03+2] "`$this->markTestSkippedIfExtensionNotLoaded( 'MobileFrontend' );`" [integration/config] - 10https://gerrit.wikimedia.org/r/1130957 (https://phabricator.wikimedia.org/T389998) (owner: 10Krinkle) [21:39:15] !log Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1130957 [21:39:16] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:47:34] (03PS1) 10Krinkle: zuul: Add MobileFrontend as extra for CentralAuth [integration/config] - 10https://gerrit.wikimedia.org/r/1132037 [21:47:58] (03PS2) 10Krinkle: zuul: Add MobileFrontend as extra for CentralAuth [integration/config] - 10https://gerrit.wikimedia.org/r/1132037 [21:48:09] (03CR) 10Krinkle: [C:03+2] zuul: Add MobileFrontend as extra for CentralAuth [integration/config] - 10https://gerrit.wikimedia.org/r/1132037 (owner: 10Krinkle) [21:48:40] (03CR) 10Krinkle: [C:03+2] "Revert, done." [integration/config] - 10https://gerrit.wikimedia.org/r/1130957 (https://phabricator.wikimedia.org/T389998) (owner: 10Krinkle) [21:49:25] (03Merged) 10jenkins-bot: zuul: Add MobileFrontend as extra for CentralAuth [integration/config] - 10https://gerrit.wikimedia.org/r/1132037 (owner: 10Krinkle) [21:51:22] 10Gerrit, 06Release-Engineering-Team, 06collaboration-services: Enable browser notifications system in Gerrit - https://phabricator.wikimedia.org/T389327#10687838 (10Quiddity) If I understand correctly, I believe the communication about this feature change, has been covered/resolved by [[https://lists.wi... [22:01:54] !log Disable duplicate publishing noise from extension-LoginNotify, ref T143162, T390315 [22:01:57] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:01:57] T143162: Reduce task notification noise/frequency of changes to associated open patchsets - https://phabricator.wikimedia.org/T143162 [22:01:57] T390315: LoginNotify: Provide configuration to deny login if attempt is made from new IP - https://phabricator.wikimedia.org/T390315 [23:00:34] 10Scap (SpiderPig 🕸️): SpiderPig UI: add pagination support - https://phabricator.wikimedia.org/T385029#10687951 (10thcipriani) 05Open→03Resolved a:03dancy I see a "load more history" button today. Looks like @dancy added in https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/688 [23:14:28] 10Release-Engineering-Team (Radar), 10Scap (SpiderPig 🕸️), 06Infrastructure-Foundations, 07LDAP: Create 'spiderpig-users' ldap group - https://phabricator.wikimedia.org/T390338 (10thcipriani) 03NEW [23:15:22] 10Release-Engineering-Team (Doing 😎), 10Scap (SpiderPig 🕸️), 13Patch-For-Review: Scap SpiderPig: Daemonize services for deployment hosts - https://phabricator.wikimedia.org/T383945#10687983 (10thcipriani) 05In progress→03Resolved