[00:13:05] Is it just me, or does doing `composer test /path/to/file/you/want/to/check.php` no longer work? [00:13:10] in mediawiki core [01:49:59] 06Release-Engineering-Team, 06SRE: docker-registry will show different last updated time as you refresh the page... - https://phabricator.wikimedia.org/T404011#11160935 (10Reedy) [06:16:25] 10Phabricator, 06Project-Admins: Create project tag for ESEAP Hub - https://phabricator.wikimedia.org/T403908#11161096 (10Biyanto-ESEAP) [06:35:23] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for [YOUR DEVELOPER ACCOUNT USERNAME HERE] - https://phabricator.wikimedia.org/T404038 (10Plutus) 03NEW [06:36:18] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for Cactusisme - https://phabricator.wikimedia.org/T404038#11161111 (10Plutus) [06:43:47] (03CR) 10Hashar: "I had a last look at `mwext-phpunit-coverage.sh` to see whether it would fit skin, and it does not cause it hardcoding `/extensions/` in t" [integration/config] - 10https://gerrit.wikimedia.org/r/1185925 (https://phabricator.wikimedia.org/T395470) (owner: 10Hashar) [06:48:02] 10GitLab (Account Approval), 10Release-Engineering-Team (Doing 😎): Requesting GitLab account activation for Cactusisme - https://phabricator.wikimedia.org/T404038#11161129 (10Aklapper) 05Open→03Resolved a:03Aklapper Approved! [06:49:27] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for SaintWolf - https://phabricator.wikimedia.org/T399836#11161135 (10Aklapper) 05Stalled→03Declined Unfortunately closing this Phabricator task as no further information has been provided. @SaintWolf: After... [06:55:22] (03PS1) 10Hashar: quibble-coverage: make mwext-phpunit-coverage usable by skins [integration/config] - 10https://gerrit.wikimedia.org/r/1186397 (https://phabricator.wikimedia.org/T402398) [06:56:12] (03CR) 10Hashar: [C:03+2] quibble-coverage: make mwext-phpunit-coverage usable by skins [integration/config] - 10https://gerrit.wikimedia.org/r/1186397 (https://phabricator.wikimedia.org/T402398) (owner: 10Hashar) [06:57:07] 10GitLab (Account Approval), 10Release-Engineering-Team (Doing 😎): Requesting GitLab account activation for Cactusisme - https://phabricator.wikimedia.org/T404038#11161153 (10Plutus) Hi Aklapper, I want to request to disable my 2nd dev account (which I had created unknowingly while trying to log onto Gitla... [06:59:08] 10Phabricator (Search), 10Release-Engineering-Team (Seen), 10observability, 07Browser-Tests: Develop tests for phabricator search to detect regressions / search quality issues - https://phabricator.wikimedia.org/T182160#11161154 (10Aklapper) > I have just pushed the first test to `T182160` branch. There i... [07:04:29] 10GitLab (Account Approval), 10Release-Engineering-Team (Doing 😎): Requesting GitLab account activation for Cactusisme - https://phabricator.wikimedia.org/T404038#11161161 (10Aklapper) I could block or ban the GitLab account https://gitlab.wikimedia.org/plutus if there was some way to prove that it was cre... [07:05:41] 10GitLab (Account Approval), 10Release-Engineering-Team (Doing 😎): Requesting GitLab account activation for Cactusisme - https://phabricator.wikimedia.org/T404038#11161162 (10Plutus) If there is a way to prove it p, I would. But not really sure how. [07:06:05] (03PS2) 10Hashar: quibble-coverage: make mwext-phpunit-coverage usable by skins [integration/config] - 10https://gerrit.wikimedia.org/r/1186397 (https://phabricator.wikimedia.org/T402398) [07:06:05] (03PS2) 10Hashar: jjb: switch skins coverage to mwext-phpunit-coverage [integration/config] - 10https://gerrit.wikimedia.org/r/1185925 (https://phabricator.wikimedia.org/T395470) [07:06:05] (03PS4) 10Hashar: zuul: switch skins coverage to same job as extensions [integration/config] - 10https://gerrit.wikimedia.org/r/1185926 (https://phabricator.wikimedia.org/T395470) [07:06:06] (03PS2) 10Hashar: zuul: remove remnant of 'mwskin' [integration/config] - 10https://gerrit.wikimedia.org/r/1185929 (https://phabricator.wikimedia.org/T402398) [07:06:07] (03PS2) 10Hashar: jjb: delete mwskin-phpunit-coverage-publish, unused [integration/config] - 10https://gerrit.wikimedia.org/r/1185928 (https://phabricator.wikimedia.org/T402398) [07:06:08] (03PS2) 10Hashar: quibble-coverage: remove mwskin-phpunit-coverage [integration/config] - 10https://gerrit.wikimedia.org/r/1185930 (https://phabricator.wikimedia.org/T402398) [07:06:12] (03PS2) 10Hashar: jjb: update jobs to remove mwskin-phpunit-coverage [integration/config] - 10https://gerrit.wikimedia.org/r/1185931 (https://phabricator.wikimedia.org/T402398) [07:06:16] (03PS1) 10Hashar: jjb: update quibble-coverage jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1186399 (https://phabricator.wikimedia.org/T402398) [07:08:24] (03CR) 10Hashar: [C:03+2] quibble-coverage: make mwext-phpunit-coverage usable by skins [integration/config] - 10https://gerrit.wikimedia.org/r/1186397 (https://phabricator.wikimedia.org/T402398) (owner: 10Hashar) [07:08:52] (03CR) 10Hashar: "Done with I39c802f9a2ebee0ce923cdf609f6aa05ba41918d and I have rebased the chain of patches." [integration/config] - 10https://gerrit.wikimedia.org/r/1185925 (https://phabricator.wikimedia.org/T395470) (owner: 10Hashar) [07:09:50] (03Merged) 10jenkins-bot: quibble-coverage: make mwext-phpunit-coverage usable by skins [integration/config] - 10https://gerrit.wikimedia.org/r/1186397 (https://phabricator.wikimedia.org/T402398) (owner: 10Hashar) [07:18:56] (03CR) 10Hashar: [C:03+2] jjb: update quibble-coverage jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1186399 (https://phabricator.wikimedia.org/T402398) (owner: 10Hashar) [07:20:19] (03Merged) 10jenkins-bot: jjb: update quibble-coverage jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1186399 (https://phabricator.wikimedia.org/T402398) (owner: 10Hashar) [07:25:18] (03CR) 10Hashar: [C:03+2] "The coverage publish worked for https://gerrit.wikimedia.org/r/c/mediawiki/extensions/IPReputation/+/1186400" [integration/config] - 10https://gerrit.wikimedia.org/r/1186399 (https://phabricator.wikimedia.org/T402398) (owner: 10Hashar) [07:28:01] (03CR) 10Hashar: [C:03+2] "I have verified it by manually rebuilding the last build of `mwskin-phpunit-coverage-publish` which was for https://gerrit.wikimedia.org/r" [integration/config] - 10https://gerrit.wikimedia.org/r/1185925 (https://phabricator.wikimedia.org/T395470) (owner: 10Hashar) [07:29:15] (03CR) 10Hashar: [C:03+2] zuul: switch skins coverage to same job as extensions [integration/config] - 10https://gerrit.wikimedia.org/r/1185926 (https://phabricator.wikimedia.org/T395470) (owner: 10Hashar) [07:30:27] (03Merged) 10jenkins-bot: jjb: switch skins coverage to mwext-phpunit-coverage [integration/config] - 10https://gerrit.wikimedia.org/r/1185925 (https://phabricator.wikimedia.org/T395470) (owner: 10Hashar) [07:30:47] (03Merged) 10jenkins-bot: zuul: switch skins coverage to same job as extensions [integration/config] - 10https://gerrit.wikimedia.org/r/1185926 (https://phabricator.wikimedia.org/T395470) (owner: 10Hashar) [07:30:48] 10GitLab (Integrations), 10Release-Engineering-Team (Radar), 06collaboration-services, 06Infrastructure-Foundations, 06serviceops: Container image reports in debmonitor are broken - https://phabricator.wikimedia.org/T348876#11161195 (10elukey) 05Open→03Resolved Getting back to this after all the... [07:30:50] (03CR) 10Hashar: [C:03+2] zuul: remove remnant of 'mwskin' [integration/config] - 10https://gerrit.wikimedia.org/r/1185929 (https://phabricator.wikimedia.org/T402398) (owner: 10Hashar) [07:32:21] (03Merged) 10jenkins-bot: zuul: remove remnant of 'mwskin' [integration/config] - 10https://gerrit.wikimedia.org/r/1185929 (https://phabricator.wikimedia.org/T402398) (owner: 10Hashar) [07:33:35] (03CR) 10Hashar: [C:03+2] "p" [integration/config] - 10https://gerrit.wikimedia.org/r/1185928 (https://phabricator.wikimedia.org/T402398) (owner: 10Hashar) [07:34:07] (03CR) 10Hashar: [C:03+2] quibble-coverage: remove mwskin-phpunit-coverage [integration/config] - 10https://gerrit.wikimedia.org/r/1185930 (https://phabricator.wikimedia.org/T402398) (owner: 10Hashar) [07:35:17] (03Merged) 10jenkins-bot: jjb: delete mwskin-phpunit-coverage-publish, unused [integration/config] - 10https://gerrit.wikimedia.org/r/1185928 (https://phabricator.wikimedia.org/T402398) (owner: 10Hashar) [07:35:33] (03Merged) 10jenkins-bot: quibble-coverage: remove mwskin-phpunit-coverage [integration/config] - 10https://gerrit.wikimedia.org/r/1185930 (https://phabricator.wikimedia.org/T402398) (owner: 10Hashar) [07:41:07] 10Phabricator (Search), 10Release-Engineering-Team (Seen), 10observability, 07Browser-Tests: Develop tests for phabricator search to detect regressions / search quality issues - https://phabricator.wikimedia.org/T182160#11161235 (10zeljkofilipin) As far as I can see from T182160#3834507, this was pushed to... [07:41:10] (03CR) 10Hashar: [C:03+2] jjb: update jobs to remove mwskin-phpunit-coverage [integration/config] - 10https://gerrit.wikimedia.org/r/1185931 (https://phabricator.wikimedia.org/T402398) (owner: 10Hashar) [07:42:31] (03Merged) 10jenkins-bot: jjb: update jobs to remove mwskin-phpunit-coverage [integration/config] - 10https://gerrit.wikimedia.org/r/1185931 (https://phabricator.wikimedia.org/T402398) (owner: 10Hashar) [08:22:50] 10Continuous-Integration-Config, 10MediaWiki-Core-Tests, 10MW-1.45-notes (1.45.0-wmf.16; 2025-08-26), 13Patch-For-Review, 07Technical-Debt: Drop deprecated phpunit.php and suite.xml - https://phabricator.wikimedia.org/T395470#11161393 (10hashar) [08:54:33] 10Phabricator: display "policy" group memberships in phabricator user profiles - https://phabricator.wikimedia.org/T403778#11161535 (10Aklapper) I'm leaning towards reverting downstream rPHAB55ed8751cebce2198c8d2566542a309c85efc765 and rPHABa1304fc3756cc6a9e768f97a6731c027fb33197c as I do not see much advantage... [08:54:34] 06Release-Engineering-Team, 06serviceops, 06SRE: docker-registry "Last updated at" text hiding under scrollbar - https://phabricator.wikimedia.org/T404008#11161533 (10LSobanski) [08:54:43] 06Release-Engineering-Team, 06serviceops, 06SRE: docker-registry "Last updated at" time should specify TZ - https://phabricator.wikimedia.org/T404010#11161536 (10LSobanski) [08:54:51] 06Release-Engineering-Team, 06serviceops, 06SRE: docker-registry will show different last updated time as you refresh the page... - https://phabricator.wikimedia.org/T404011#11161537 (10LSobanski) [10:30:50] 10Continuous-Integration-Config, 06Growth-Team, 10Metrics Platform: Add MetricsPlatform to gated extensions - https://phabricator.wikimedia.org/T404058 (10Sgs) 03NEW [10:32:13] (03PS2) 10Sergio Gimeno: zuul: [mediawiki/extensions/MetricsPlatform] Add to gate [integration/config] - 10https://gerrit.wikimedia.org/r/1185945 (https://phabricator.wikimedia.org/T404058) [10:34:54] 10Continuous-Integration-Config, 06Growth-Team, 10Metrics Platform, 13Patch-For-Review: Add MetricsPlatform to gated extensions - https://phabricator.wikimedia.org/T404058#11161935 (10Sgs) [10:35:16] (03CR) 10Sergio Gimeno: "Oh apologies, here it is T404058. I followed https://www.mediawiki.org/wiki/Continuous_integration/Tutorials/Adding_your_extension_to_the_" [integration/config] - 10https://gerrit.wikimedia.org/r/1185945 (https://phabricator.wikimedia.org/T404058) (owner: 10Sergio Gimeno) [10:36:27] 10Continuous-Integration-Config, 06Growth-Team, 10Metrics Platform, 13Patch-For-Review: Add MetricsPlatform to gated extensions - https://phabricator.wikimedia.org/T404058#11161953 (10Sgs) p:05Triage→03Medium [10:55:09] 06Project-Admins, 07Tracking-Neverending: Requests for addition to the #acl*Project-Admins group (in comments) - https://phabricator.wikimedia.org/T706#11162045 (10Ladsgroup) >>! In T706#11160549, @HFan-WMF wrote: > Hi there, I'm the new product manager on the Reader Experience team, and would love to be able... [10:56:18] 10Continuous-Integration-Config, 06Growth-Team, 10Metrics Platform, 13Patch-For-Review: Add MetricsPlatform to gated extensions - https://phabricator.wikimedia.org/T404058#11162047 (10Sgs) [11:44:25] (03CR) 10Jforrester: "Oh wow, I'd never seen that tutorial before. I've added a notice about how it's a major step and generally rejected. Thanks for the link " [integration/config] - 10https://gerrit.wikimedia.org/r/1185945 (https://phabricator.wikimedia.org/T404058) (owner: 10Sergio Gimeno) [11:45:12] 10Continuous-Integration-Config, 06Growth-Team, 10Metrics Platform, 13Patch-For-Review: Add MetricsPlatform to gated extensions - https://phabricator.wikimedia.org/T404058#11162214 (10Jdforrester-WMF) [11:45:14] 10Continuous-Integration-Config, 10Release-Engineering-Team (Seen), 10MW-on-K8s, 07Epic, 13Patch-For-Review: Have all Wikimedia production extensions and skins in the CI gate - https://phabricator.wikimedia.org/T249674#11162215 (10Jdforrester-WMF) [11:45:24] hashar: ^^ You may want to weigh in there too. [11:49:31] FIRING: [2x] ProbeDown: Service gerrit2003:443 has failed probes (http_gerrit_tls_ip4) - https://wikitech.wikimedia.org/wiki/Runbook#gerrit2003:443 - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://alerts.wikimedia.org/?q=alertname%3DProbeDown [11:49:37] 06Release-Engineering-Team, 06collaboration-services: ProbeDown - https://phabricator.wikimedia.org/T404070 (10phaultfinder) 03NEW [11:50:06] still me ↑, handling it [11:58:16] James_F: I guess that task is largely obsolete by now [11:58:24] hashar: Which task? [11:58:30] I think the idea I had a decade ago was to run every single tests of every extensions together [11:58:37] to ensure they would never fail [11:58:42] err [11:58:46] T249674 [11:58:46] T249674: Have all Wikimedia production extensions and skins in the CI gate - https://phabricator.wikimedia.org/T249674 [12:01:20] there are MetricsPlatform and ReadingLists asking to enter the gate ( https://gerrit.wikimedia.org/r/c/integration/config/+/1185945 and https://gerrit.wikimedia.org/r/c/integration/config/+/1184176 ) [12:02:04] Tyler raised the case of ReadingLists to me yesterday during our 1/1 [12:02:25] and I wrote a couple comments on https://phabricator.wikimedia.org/T403560#11158237 [12:02:30] Oh, right, yeah. [12:02:40] Well, I think it's not an unreasonable long-term aim. [12:02:53] I think it might be sufficient to tests the repos both way [12:02:58] But I definitely think we should make clear that it's not going to happen any time soon, and needs major investment. [12:02:59] aka ReadingLists + its dependencies [12:03:05] and for its dependencies, ensure they run ReadingLists tests [12:03:21] which I think should be done in the next Zuul [12:04:03] also the bundlesize thing is broken by design [12:04:11] Yes, next Zuul definitely, not now. [12:04:14] :) [12:05:21] but for the bundlesize we could consider having a job that clones all extensions and verifies the bundles [12:05:29] Eh. We could. [12:05:39] A new job is generally something we're trying to avoid. [12:05:46] which is I guess why they want to add ReadingLists there [12:07:50] maybe it is worth adding a job dedicated to bundlesize [12:08:14] I guess that still needs a complete mediawiki installation to have resourceloader loaded and ready [12:09:12] and MetricsPlatform might want to be added as well [12:09:25] since I imagine lot of extensions will end up using it [12:09:45] I need to be off for a bit [12:19:31] RESOLVED: [2x] ProbeDown: Service gerrit2003:443 has failed probes (http_gerrit_tls_ip4) - https://wikitech.wikimedia.org/wiki/Runbook#gerrit2003:443 - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://alerts.wikimedia.org/?q=alertname%3DProbeDown [12:28:24] 10Gerrit, 06collaboration-services: refresh Gerrit dashboards - improve monitoring on gerrit-spare - https://phabricator.wikimedia.org/T398854#11162434 (10ABran-WMF) [12:40:03] maintenance-disconnect-full-disks build 735538 integration-agent-docker-1044 (/: 33%, /srv: 97%, /var/lib/docker: 28%): OFFLINE due to disk space [12:45:03] maintenance-disconnect-full-disks build 735539 integration-agent-docker-1044 (/: 33%, /srv: 44%, /var/lib/docker: 28%): RECOVERY disk space OK [14:16:55] (03PS1) 10Jforrester: Docker: Bump Node base images to September releases and cascade [integration/config] - 10https://gerrit.wikimedia.org/r/1186521 [14:29:19] (03PS2) 10Jforrester: Docker: Bump Node base images to September releases and cascade [integration/config] - 10https://gerrit.wikimedia.org/r/1186521 [14:29:29] (03CR) 10Jforrester: [C:03+2] Docker: Bump Node base images to September releases and cascade [integration/config] - 10https://gerrit.wikimedia.org/r/1186521 (owner: 10Jforrester) [14:31:09] (03Merged) 10jenkins-bot: Docker: Bump Node base images to September releases and cascade [integration/config] - 10https://gerrit.wikimedia.org/r/1186521 (owner: 10Jforrester) [14:31:30] !og Docker: Bump Node base images to September releases and cascade [14:31:33] Eurgh. [14:31:37] !log Docker: Bump Node base images to September releases and cascade [14:31:38] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [14:32:02] (03PS1) 10Jforrester: jjb: Switch images to Node new September releases everywhere [integration/config] - 10https://gerrit.wikimedia.org/r/1186524 [14:43:03] (03PS2) 10Jforrester: jjb: Switch images to Node new September releases everywhere [integration/config] - 10https://gerrit.wikimedia.org/r/1186524 [14:45:05] 10Release-Engineering-Team (Priority Backlog 📥), 05Release, 05Train Deployments: 1.45.0-wmf.18 deployment blockers - https://phabricator.wikimedia.org/T396379#11163139 (10Reedy) [15:19:11] (03PS5) 10Hashar: Script to sync dependencies with extension registry [integration/config] - 10https://gerrit.wikimedia.org/r/1182115 (https://phabricator.wikimedia.org/T389998) [15:19:11] (03PS1) 10Hashar: zuul: add OOJSPlus deps explicitly [integration/config] - 10https://gerrit.wikimedia.org/r/1186534 (https://phabricator.wikimedia.org/T389998) [15:21:06] (03CR) 10CI reject: [V:04-1] Script to sync dependencies with extension registry [integration/config] - 10https://gerrit.wikimedia.org/r/1182115 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [15:21:49] (03CR) 10Hashar: "This is to reduce the diff from Ia268663f87efbf5f6d4e37a4a4ffff2efddbb16e (zuul: Remove uncontrolled recursion in extra deps in MW jobs (t" [integration/config] - 10https://gerrit.wikimedia.org/r/1186534 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [15:22:13] (03PS1) 10Jforrester: Provide fresh-node24 [fresh] - 10https://gerrit.wikimedia.org/r/1186535 [15:22:13] (03PS1) 10Jforrester: fresh-node20,fresh-node22: Update to Node.js 20.19.5 and 22.19.0 [fresh] - 10https://gerrit.wikimedia.org/r/1186536 [15:22:14] (03PS1) 10Jforrester: Prepare v25.09.1 release [fresh] - 10https://gerrit.wikimedia.org/r/1186537 [15:22:25] (03PS6) 10Hashar: Script to sync dependencies with extension registry [integration/config] - 10https://gerrit.wikimedia.org/r/1182115 (https://phabricator.wikimedia.org/T389998) [15:27:12] (03PS2) 10Jforrester: Prepare v25.09.1 release [fresh] - 10https://gerrit.wikimedia.org/r/1186537 [15:27:12] (03PS1) 10Jforrester: Release v25.09.1 [fresh] - 10https://gerrit.wikimedia.org/r/1186538 [15:28:49] (03PS2) 10Jforrester: Provide fresh-node24 [fresh] - 10https://gerrit.wikimedia.org/r/1186535 [15:28:50] (03PS2) 10Jforrester: fresh-node20,fresh-node22: Update to Node.js 20.19.5 and 22.19.0 [fresh] - 10https://gerrit.wikimedia.org/r/1186536 [15:28:50] (03PS3) 10Jforrester: Prepare v25.09.1 release [fresh] - 10https://gerrit.wikimedia.org/r/1186537 [15:28:50] (03PS2) 10Jforrester: Release v25.09.1 [fresh] - 10https://gerrit.wikimedia.org/r/1186538 [15:35:27] (03open) 10dancy: branch-cut-test-patches: Remove patches pre-commit hook [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/212 (https://phabricator.wikimedia.org/T401672) [15:35:30] (03update) 10dancy: branch-cut-test-patches: Remove patches pre-commit hook [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/212 (https://phabricator.wikimedia.org/T401672) [15:39:28] (03CR) 10Jforrester: [C:04-2] Release v25.09.1 [fresh] - 10https://gerrit.wikimedia.org/r/1186538 (owner: 10Jforrester) [15:39:32] (03CR) 10Jforrester: [C:04-1] Prepare v25.09.1 release [fresh] - 10https://gerrit.wikimedia.org/r/1186537 (owner: 10Jforrester) [15:41:16] (03merge) 10dancy: branch-cut-test-patches: Remove patches pre-commit hook [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/212 (https://phabricator.wikimedia.org/T401672) [15:42:36] (03CR) 10Krinkle: [C:03+1] zuul: add OOJSPlus deps explicitly [integration/config] - 10https://gerrit.wikimedia.org/r/1186534 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [15:45:13] (03CR) 10Hslater: [C:03+1] zuul: add OOJSPlus deps explicitly [integration/config] - 10https://gerrit.wikimedia.org/r/1186534 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [15:47:29] (03CR) 10Jforrester: [C:03+2] jjb: Switch images to Node new September releases everywhere [integration/config] - 10https://gerrit.wikimedia.org/r/1186524 (owner: 10Jforrester) [15:48:55] (03Merged) 10jenkins-bot: jjb: Switch images to Node new September releases everywhere [integration/config] - 10https://gerrit.wikimedia.org/r/1186524 (owner: 10Jforrester) [15:50:42] (03CR) 10Jforrester: [C:03+2] zuul: add OOJSPlus deps explicitly [integration/config] - 10https://gerrit.wikimedia.org/r/1186534 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [15:52:12] (03Merged) 10jenkins-bot: zuul: add OOJSPlus deps explicitly [integration/config] - 10https://gerrit.wikimedia.org/r/1186534 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [15:52:39] !log zuul: add OOJSPlus deps explicitly, for T389998 [15:52:41] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:52:42] T389998: Allow control over which extra extensions are installed (Math REL1_43 jobs exceed 60min timeout) - https://phabricator.wikimedia.org/T389998 [15:55:36] it looks to me like https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CognitiveProcessDesigner/+/1186453 got force-merged and wedged gate-and-submit. assuming i'm right about that, somebody remind me of correct fix? [15:56:50] I don't think you'll need to explicitely do anything, it'll just take a while as zuul will have to re-run a bunch of jobs then ://// [15:57:16] Yup, force-merges just make Zuul-cloner's output wrong and cause all running gated jobs to be invalid. [15:57:28] gotcha [15:57:29] So eventually it should fix itself. [15:57:32] /me glances in direction of T403196 [15:57:32] T403196: Disable meaningless, never green CI jobs for BlueSpice (Hallo Welt! GmbH) extensions - https://phabricator.wikimedia.org/T403196 [15:57:42] Indeed. [15:59:03] or we could restrict the ability to force merge patches... [16:00:35] Indeed. [16:02:19] If Cindy were still here I would nudge her to trout Vogel :/ [16:04:59] semi-relatedly, it seems like the Wikibase patch before it is stuck somehow? https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php83/12150/console [16:05:37] Unless there's a batch of phpunit tests that are just ridiculously slow and haven't returned.. [16:05:56] It'd be awesome if we had better visibility into the split groups [16:06:22] Some of the tests are slow, and the splitting isn't time-based. [16:07:23] 10Continuous-Integration-Config, 10BlueSpice, 07ci-test-error: Disable meaningless, never green CI jobs for BlueSpice (Hallo Welt! GmbH) extensions - https://phabricator.wikimedia.org/T403196#11163626 (10bd808) This task came up on IRC today after @Osnard force merged https://gerrit.wikimedia.org/r/c/mediawi... [16:08:37] 10Beta-Cluster-Infrastructure, 10Automoderator, 10Moderator-Tools-Team (Kanban), 07Wikimedia-production-error: AutoModerator WikiPageConfig::getConfigData failed to load config from wiki: {error} - https://phabricator.wikimedia.org/T403756#11163636 (10Samwalton9-WMF) @Krinkle can you confirm if this looks... [16:10:48] 10Beta-Cluster-Infrastructure, 10Automoderator, 10Moderator-Tools-Team (Kanban), 07Wikimedia-production-error: AutoModerator WikiPageConfig::getConfigData failed to load config from wiki: {error} - https://phabricator.wikimedia.org/T403756#11163676 (10Krinkle) The Logstash URL is in the task description. V... [16:11:00] 10Continuous-Integration-Config, 10BlueSpice, 07ci-test-error: Disable meaningless, never green CI jobs for BlueSpice (Hallo Welt! GmbH) extensions - https://phabricator.wikimedia.org/T403196#11163680 (10Jdforrester-WMF) Disabling CI in the master branch (that BS don't actively use), but retaining it in the... [16:12:07] I'm going to abort https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php83/12150/console as it's clearly stuck (no console updates in > 40 mins). [16:12:43] Which will reset the gate even more, sadly. (Or rather, it was already going to do this, my action just brings it forward in time.) [16:12:46] 10WikimediaDebug, 06Wikipedia-Android-App-Backlog: Add WikimediaDebug functionality to the Wikimedia Android app - https://phabricator.wikimedia.org/T403824#11163693 (10Seddon) p:05Triage→03Medium [16:12:59] 10Beta-Cluster-Infrastructure, 10Automoderator, 10Moderator-Tools-Team (Kanban), 07Wikimedia-production-error: AutoModerator WikiPageConfig::getConfigData failed to load config from wiki: {error} - https://phabricator.wikimedia.org/T403756#11163695 (10Samwalton9-WMF) 05In progress→03Resolved Thanks... [16:13:38] > It'd be awesome if we had better visibility into the split groups -- this does feel like a currently missing thing. [16:14:07] 06Release-Engineering-Team, 10MediaWiki-Interwiki, 06translatewiki.net: Merge Interwiki extension into MediaWiki core - https://phabricator.wikimedia.org/T33951#11163701 (10Pppery) 05In progress→03Resolved [16:15:17] 10Continuous-Integration-Infrastructure: Better visibility into split group status in CI - https://phabricator.wikimedia.org/T404107 (10Reedy) 03NEW [16:16:44] 10Continuous-Integration-Infrastructure: Better visibility into PHPUnit split group status in CI - https://phabricator.wikimedia.org/T404107#11163761 (10Reedy) [16:17:31] 10Continuous-Integration-Infrastructure: Better visibility into PHPUnit split group status in CI - https://phabricator.wikimedia.org/T404107#11163770 (10bd808) [16:21:30] (03CR) 10Jforrester: "When I run this locally I get:" [integration/config] - 10https://gerrit.wikimedia.org/r/1182115 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [16:24:12] !log Remove unused deployment-snapshot- prefix Hiera file. https://gerrit.wikimedia.org/r/plugins/gitiles/cloud/instance-puppet/+/0c6a99c4e12e9d1a55ce353e302d7dc891ae1177 - Follows-up decom of deployment-snapshot05 host. ref T398438, T360995 [16:24:16] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:24:17] T398438: Decommission or recommission all snapshot and dumpsdata servers - https://phabricator.wikimedia.org/T398438 [16:24:17] T360995: Migrate Wikimedia production from PHP 8.1 to PHP 8.3 - https://phabricator.wikimedia.org/T360995 [17:03:38] 10Continuous-Integration-Config, 06Growth-Team, 10Metrics Platform, 13Patch-For-Review: Add MetricsPlatform to gated extensions - https://phabricator.wikimedia.org/T404058#11164064 (10Urbanecm_WMF) > @Jdforrester-WMF wrote on Gerrit > This needs a significant discussion first, and then sign-off from Releas... [17:05:32] (03CR) 10Urbanecm: "Moved discussion to T404058#11164063 (and responded)." [integration/config] - 10https://gerrit.wikimedia.org/r/1185945 (https://phabricator.wikimedia.org/T404058) (owner: 10Sergio Gimeno) [17:21:09] 06Release-Engineering-Team, 10Catalyst (PatchDemo): Enable creation of a patchdemo with all repos rolled back to a specific custom moment in time - https://phabricator.wikimedia.org/T403854#11164150 (10thcipriani) I think this might "just work" if we convert branch refs to tag refs instead of deleting tag refs... [17:29:32] (03CR) 10Umherirrender: Zuul: [../extensions/AutoModerator] Add CommunityConfiguration for phan (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/1186034 (owner: 10Umherirrender) [18:06:56] 10GitLab (Account Approval), 10Release-Engineering-Team (Doing 😎): Requesting GitLab account activation for Cactusisme - https://phabricator.wikimedia.org/T404038#11164395 (10bd808) >>! In T404038#11161161, @Aklapper wrote: > I could block or ban the GitLab account https://gitlab.wikimedia.org/plutus if th... [18:06:59] > It'd be awesome if we had better visibility into the split groups -- I'm late to the party, but we would eventually get that for free when we switch to paratest (T50217), currently blocked by the same stuff that blocks the PHPUnit 10 update. [18:07:00] T50217: Speed up MediaWiki PHPUnit build by running integration tests in parallel - https://phabricator.wikimedia.org/T50217 [18:07:22] I'm not sure if it's worthwhile investing more time and effort into our custom PHPUnit splitter, as opposed to just getting the PHPUnit update done. [18:07:38] Fair. [18:10:17] Daimona: Please link/comment on the task :P [18:10:22] If we get stuff for free, even better [18:10:24] I'm doing that rn :D [18:10:32] <3 [18:10:33] I saw this here before phab [18:10:45] UPGRADE ALL THE THINGS [18:12:07] 10GitLab (Account Approval), 10Release-Engineering-Team (Doing 😎): Requesting GitLab account activation for Cactusisme - https://phabricator.wikimedia.org/T404038#11164436 (10bd808) Deep links to individual logs are not obviously possible, but https://idm.wikimedia.org/wikimedia/log/ now shows the disable... [18:12:50] 10Continuous-Integration-Infrastructure: Better visibility into PHPUnit split group status in CI - https://phabricator.wikimedia.org/T404107#11164442 (10Daimona) I commented in IRC before seeing this task: > we would eventually get that for free when we switch to paratest (T50217), currently blocked by the same... [18:13:50] We get this for "free" by paying the update cost, yes :D [18:21:35] Huh, the gate-and-submit queue just reset again [18:22:30] I wanna find what test caused it and go burn it [18:22:44] Aka drop Wikibase from the gate? [18:23:15] https://integration.wikimedia.org/ci/job/wmf-quibble-selenium-php81/38601/console yep that's it [18:23:25] Why does it always have to be selenium? [18:24:03] T389998 would do I guess [18:24:04] T389998: Allow control over which extra extensions are installed (Math REL1_43 jobs exceed 60min timeout) - https://phabricator.wikimedia.org/T389998 [18:24:54] tbh we should make something that automatically disables the most flappy selenium tests [18:25:16] I think at this point it's not the tests themselves, but the infrastructure. I'm seeing failures all over the place [18:26:11] Still, waiting the entire day for a patch to be merged isn't great. [18:30:46] 10Phabricator, 10Release-Engineering-Team (Doing 😎), 13Patch-For-Review: Identify and update EditEngine Forms with disabled project tags or subscribers or assignee - https://phabricator.wikimedia.org/T403887#11164544 (10Dzahn) changes deployed on phab1004 [18:36:24] hi folks, we're had a DonationInterface patch stuck on submit for a few hours now: https://integration.wikimedia.org/zuul/#q=1186051 [18:36:25] 10Continuous-Integration-Infrastructure (Zuul upgrade), 10Release-Engineering-Team (Priority Backlog 📥), 06collaboration-services: Deploy zuul executor on executor VM - https://phabricator.wikimedia.org/T403847#11164579 (10Dzahn) >>! In T403847#11160535, @thcipriani wrote: > where are you seeing that error?... [18:36:37] any idea what's holding it up? [18:36:52] the patch before it in the queue :-) [18:37:45] ejegg: Everyone has, sadly. [18:37:51] ejegg: as far as I can tell, your patch was +2'd about half an hour ago, not "some hours" ago? [18:38:25] we've removed and re-added the C+2 taavi [18:38:36] yes, and by doing that you reset it to the back of the queue [18:38:59] 10Continuous-Integration-Infrastructure (Zuul upgrade), 10Release-Engineering-Team (Priority Backlog 📥), 06collaboration-services: Deploy zuul executor on executor VM - https://phabricator.wikimedia.org/T403847#11164585 (10Dzahn) edit: Oh wait, I need to fix the path to the private key file. WIP [18:39:44] ahh, i didn't realize it was still queued at that point - there was nothing in the zuul search screen for it when I re-C+2ed it [18:40:12] is there somewhere else you can see them queued? [18:40:16] 10Phabricator, 06Release-Engineering-Team: Merge Phorge's upstream master into our wmf/stable - https://phabricator.wikimedia.org/T404134 (10brennen) 03NEW [18:40:30] https://integration.wikimedia.org/zuul/ [18:41:12] ok, so if it is gone from that screen it's dropped off the queue? [18:41:23] anyway, it's merged now [18:41:28] oh nice! [18:41:35] thank you [18:45:53] 10Phabricator, 06Release-Engineering-Team: Merge Phorge's upstream master into our wmf/stable - https://phabricator.wikimedia.org/T404134#11164630 (10brennen) [18:47:06] OH MY GOD, and of course the next run in gate-and-submit hit the random selenium failure [18:47:24] No, actually, "npm error 503 Service Temporarily Unavailable - GET https://registry.npmjs.org/supports-color" [18:47:49] and the other due to "21:33:12 You are not running the required node version" [18:47:58] Still doesn't change the fact that I've been waiting to see this damn patch merged for the entire day [18:48:53] And yeah, I saw that other one too and I don't even begin to understand what it means [18:49:12] Or rather, I understand what it means. I don't understand why it's happening. [18:49:38] Oh, damn, I didn't push some nvmrc patches. [18:49:47] We switched earlier today from 20.19.1 to 20.19.5. [18:50:21] Now you're telling me we have a real, non-random CI failure? I don't think I'll believe that. [18:50:26] Ha. [18:50:34] (03open) 10brennen: Merge Phorge upstream master as of 2025-09-08 into wmf/stable [repos/phabricator/phabricator] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/merge_requests/99 (https://phabricator.wikimedia.org/T404134) [18:50:44] Daimona: mind if I bash that? [18:50:47] (03update) 10brennen: Merge Phorge upstream master as of 2025-09-08 into wmf/stable [repos/phabricator/phabricator] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/merge_requests/99 (https://phabricator.wikimedia.org/T404134) [18:51:14] Sure [18:51:23] !bash Now you're telling me we have a real, non-random CI failure? I don't think I'll believe that. [18:51:23] taavi: Stored quip at https://bash.toolforge.org/quip/8gjRL5kBffdvpiTrgd_B [18:51:36] I better go make dinner instead of ranting :D [18:51:56] haha [18:52:11] Then I'll come back to see what's the new exciting CI failure that blocked my patch again [18:56:54] (03open) 10dduvall: Update debian base image used in blubber.yaml [repos/releng/k8s-pvc-cleaner] - 10https://gitlab.wikimedia.org/repos/releng/k8s-pvc-cleaner/-/merge_requests/32 [18:57:04] (03update) 10dduvall: Update debian base image used in blubber.yaml [repos/releng/k8s-pvc-cleaner] - 10https://gitlab.wikimedia.org/repos/releng/k8s-pvc-cleaner/-/merge_requests/32 [18:59:49] (03update) 10dduvall: Update debian base image used in blubber.yaml [repos/releng/k8s-pvc-cleaner] - 10https://gitlab.wikimedia.org/repos/releng/k8s-pvc-cleaner/-/merge_requests/32 [19:02:28] 10Continuous-Integration-Infrastructure (Zuul upgrade), 10Release-Engineering-Team (Priority Backlog 📥), 06collaboration-services, 13Patch-For-Review: Deploy zuul executor on executor VM - https://phabricator.wikimedia.org/T403847#11164757 (10Dzahn) running bubblewrap directly (`zuul-wrap /tmp id` mentione... [19:03:56] .nvmrc fixes, for the interested: https://gerrit.wikimedia.org/r/q/topic:%22nvmrc-20.19.5%22 [19:06:06] 10Continuous-Integration-Infrastructure (Zuul upgrade), 10Release-Engineering-Team (Priority Backlog 📥), 06collaboration-services, 13Patch-For-Review: Deploy zuul executor on executor VM - https://phabricator.wikimedia.org/T403847#11164777 (10Dzahn) //"On systems with AppArmor (common on Ubuntu), it may be... [19:24:26] (03approved) 10dancy: Update debian base image used in blubber.yaml [repos/releng/k8s-pvc-cleaner] - 10https://gitlab.wikimedia.org/repos/releng/k8s-pvc-cleaner/-/merge_requests/32 (owner: 10dduvall) [19:48:56] 10Continuous-Integration-Infrastructure (Zuul upgrade), 10Release-Engineering-Team (Priority Backlog 📥), 06collaboration-services: Deploy zuul executor on executor VM - https://phabricator.wikimedia.org/T403847#11164972 (10Dzahn) I tried: - adding --security-opt seccomp=unconfined to docker command ("//... [19:50:17] (03merge) 10dduvall: Update debian base image used in blubber.yaml [repos/releng/k8s-pvc-cleaner] - 10https://gitlab.wikimedia.org/repos/releng/k8s-pvc-cleaner/-/merge_requests/32 [20:09:34] (03open) 10dancy: create-databases: Revise for reliability [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/156 [20:09:37] (03update) 10dancy: create-databases: Revise for reliability [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/156 [20:11:25] (03merge) 10dancy: create-databases: Revise for reliability [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/156 [20:18:10] (03open) 10dancy: Install a specific version of helm, during container build [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/157 [20:18:13] (03update) 10dancy: Install a specific version of helm, during container build [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/157 [20:19:14] (03merge) 10dancy: Install a specific version of helm, during container build [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/157 [20:59:17] 06Release-Engineering-Team, 06serviceops, 06SRE: docker-registry will show different last updated time as you refresh the page... - https://phabricator.wikimedia.org/T404011#11165251 (10bd808) `lang=shell-session bd808@deploy1003:~$ curl -s https://docker-registry.discovery.wmnet | grep 'Last updated at'... [21:21:41] (03open) 10dancy: Drop www and mw-debug containers [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/158 [21:21:44] (03update) 10dancy: Drop www and mw-debug containers [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/158 [21:22:48] (03merge) 10dancy: Drop www and mw-debug containers [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/158 [21:29:35] 06Release-Engineering-Team, 06serviceops, 06SRE: docker-registry will show different last updated time as you refresh the page... - https://phabricator.wikimedia.org/T404011#11165359 (10bd808) Those content pages are generated by [[https://gerrit.wikimedia.org/g/operations/puppet/+/refs/heads/production/modu... [21:31:21] 06Release-Engineering-Team, 06serviceops, 06SRE: docker-registry will show different last updated time as you refresh the page... - https://phabricator.wikimedia.org/T404011#11165361 (10bd808) 05Open→03Invalid Closing as "works as designed". I guess reopen if the timer skew feels problematic and not... [21:43:31] 10Release-Engineering-Team (Radar), 10Cloud-VPS (Quota-requests): Additional floating IPs for gitlab-cloud-runner testing in testlabs project - https://phabricator.wikimedia.org/T404150 (10dduvall) 03NEW [21:44:18] 10Phabricator, 10Release-Engineering-Team (Doing 😎): Identify and update EditEngine Forms with disabled project tags or subscribers or assignee - https://phabricator.wikimedia.org/T403887#11165407 (10Aklapper) 05Open→03Resolved [21:52:15] 10Phabricator, 06Release-Engineering-Team, 13Patch-For-Review: Merge Phorge's upstream master into our wmf/stable - https://phabricator.wikimedia.org/T404134#11165420 (10Aklapper) [21:53:03] 10Release-Engineering-Team (Priority Backlog 📥), 06collaboration-services: Upgrade phab (phorge) hosts to bookworm - https://phabricator.wikimedia.org/T372619#11165423 (10Aklapper) [21:53:06] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥): Update to Phorge upstream 2025.18 release - https://phabricator.wikimedia.org/T386558#11165425 (10Aklapper) [21:53:08] 10Phabricator, 06Release-Engineering-Team, 13Patch-For-Review: Merge Phorge's upstream master into our wmf/stable - https://phabricator.wikimedia.org/T404134#11165424 (10Aklapper) [21:53:13] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥): Update to Phorge upstream 2025.18 release - https://phabricator.wikimedia.org/T386558#11165427 (10Aklapper) [21:53:16] 10Phabricator, 06Release-Engineering-Team, 13Patch-For-Review: Merge Phorge's upstream master into our wmf/stable - https://phabricator.wikimedia.org/T404134#11165426 (10Aklapper) [22:28:54] 10Beta-Cluster-Infrastructure: Re-create poolcounter instance in Beta Cluster (deployment-prep) - https://phabricator.wikimedia.org/T380881#11165565 (10Krinkle) Noise while monitoring after PHP 8.3 upgrade in Beta Cluster, I noticed tons of `mwmaint` and `jobrunner` procs fail/degrade due to PoolCounter absence,... [22:33:21] 10Continuous-Integration-Infrastructure (Zuul upgrade), 10Release-Engineering-Team (Priority Backlog 📥), 06collaboration-services: Deploy zuul executor on executor VM - https://phabricator.wikimedia.org/T403847#11165572 (10Dzahn) Checked if this also happens on trixie with newer kernel version and on cloud V... [22:41:32] 10Continuous-Integration-Infrastructure (Zuul upgrade), 10Release-Engineering-Team (Priority Backlog 📥), 06collaboration-services: Deploy zuul executor on executor VM - https://phabricator.wikimedia.org/T403847#11165587 (10thcipriani) Played around a bit with this. It looks like zuul-bwrap is trying to use b... [22:43:48] 10Continuous-Integration-Infrastructure (Zuul upgrade), 10Release-Engineering-Team (Priority Backlog 📥), 06collaboration-services: Deploy zuul executor on executor VM - https://phabricator.wikimedia.org/T403847#11165603 (10Dzahn) Found what makes a difference! ` sudo sysctl -w kernel.unprivileged_userns_cl... [22:55:49] dduvall: I have a blubber + kokkuri question that I'd bet you can answer quickly. Is there a way to declare an ARG in blubber and pass a value for it to kokkuri? My imagined use case is creating a .netrc file as part of a builders.custom.script build step. [22:56:26] * bd808 is tired of github ratelimit problems in a container build [22:57:49] ARGS are not supported yet but there were plans to change that [23:01:58] ack. I thought that might be where things are. I added some stuff that exposes specific args for the RUNS_AS stuff in the past, but didn't make generic ARG handling at that time. [23:05:37] just because I once asked when trying to "translate" an upstream Dockerfile that made heavy use of ARGS [23:06:06] Dan said for now I can use literals instead [23:09:12] yeah. I might just decide to make the PAT for this github access public and just hope it isn't found by the whole world. There is one in Jenkins CI already that I'm pretty sure someone could extract with some test code if they really wanted. [23:10:05] * bd808 grumbles some more about github as a software distribution hub for various systems [23:14:37] 10Continuous-Integration-Infrastructure (Zuul upgrade), 10Release-Engineering-Team (Priority Backlog 📥), 06collaboration-services, 13Patch-For-Review: Deploy zuul executor on executor VM - https://phabricator.wikimedia.org/T403847#11165664 (10Dzahn) @thcipriani ` sudo docker run --privileged --mount typ... [23:18:47] * p858snake|cloud reverts bd808 back to sourceforge [23:19:09] 10Continuous-Integration-Infrastructure (Zuul upgrade), 10Release-Engineering-Team (Priority Backlog 📥), 06collaboration-services, 13Patch-For-Review: Deploy zuul executor on executor VM - https://phabricator.wikimedia.org/T403847#11165684 (10thcipriani) huzzah! Nice work @Dzahn [23:22:15] p858snake|cloud: fair, but also I don't think Packagist, golang, et al really thought about "what will happen when GitHub stops making it simple for anon downloads from their platform?" [23:23:41] hiding auth tokens in many obscure config files is yucky. trying to make it transparent for devs in a CI setting is horrible when you do it all in public like we do [23:45:05] 10Release-Engineering-Team (Priority Backlog 📥), 07OKR-Work: Audit alerting for `wmf/next` branch creation/update failures to ensure that failures are visible to RelEng and other interested parties - https://phabricator.wikimedia.org/T399970#11165742 (10bd808) 05Open→03Resolved Closing per the T399970#... [23:59:22] 10GitLab (Integrations), 10Release-Engineering-Team (Priority Backlog 📥), 10Wikibugs: Connect WikiBugs IRC bot to Wikimedia GitLab - https://phabricator.wikimedia.org/T288381#11165776 (10bd808) 05Open→03Resolved Let's call this done. {T364615} tracks the biggest missing aspect which itself needs {T36...