[00:05:13] (03update) 10project_16_bot_841eade40a22c56a6272172b8cb67384: Bump docker-registry.wikimedia.org/repos/data-engineering/eventgate-wikimedia from v1.36.0 to v1.38.0 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/798 [00:05:15] (03update) 10project_16_bot_841eade40a22c56a6272172b8cb67384: Bump docker-registry.wikimedia.org/releng/quibble-bookworm-php83 from 1.22.0-s8 to 1.22.0-s9 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/799 [00:05:17] (03open) 10project_16_bot_841eade40a22c56a6272172b8cb67384: Bump golang from 1.26.10 to 1.26.11 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/805 [00:05:25] (03open) 10project_16_bot_841eade40a22c56a6272172b8cb67384: Bump traefik from v3.7.17 to v3.7.18 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/806 [00:05:27] (03open) 10project_16_bot_841eade40a22c56a6272172b8cb67384: Bump docker-registry.wikimedia.org/repos/search-platform/cirrussearch-opensearch-image from v2.19.5-3 to v2.19.5-4 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/807 [00:05:31] (03open) 10project_16_bot_841eade40a22c56a6272172b8cb67384: Bump prom/prometheus from v3.15.1 to v3.16.0 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/808 [00:05:35] (03open) 10project_16_bot_841eade40a22c56a6272172b8cb67384: Bump docker-registry.wikimedia.org/wikimedia/mediawiki-libs-shellbox image group to 2026-11-10-141215 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/809 [00:05:37] (03update) 10project_16_bot_841eade40a22c56a6272172b8cb67384: Bump traefik from v3.7.17 to v3.7.18 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/806 [00:05:41] (03update) 10project_16_bot_841eade40a22c56a6272172b8cb67384: Bump docker-registry.wikimedia.org/repos/search-platform/cirrussearch-opensearch-image from v2.19.5-3 to v2.19.5-4 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/807 [00:05:45] (03update) 10project_16_bot_841eade40a22c56a6272172b8cb67384: Bump prom/prometheus from v3.15.1 to v3.16.0 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/808 [00:05:47] (03update) 10project_16_bot_841eade40a22c56a6272172b8cb67384: Bump docker-registry.wikimedia.org/wikimedia/mediawiki-libs-shellbox image group to 2026-11-10-141215 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/809 [00:05:48] (03update) 10project_16_bot_841eade40a22c56a6272172b8cb67384: Bump docker-registry.wikimedia.org/repos/data-engineering/eventgate-wikimedia from v1.36.0 to v1.38.0 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/798 [00:05:49] (03update) 10project_16_bot_841eade40a22c56a6272172b8cb67384: Bump docker-registry.wikimedia.org/releng/quibble-bookworm-php83 from 1.22.0-s8 to 1.22.0-s9 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/799 [03:52:49] 10Beta-Cluster-Infrastructure: Beta cluster is being crawled to death by bot traffic coming from residential proxies - https://phabricator.wikimedia.org/T420833#12547202 (10AlexisJazz) >>! In T420833#12542901, @bd808 wrote: > I have [[https://github.com/commons-app/apps-android-commons/issues/7304|opened a task... [06:37:10] 10Beta-Cluster-Infrastructure: Beta cluster is being crawled to death by bot traffic coming from residential proxies - https://phabricator.wikimedia.org/T420833#12547315 (10Nicolas_Raoul) I hope I won't get banned for saying this, but: Recently, to perform realistic integration tests, there is no alternative to... [06:59:52] (03CR) 10Jforrester: [C:04-1] Use generic-node26 for the WdTmCollab lab tool (032 comments) [integration/config] - 10https://gerrit.wikimedia.org/r/1368765 (https://phabricator.wikimedia.org/T444548) (owner: 10Wandji collins) [08:21:53] (03PS1) 10Jforrester: Docker: [node26] Update Node to 26.13.1 [integration/config] - 10https://gerrit.wikimedia.org/r/1368930 [08:22:44] (03CR) 10Jforrester: [C:03+2] Docker: [node26] Update Node to 26.13.1 [integration/config] - 10https://gerrit.wikimedia.org/r/1368930 (owner: 10Jforrester) [08:24:29] (03Merged) 10jenkins-bot: Docker: [node26] Update Node to 26.13.1 [integration/config] - 10https://gerrit.wikimedia.org/r/1368930 (owner: 10Jforrester) [08:24:53] !log Docker: [node26] Update Node to 26.13.1, for T444199 [08:24:54] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [08:29:00] hashar: There are a bunch of jobs not yet moved to latest docker images for the re-build around maven/java. Should I land those now? [08:36:34] (03PS1) 10Jforrester: Dockerfiles: [node26] Actually set the new Node version, oops [integration/config] - 10https://gerrit.wikimedia.org/r/1368938 [08:36:44] (03CR) 10Jforrester: [C:03+2] Dockerfiles: [node26] Actually set the new Node version, oops [integration/config] - 10https://gerrit.wikimedia.org/r/1368938 (owner: 10Jforrester) [08:38:40] (03Merged) 10jenkins-bot: Dockerfiles: [node26] Actually set the new Node version, oops [integration/config] - 10https://gerrit.wikimedia.org/r/1368938 (owner: 10Jforrester) [08:39:37] !log Docker: [node26] Update Node to 26.13.1 take II, for T444199 [08:39:38] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [08:52:27] I'm seeing cache corruption on mediawiki-node26 jobs [08:52:41] https://integration.wikimedia.org/ci/job/mediawiki-node26/12218/console https://integration.wikimedia.org/ci/job/mediawiki-node26/12219/console [08:53:01] could that be related to your changes James_F? or should I just blow away the castor cache [08:54:47] (03PS2) 10Jforrester: jjb: update jobs for "rebuild all base images" [integration/config] - 10https://gerrit.wikimedia.org/r/1368780 (https://phabricator.wikimedia.org/T444391) (owner: 10Hashar) [08:54:47] (03PS1) 10Jforrester: jjb: Switch to images with Node 26.13.1 [integration/config] - 10https://gerrit.wikimedia.org/r/1368945 (https://phabricator.wikimedia.org/T444199) [08:55:11] codders: Iโ€™ve not deployed yet, so not from me, no. [08:55:28] codders: If you clear the cache that might fix it, yes. [08:57:04] k. then I'll do that. thanks! [08:58:24] !log integration: integration-castor06: rm -fR /srv/castor/castor-mw-ext-and-skins/master/mediawiki-node26 [08:58:25] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [09:00:28] yeah - looks better [09:01:23] Excellent. [09:05:29] (03CR) 10Jforrester: [C:03+2] jjb: update jobs for "rebuild all base images" [integration/config] - 10https://gerrit.wikimedia.org/r/1368780 (https://phabricator.wikimedia.org/T444391) (owner: 10Hashar) [09:07:27] (03Merged) 10jenkins-bot: jjb: update jobs for "rebuild all base images" [integration/config] - 10https://gerrit.wikimedia.org/r/1368780 (https://phabricator.wikimedia.org/T444391) (owner: 10Hashar) [09:07:58] (03PS2) 10Jforrester: jjb: Switch to images with Node 26.13.1 [integration/config] - 10https://gerrit.wikimedia.org/r/1368945 (https://phabricator.wikimedia.org/T444199) [09:09:00] 06Release-Engineering-Team (Doing ๐Ÿ˜Ž), 06WMF-NDA-Requests: Requesting access to WMF-NDA project in Phabricator for Raymond_Ndibe - https://phabricator.wikimedia.org/T444467#12547584 (10Raymond_Ndibe) >>! In T444467#12546837, @Aklapper wrote: > Hi, looking at https://phabricator.wikimedia.org/p/Raymond_Ndibe/, y... [09:11:10] (03CR) 10Jforrester: [C:03+2] jjb: Switch to images with Node 26.13.1 [integration/config] - 10https://gerrit.wikimedia.org/r/1368945 (https://phabricator.wikimedia.org/T444199) (owner: 10Jforrester) [09:12:05] ccccccknjiurrkceuljeetltinfektrjgruheuncvknf [09:12:11] ccccccknjiurikfcglltdjbbfledbubginvcjdblejcb [09:12:20] Oops,. [09:12:59] Top tip: Donโ€™t brush against the side of your computer. [09:14:05] (03Merged) 10jenkins-bot: jjb: Switch to images with Node 26.13.1 [integration/config] - 10https://gerrit.wikimedia.org/r/1368945 (https://phabricator.wikimedia.org/T444199) (owner: 10Jforrester) [09:18:59] 06Release-Engineering-Team (Doing ๐Ÿ˜Ž), 10Catalyst (Luka Ijo Pimeja Jan), 07Essential-Work: [Oct 2026] Upgrade K3s cluster to most recent stable version - https://phabricator.wikimedia.org/T426866#12547647 (10jnuche) [09:23:10] 06Release-Engineering-Team (Doing ๐Ÿ˜Ž), 10Catalyst (Luka Ijo Pimeja Jan), 07Essential-Work: [Oct 2026] Upgrade K3s cluster to most recent stable version - https://phabricator.wikimedia.org/T426866#12547690 (10jnuche) We're targeting [[ https://github.com/k3s-io/k3s/releases/tag/v1.37.2%2Bk3s1 | v1.37.2+k3s1 ]] [09:35:33] 06Release-Engineering-Team (Doing ๐Ÿ˜Ž), 06WMF-NDA-Requests: Requesting access to WMF-NDA project in Phabricator for Raymond_Ndibe - https://phabricator.wikimedia.org/T444467#12547780 (10Aklapper) 05Openโ†’03Resolved Thank you a lot! :) [09:46:47] (03open) 10jforrester: patchdemo-trixie-node24-php*-apache2: Re-build for Node 24.22.3 [repos/releng/dev-images] - 10https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/136 (https://phabricator.wikimedia.org/T444199) [09:47:40] (03merge) 10jforrester: patchdemo-trixie-node24-php*-apache2: Re-build for Node 24.22.3 [repos/releng/dev-images] - 10https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/136 (https://phabricator.wikimedia.org/T444199) [10:54:15] 10GitLab, 06Release-Engineering-Team (Radar), 06Collaboration-Services, 13Patch-For-Review: gitlab behind CDN - https://phabricator.wikimedia.org/T425441#12548203 (10brouberol) [11:00:24] 06Release-Engineering-Team (Doing ๐Ÿ˜Ž), 07Essential-Work, 05Release, 05Train Deployments: 1.48.0-wmf.4 deployment blockers - https://phabricator.wikimedia.org/T442222#12548240 (10Aklapper) 05Openโ†’03Resolved How uneventful [11:05:05] (03PS4) 10Mpostoronca: Zuul: [mediawiki/extensions/WikimediaAntiAbuse] Add GlobalBlocking, CentralAuth [integration/config] - 10https://gerrit.wikimedia.org/r/1367849 (https://phabricator.wikimedia.org/T443475) [11:23:47] (03PS5) 10Mpostoronca: Zuul: [mediawiki/extensions/WikimediaAntiAbuse] Add GlobalBlocking, CentralAuth [integration/config] - 10https://gerrit.wikimedia.org/r/1367849 (https://phabricator.wikimedia.org/T443475) [11:24:31] (03PS6) 10Mpostoronca: Zuul: [mediawiki/extensions/WikimediaAntiAbuse] Add GlobalBlocking, CentralAuth [integration/config] - 10https://gerrit.wikimedia.org/r/1367849 (https://phabricator.wikimedia.org/T443475) [11:25:43] (03PS7) 10Dreamy Jazz: Zuul: [mediawiki/extensions/WikimediaAntiAbuse] Add GlobalBlocking, CentralAuth [integration/config] - 10https://gerrit.wikimedia.org/r/1367849 (https://phabricator.wikimedia.org/T443475) (owner: 10Mpostoronca) [11:26:24] (03CR) 10Dreamy Jazz: [C:03+1] Zuul: [mediawiki/extensions/WikimediaAntiAbuse] Add GlobalBlocking, CentralAuth [integration/config] - 10https://gerrit.wikimedia.org/r/1367849 (https://phabricator.wikimedia.org/T443475) (owner: 10Mpostoronca) [11:48:05] 10Continuous-Integration-Config, 10MediaWiki-Docker, 10MediaWiki-Vendor, 07Composer, 05MW-1.48-notes (1.48.0-wmf.5; 2026-11-17): Update composer from 2.10.3 to 2.11.0 - https://phabricator.wikimedia.org/T442976#12548426 (10Jdforrester-WMF) [12:02:52] (03update) 10jforrester: fundraising: Re-build for libbrotli1, and install Node.js 22 from nodejs.org [repos/releng/dev-images] - 10https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/133 (https://phabricator.wikimedia.org/T442495) [13:18:52] 10Phabricator: Disable H481 - https://phabricator.wikimedia.org/T444227#12548813 (10NicoV) Hi @Aklapper , I've disabled the rule, but I don't find in the notifications or email settings an option to receive an email when a task is added to WPCleaner project. I missed several tasks added to the project without it... [13:22:37] (03CR) 10Bartosz Wรณjtowicz: [C:03+1] inference-services: Remove llm, qwen36 and gpt-oss-safeguard jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1368701 (https://phabricator.wikimedia.org/T444471) (owner: 10Gkyziridis) [13:23:51] 10Phabricator: Disable H481 - https://phabricator.wikimedia.org/T444227#12548819 (10VeritasSapientiae) Check your email settings. [13:27:54] 10Phabricator: Disable H481 - https://phabricator.wikimedia.org/T444227#12548832 (10NicoV) >>! In T444227#12548813, @NicoV wrote: > I don't find in the notifications or email settings So, which option allows me to receive an email each time a new task appears in the WPCleaner project, without getting emails for... [13:30:13] 10Phabricator, 06Moderator-Tools-Team, 10The-Wikipedia-Library: Move all tasks from Library-Card-Platform to The-Wikipedia-Library - https://phabricator.wikimedia.org/T441640#12548837 (10Aklapper) >>! In T441640#12511058, @Samwalton9-WMF wrote: > Would there be any benefit to making #the-wikipedia-librar... [13:39:04] 10Phabricator: Disable H481 - https://phabricator.wikimedia.org/T444227#12548857 (10Aklapper) Hmmm. You're listed as a watcher and member on https://phabricator.wikimedia.org/project/members/4842/, so in theory you should receive notifications about tasks in WPCleaner, if you didn't disable many things under "Ma... [13:47:06] 10Phabricator: Disable H481 - https://phabricator.wikimedia.org/T444227#12548885 (10NicoV) 05Openโ†’03Resolved Thanks @Aklapper . I will check that. [14:17:09] 10MediaWiki-Releasing, 10MediaWiki-General, 06MediaWiki-Core-Platform-Team (Radar), 13Patch-For-Review: Remove unused deprecated methods after 1.47 release - https://phabricator.wikimedia.org/T444454#12549006 (10JTweed-WMF) [14:21:59] (03CR) 10Hashar: [C:03+2] Zuul: [mediawiki/extensions/WikimediaAntiAbuse] Add GlobalBlocking, CentralAuth [integration/config] - 10https://gerrit.wikimedia.org/r/1367849 (https://phabricator.wikimedia.org/T443475) (owner: 10Mpostoronca) [14:22:47] (03CR) 10Hashar: [C:03+2] inference-services: Remove llm, qwen36 and gpt-oss-safeguard jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1368701 (https://phabricator.wikimedia.org/T444471) (owner: 10Gkyziridis) [14:23:48] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/WikimediaAntiAbuse] Add GlobalBlocking, CentralAuth [integration/config] - 10https://gerrit.wikimedia.org/r/1367849 (https://phabricator.wikimedia.org/T443475) (owner: 10Mpostoronca) [14:23:54] (03Merged) 10jenkins-bot: inference-services: Remove llm, qwen36 and gpt-oss-safeguard jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1368701 (https://phabricator.wikimedia.org/T444471) (owner: 10Gkyziridis) [14:23:59] (03open) 10esanders: runner: Support repositories with an ESLint flat config [repos/ci-tools/libup] - 10https://gitlab.wikimedia.org/repos/ci-tools/libup/-/merge_requests/128 (https://phabricator.wikimedia.org/T267941) [14:43:42] (03PS1) 10Hashar: Use "fresh-node" in tests and doc [fresh] - 10https://gerrit.wikimedia.org/r/1369078 [14:43:42] (03PS1) 10Hashar: fresh-node20: remove and uninstall [fresh] - 10https://gerrit.wikimedia.org/r/1369079 [14:47:42] (03PS2) 10Hashar: Remove and uninstall fresh-node20 and fresh-node22 [fresh] - 10https://gerrit.wikimedia.org/r/1369079 [14:50:09] 06Project-Admins, 10Orchestrator: Re-label "#orchestrator" to be less generic so it's less confusing - https://phabricator.wikimedia.org/T444624 (10Jdforrester-WMF) 03NEW [15:04:27] 06Project-Admins, 06DBA, 10Orchestrator: Re-label "#orchestrator" to be less generic so it's less confusing - https://phabricator.wikimedia.org/T444624#12549346 (10Aklapper) Cool with me, let's CC #DBA if they have any objections [15:08:01] (03PS1) 10Hashar: Update Node 24 and 26 to latest images [fresh] - 10https://gerrit.wikimedia.org/r/1369082 [15:09:04] 06Project-Admins, 06DBA, 10Orchestrator: Re-label "#orchestrator" to be less generic so it's less confusing - https://phabricator.wikimedia.org/T444624#12549358 (10Marostegui) No problem from my side, go ahead! [15:10:50] (03PS2) 10Hashar: Prepare v26.11.1 release [fresh] - 10https://gerrit.wikimedia.org/r/1369082 [15:11:22] 06Project-Admins, 10database-orchestrator, 06DBA: Re-label "#orchestrator" to be less generic so it's less confusing - https://phabricator.wikimedia.org/T444624#12549367 (10Jdforrester-WMF) I've added `database-orchestrator` as the primary label, with the old label as an alias. Should we leave it a week or s... [15:17:17] (03PS3) 10Hashar: Prepare v26.11.1 release [fresh] - 10https://gerrit.wikimedia.org/r/1369082 [15:18:49] (03PS1) 10Hashar: Release 26.11.1 [fresh] - 10https://gerrit.wikimedia.org/r/1369083 [15:19:48] (03PS4) 10Hashar: Prepare 26.11.1 release [fresh] - 10https://gerrit.wikimedia.org/r/1369082 [15:19:48] (03PS2) 10Hashar: Release 26.11.1 [fresh] - 10https://gerrit.wikimedia.org/r/1369083 [15:19:56] (03CR) 10Hashar: [C:03+2] Use "fresh-node" in tests and doc [fresh] - 10https://gerrit.wikimedia.org/r/1369078 (owner: 10Hashar) [15:20:05] (03CR) 10Hashar: [C:03+2] Remove and uninstall fresh-node20 and fresh-node22 [fresh] - 10https://gerrit.wikimedia.org/r/1369079 (owner: 10Hashar) [15:20:09] (03CR) 10Hashar: [C:03+2] Prepare 26.11.1 release [fresh] - 10https://gerrit.wikimedia.org/r/1369082 (owner: 10Hashar) [15:20:13] (03CR) 10Hashar: [C:03+2] Release 26.11.1 [fresh] - 10https://gerrit.wikimedia.org/r/1369083 (owner: 10Hashar) [15:47:47] 10Phabricator, 07SecTeam-Processed, 07Security: Move the #acl_security_volunteer policy outside of #acl_security - https://phabricator.wikimedia.org/T305890#12549598 (10sbassett) a:03sbassett [15:49:53] (03CR) 10CI reject: [V:04-1] Release 26.11.1 [fresh] - 10https://gerrit.wikimedia.org/r/1369083 (owner: 10Hashar) [15:55:20] (03Merged) 10jenkins-bot: Use "fresh-node" in tests and doc [fresh] - 10https://gerrit.wikimedia.org/r/1369078 (owner: 10Hashar) [16:06:19] (03Merged) 10jenkins-bot: Remove and uninstall fresh-node20 and fresh-node22 [fresh] - 10https://gerrit.wikimedia.org/r/1369079 (owner: 10Hashar) [16:06:19] (03Merged) 10jenkins-bot: Prepare 26.11.1 release [fresh] - 10https://gerrit.wikimedia.org/r/1369082 (owner: 10Hashar) [16:11:48] (03Merged) 10jenkins-bot: Release 26.11.1 [fresh] - 10https://gerrit.wikimedia.org/r/1369083 (owner: 10Hashar) [16:47:22] 10Beta-Cluster-Infrastructure: Beta cluster is being crawled to death by bot traffic coming from residential proxies - https://phabricator.wikimedia.org/T420833#12549846 (10bd808) >>! In T420833#12547202, @AlexisJazz wrote: > If beta cluster ends up whitelisted for WMF only, Tim's wild idea was Tim's wild idea.... [16:52:41] 06Release-Engineering-Team (Seen), 10phan, 06Security-Team, 07Documentation, 07Wikimedia-extension-review-queue: Document that Phan is required for WMF deployed and MW bundled extensions - https://phabricator.wikimedia.org/T250038#12549878 (10sbassett) >>! In T250038#12543551, @Nikerabbit wrote: > Phan i... [16:54:11] 10GitLab (Infrastructure), 06Collaboration-Services, 06Security-Team, 07SecTeam-Processed, and 2 others: GitLab Security Release: 19.5.4, 19.4.6, 19.3.8 - https://phabricator.wikimedia.org/T444442#12549886 (10sbassett) [16:54:12] 10GitLab (Infrastructure), 06Collaboration-Services, 06Security-Team, 07SecTeam-Processed, and 2 others: GitLab Security Release: 19.5.4, 19.4.6, 19.3.8 - https://phabricator.wikimedia.org/T444442#12549887 (10sbassett) [16:57:10] (03update) 10ejegg: fundraising: Re-build for libbrotli1, and install Node.js 22 from nodejs.org [repos/releng/dev-images] - 10https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/133 (https://phabricator.wikimedia.org/T442495) (owner: 10jforrester) [16:57:11] (03approved) 10ejegg: fundraising: Re-build for libbrotli1, and install Node.js 22 from nodejs.org [repos/releng/dev-images] - 10https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/133 (https://phabricator.wikimedia.org/T442495) (owner: 10jforrester) [16:57:25] (03merge) 10ejegg: fundraising: Re-build for libbrotli1, and install Node.js 22 from nodejs.org [repos/releng/dev-images] - 10https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/133 (https://phabricator.wikimedia.org/T442495) (owner: 10jforrester) [17:01:15] 10Beta-Cluster-Infrastructure: Beta cluster is being crawled to death by bot traffic coming from residential proxies - https://phabricator.wikimedia.org/T420833#12549946 (10bd808) >>! In T420833#12547315, @Nicolas_Raoul wrote: > I hope I won't get banned for saying this, but: > Recently, to perform realistic int... [17:11:26] (03open) 10project_16_bot_622d7b6d682aa66907ad1bffedf5b9e6: build(deps): bump golang.org/x/image from 0.47.0 to 0.48.0 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/810 [17:11:33] (03update) 10project_16_bot_622d7b6d682aa66907ad1bffedf5b9e6: build(deps): bump golang.org/x/image from 0.47.0 to 0.48.0 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/810 [17:11:35] (03update) 10project_16_bot_622d7b6d682aa66907ad1bffedf5b9e6: build(deps): bump golang.org/x/image from 0.47.0 to 0.48.0 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/810 [17:15:54] (03open) 10project_16_bot_622d7b6d682aa66907ad1bffedf5b9e6: build(deps): bump github.com/gofrs/flock from 0.13.1 to 0.13.2 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/811 [17:15:54] (03update) 10project_16_bot_622d7b6d682aa66907ad1bffedf5b9e6: build(deps): bump github.com/gofrs/flock from 0.13.1 to 0.13.2 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/811 [17:19:29] (03open) 10brennen: catalyst-base and children: rebuild [repos/releng/dev-images] - 10https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/137 (https://phabricator.wikimedia.org/T444199) [17:19:52] (03update) 10jnuche: catalyst-base and children: rebuild [repos/releng/dev-images] - 10https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/137 (https://phabricator.wikimedia.org/T444199) (owner: 10brennen) [17:19:53] (03approved) 10jnuche: catalyst-base and children: rebuild [repos/releng/dev-images] - 10https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/137 (https://phabricator.wikimedia.org/T444199) (owner: 10brennen) [17:21:57] 06Release-Engineering-Team (Seen), 10phan, 06Security-Team, 07Documentation, 07Wikimedia-extension-review-queue: Document that Phan is required for WMF deployed and MW bundled extensions - https://phabricator.wikimedia.org/T250038#12550030 (10Rsilvola) [17:22:41] (03PS1) 10Hslater: Zuul: [BlueSpiceExtendedStatistics] Add HitCounters dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1369107 [17:23:16] (03open) 10project_16_bot_622d7b6d682aa66907ad1bffedf5b9e6: build(deps): bump golang.org/x/term from 0.47.0 to 0.48.0 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/812 [17:23:18] (03update) 10project_16_bot_622d7b6d682aa66907ad1bffedf5b9e6: build(deps): bump golang.org/x/term from 0.47.0 to 0.48.0 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/812 [17:25:28] (03merge) 10brennen: catalyst-base and children: rebuild [repos/releng/dev-images] - 10https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/137 (https://phabricator.wikimedia.org/T444199) [17:26:52] !log Updating development images on contint primary for https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/137 (T444199) [17:26:54] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:41:23] (03merge) 10dancy: gitlab-runner: update gitlab-runner image to alpine-v19.5.3 [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/665 (https://phabricator.wikimedia.org/T444442) (owner: 10jelto) [18:06:08] 10GitLab (Infrastructure), 06Collaboration-Services, 06Security-Team, 07SecTeam-Processed, and 2 others: GitLab Security Release: 19.5.4, 19.4.6, 19.3.8 - https://phabricator.wikimedia.org/T444442#12550165 (10dancy) https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/665 has... [18:19:24] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team (Radar), 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, and 2 others: CI build slow and timing out sometimes - https://phabricator.wikimedia.org/T434024#12550232 (10pwangai) [18:28:08] (03PS1) 10Pwangai: Docker: [civicrm-trixie] Run the tests in parallel [integration/config] - 10https://gerrit.wikimedia.org/r/1369116 (https://phabricator.wikimedia.org/T442953) [18:28:10] (03PS1) 10Pwangai: jjb: Switch the civicrm jobs to civicrm-trixie 0.1.1 [integration/config] - 10https://gerrit.wikimedia.org/r/1369117 (https://phabricator.wikimedia.org/T442953) [19:50:13] (03open) 10dancy: config: Remove the unused `scap_source_dir` option [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1427 [19:50:31] (03update) 10dancy: config: Remove the unused `scap_source_dir` option [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1427 [19:53:13] (03merge) 10dancy: config: Remove the unused `scap_source_dir` option [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1427 [20:02:03] (03CR) 10Krinkle: "(Marking as Done). FWIW, I indeed use bin/fresh-node22 like this as well." [fresh] - 10https://gerrit.wikimedia.org/r/1347594 (https://phabricator.wikimedia.org/T439828) (owner: 10Hashar) [20:05:08] FIRING: [3x] PuppetAgentFailure: Puppet agent failure detected on instance deployment-cp-text09 in project deployment-prep - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentFailure [20:12:37] (03PS1) 10Krinkle: fresh-install: Restore muted grey for install steps [fresh] - 10https://gerrit.wikimedia.org/r/1369132 (https://phabricator.wikimedia.org/T439828) [20:16:10] (03PS1) 10Krinkle: fresh-install: factor out a verbose() function [fresh] - 10https://gerrit.wikimedia.org/r/1369133 [20:16:15] (03PS3) 10Wandji collins: Zuul: [labs/tools/WdTmCollab] Replace tox with generic-node26 [integration/config] - 10https://gerrit.wikimedia.org/r/1368765 (https://phabricator.wikimedia.org/T444548) [20:16:39] (03CR) 10Wandji collins: Zuul: [labs/tools/WdTmCollab] Replace tox with generic-node26 (032 comments) [integration/config] - 10https://gerrit.wikimedia.org/r/1368765 (https://phabricator.wikimedia.org/T444548) (owner: 10Wandji collins) [20:34:55] (03PS1) 10Krinkle: docs: Fix welcome text instruction [fresh] - 10https://gerrit.wikimedia.org/r/1369134 (https://phabricator.wikimedia.org/T439828) [20:53:26] (03open) 10dancy: deploy: Pull docker-registry.discovery.wmnet images from the public registry [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/208 [20:54:03] (03update) 10dancy: deploy: Pull docker-registry.discovery.wmnet images from the public registry [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/208 [20:54:26] (03merge) 10dancy: deploy: Pull docker-registry.discovery.wmnet images from the public registry [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/208 [20:54:32] (03CR) 10Jforrester: [C:03+2] "Aha, sorry, thank you!" [integration/config] - 10https://gerrit.wikimedia.org/r/1368765 (https://phabricator.wikimedia.org/T444548) (owner: 10Wandji collins) [20:55:03] (03open) 10dancy: deploy: Add deployer01-otp script [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/209 [20:55:20] (03update) 10dancy: deploy: Add deployer01-otp script [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/209 [20:55:46] (03merge) 10dancy: deploy: Add deployer01-otp script [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/209 [20:56:20] (03Merged) 10jenkins-bot: Zuul: [labs/tools/WdTmCollab] Replace tox with generic-node26 [integration/config] - 10https://gerrit.wikimedia.org/r/1368765 (https://phabricator.wikimedia.org/T444548) (owner: 10Wandji collins) [20:57:38] !log Zuul: [labs/tools/WdTmCollab] Replace tox with generic-node26, for T444548 [20:57:40] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:57:41] T444548: Move from tox to generic-node26 - https://phabricator.wikimedia.org/T444548 [20:58:59] (03open) 10dancy: deploy: Make testrepo deployable [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/210 [20:59:13] (03update) 10dancy: deploy: Make testrepo deployable [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/210 [21:00:01] (03merge) 10dancy: deploy: Make testrepo deployable [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/210 [21:01:03] (03open) 10dancy: exp: Disable registry cleanup in scap clean-images [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/211 [21:01:09] (03update) 10dancy: exp: Disable registry cleanup in scap clean-images [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/211 [21:01:51] (03merge) 10dancy: exp: Disable registry cleanup in scap clean-images [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/211 [21:18:43] (03open) 10dancy: exp: Fetch only the needed LFS files for the Gerrit image [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/212 (https://phabricator.wikimedia.org/T443677) [21:21:20] (03merge) 10dancy: exp: Fetch only the needed LFS files for the Gerrit image [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/212 (https://phabricator.wikimedia.org/T443677) [21:23:06] FIRING: PuppetAgentStaleLastRun: Last Puppet run was over 24 hours ago on instance deployment-webperf31 in project deployment-prep - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentStaleLastRun [21:23:49] (03open) 10dancy: spiderpig: Check the event snapshot before querying a job for its log [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1428 [21:24:33] (03update) 10dancy: spiderpig: Check the event snapshot before querying a job for its log [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1428 [21:26:13] (03merge) 10dancy: spiderpig: Check the event snapshot before querying a job for its log [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1428 [21:26:41] (03open) 10dancy: spiderpig: Measure the stale logstash poll age from the current time [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1429 [21:29:10] (03CR) 10Ahmon Dancy: [C:03+1] fresh-install: Restore muted grey for install steps [fresh] - 10https://gerrit.wikimedia.org/r/1369132 (https://phabricator.wikimedia.org/T439828) (owner: 10Krinkle) [21:33:37] (03CR) 10Jforrester: "Do you want to deploy this as a second, experimental set of jobs first to check it works before merging? Or is the cut-over not high-risk?" [integration/config] - 10https://gerrit.wikimedia.org/r/1369117 (https://phabricator.wikimedia.org/T442953) (owner: 10Pwangai) [21:41:16] (03update) 10dancy: spiderpig: Measure the stale logstash poll age from the current time [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1429 [21:41:19] (03update) 10dancy: spiderpig: Measure the stale logstash poll age from the current time [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1429 [21:41:39] (03update) 10dancy: spiderpig: Measure the stale logstash poll age from the current time [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1429 [21:43:39] (03CR) 10Lars SG: "We're comfortable deploying these directly if Peter is, based on the extensive testing Peter has done." [integration/config] - 10https://gerrit.wikimedia.org/r/1369117 (https://phabricator.wikimedia.org/T442953) (owner: 10Pwangai) [21:43:45] (03merge) 10dancy: spiderpig: Measure the stale logstash poll age from the current time [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1429 [21:44:39] (03CR) 10Jforrester: "Awesome, let's ship it." [integration/config] - 10https://gerrit.wikimedia.org/r/1369117 (https://phabricator.wikimedia.org/T442953) (owner: 10Pwangai) [21:44:43] (03CR) 10Jforrester: [C:03+2] Docker: [civicrm-trixie] Run the tests in parallel [integration/config] - 10https://gerrit.wikimedia.org/r/1369116 (https://phabricator.wikimedia.org/T442953) (owner: 10Pwangai) [21:45:54] (03Merged) 10jenkins-bot: Docker: [civicrm-trixie] Run the tests in parallel [integration/config] - 10https://gerrit.wikimedia.org/r/1369116 (https://phabricator.wikimedia.org/T442953) (owner: 10Pwangai) [21:47:58] !log Docker: [civicrm-trixie] Run the tests in parallel, for T442953 [21:48:01] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:48:01] T442953: Run the fundraising CRM phpunit tests in parallel - https://phabricator.wikimedia.org/T442953 [22:04:06] FIRING: PuppetAgentNoResources: No Puppet resources found on instance deployment-webperf31 on project deployment-prep - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentNoResources [22:04:41] (03CR) 10Jforrester: [C:03+2] "Deployed, test runs ran successfully:" [integration/config] - 10https://gerrit.wikimedia.org/r/1369117 (https://phabricator.wikimedia.org/T442953) (owner: 10Pwangai) [22:06:31] (03Merged) 10jenkins-bot: jjb: Switch the civicrm jobs to civicrm-trixie 0.1.1 [integration/config] - 10https://gerrit.wikimedia.org/r/1369117 (https://phabricator.wikimedia.org/T442953) (owner: 10Pwangai) [22:19:35] (03PS1) 10Vaughn Walters: jjb: [catalyst-daily-WikibaseQualityConstraints] Add the job [integration/config] - 10https://gerrit.wikimedia.org/r/1369147 (https://phabricator.wikimedia.org/T444390) [22:19:35] (03CR) 10Vaughn Walters: [C:03+2] "Tests are live and passing on catalyst https://integration.wikimedia.org/ci/view/selenium-daily/job/catalyst-daily-WikibaseQualityConstrai" [integration/config] - 10https://gerrit.wikimedia.org/r/1369147 (https://phabricator.wikimedia.org/T444390) (owner: 10Vaughn Walters) [22:21:22] (03Merged) 10jenkins-bot: jjb: [catalyst-daily-WikibaseQualityConstraints] Add the job [integration/config] - 10https://gerrit.wikimedia.org/r/1369147 (https://phabricator.wikimedia.org/T444390) (owner: 10Vaughn Walters) [22:51:07] FIRING: PuppetStaleCertificates: Found non-revoked Puppet certificates for 1 deleted instances on deployment-puppetserver-1 - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/PuppetStaleCertificates - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetStaleCertificates [22:55:11] bd808: looks like the BSD folks have implemented essentially what I pitched for beta cluster as anti-scraper. A simple edge rendered form with a self-service button that sets a cookie. https://freshbsd.org/news/2026/11/06 via https://fosstodon.org/@vinyl_cache/117602658193474410 [22:59:36] The only thing id do different is make it two buttons. One for current wiki (eg *.wikipedia) and one for *.wikimedia (commons/auth/upload/meta) [22:59:57] Since the latter will often be embedded and not reach the user visually otherwise [23:00:04] (03PS1) 10Vaughn Walters: jjb: [catalyst-daily-Wikibase-cypress] Add Wikibase Cypress job [integration/config] - 10https://gerrit.wikimedia.org/r/1369148 (https://phabricator.wikimedia.org/T444390) [23:00:04] (03CR) 10Vaughn Walters: [C:03+2] "tests passing https://integration.wikimedia.org/ci/view/selenium-daily/job/catalyst-daily-Wikibase-cypress/1/console" [integration/config] - 10https://gerrit.wikimedia.org/r/1369148 (https://phabricator.wikimedia.org/T444390) (owner: 10Vaughn Walters) [23:01:55] (03Merged) 10jenkins-bot: jjb: [catalyst-daily-Wikibase-cypress] Add Wikibase Cypress job [integration/config] - 10https://gerrit.wikimedia.org/r/1369148 (https://phabricator.wikimedia.org/T444390) (owner: 10Vaughn Walters) [23:12:01] (03open) 10dancy: local-dev: Read the fake CAS state directory from `CAS_STATE_DIR` [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1430 [23:13:12] (03update) 10dancy: local-dev: Read the fake CAS state directory from `CAS_STATE_DIR` [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1430 [23:13:44] (03merge) 10dancy: local-dev: Read the fake CAS state directory from `CAS_STATE_DIR` [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1430 [23:14:23] (03open) 10dancy: spiderpig: Train: Update `isUpcoming` when the release time passes [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1431 [23:14:43] (03update) 10dancy: spiderpig: Train: Update `isUpcoming` when the release time passes [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1431 [23:16:00] (03merge) 10dancy: spiderpig: Train: Update `isUpcoming` when the release time passes [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1431 [23:18:07] 10GitLab (CI & Job Runners), 06Release-Engineering-Team, 10a Wikimedia CLI: runner-1036 & runner-1034.gitlab-runners.eqiad1.wikimedia.cloud running out of space - https://phabricator.wikimedia.org/T443833#12552054 (10brennen) Sorry this has slipped through the cracks since the 5th. Just got another report of... [23:25:14] !log runner-1036.gitlab-runners: sudo docker volume prune (T443833) [23:25:17] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:25:17] T443833: runner-1036 & runner-1034.gitlab-runners.eqiad1.wikimedia.cloud running out of space - https://phabricator.wikimedia.org/T443833 [23:26:08] 10GitLab (CI & Job Runners), 06Release-Engineering-Team, 10a Wikimedia CLI: runner-1036 & runner-1034.gitlab-runners.eqiad1.wikimedia.cloud running out of space - https://phabricator.wikimedia.org/T443833#12552080 (10brennen) I pruned volumes on `runner-1036`. It's my understanding that this should be safe,... [23:50:33] (03open) 10dancy: spiderpig: Ignore repeated digests when the event stream reconnects [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1432 [23:50:57] (03update) 10dancy: spiderpig: Ignore repeated digests when the event stream reconnects [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1432 [23:52:30] (03merge) 10dancy: spiderpig: Ignore repeated digests when the event stream reconnects [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1432 [23:52:43] 10GitLab (CI & Job Runners), 06Release-Engineering-Team (Seen): Runner runner-1030.gitlab-runners.eqiad1.wikimedia.cloud out of space - https://phabricator.wikimedia.org/T326558#12552520 (10brennen) 05Openโ†’03Declined This is a lingering task from 3 years ago, closing in favor of the more recent T443833. [23:55:26] (03open) 10dancy: spiderpig: Index the job finish time [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1433 [23:55:27] (03update) 10dancy: spiderpig: Index the job finish time [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1433 [23:55:48] (03update) 10dancy: spiderpig: Index the job finish time [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1433 [23:58:33] (03merge) 10dancy: spiderpig: Index the job finish time [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1433 [23:58:39] (03update) 10catrope: Add a script for checking which security patches can't be backported [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/282