[07:34:18] 10Release-Engineering-Team (Radar), 06collaboration-services: upgrade contint servers to bullseye - https://phabricator.wikimedia.org/T334517#9819898 (10hashar) 05Resolved→03Open Congratulations on having reimaged contint2002! It is missing steps though: [ ] attached as a Jenkins agent to the Jenkins con... [07:35:42] !log Jenkins: added back contint2002 as an Agent. The host got reimaged from Buster to Bullseye # T334517 [07:35:44] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [07:35:44] T334517: upgrade contint servers to bullseye - https://phabricator.wikimedia.org/T334517 [07:39:41] 10Gerrit (Gerrit 3.8), 07Mobile: Missing sign in links on mobile - https://phabricator.wikimedia.org/T344891#9819924 (10hashar) [08:04:23] (03approved) 10jelto: Operate using project paths, not project ids [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/80 (owner: 10dancy) [08:22:40] 10GitLab (Infrastructure), 10Toolforge: Whitelist Toolforge hosts in Gitlab shared runners - https://phabricator.wikimedia.org/T365561 (10Sportzpikachu) 03NEW [08:22:56] 10GitLab (Infrastructure), 10Toolforge: Whitelist Toolforge hosts in Gitlab shared runners - https://phabricator.wikimedia.org/T365561#9820083 (10Sportzpikachu) [08:27:00] 10GitLab (Infrastructure): Allow access to Toolforge web services in Gitlab shared runners - https://phabricator.wikimedia.org/T365561#9820087 (10taavi) [08:31:33] (03PS1) 10Hoo man: Allow running in folders with spaces etc. [fresh] - 10https://gerrit.wikimedia.org/r/1034847 [10:39:31] 10Phabricator, 10wikimedia.biterg.io: Closed tickets in Bugzilla migrated without a closing date - https://phabricator.wikimedia.org/T107254#9820453 (10Aklapper) p:05Lowest→03Low a:03Aklapper [10:59:37] 10Phabricator, 10Release-Engineering-Team (Yakisfaction), 10wikimedia.biterg.io: Closed tickets in Bugzilla migrated without a closing date - https://phabricator.wikimedia.org/T107254#9820484 (10Aklapper) 05Open→03Resolved Done. Thanks everyone! [11:24:48] 10Phabricator: Custom task form for #CommunityTech - https://phabricator.wikimedia.org/T365578 (10KSiebert) 03NEW [11:58:57] 10Phabricator: Custom task form for #CommunityTech - https://phabricator.wikimedia.org/T365578#9820727 (10Aklapper) Hi, which presets in which fields are wanted? [12:07:34] (03CR) 10Kosta Harlan: [WIP] Add parallel execution for PHPUnit extensions suite (0311 comments) [integration/quibble] - 10https://gerrit.wikimedia.org/r/1031903 (https://phabricator.wikimedia.org/T361190) (owner: 10Arthur taylor) [12:08:15] 10Phabricator: Custom task form for #CommunityTech - https://phabricator.wikimedia.org/T365578#9820764 (10KSiebert) I do not see any documentation about presets in https://www.mediawiki.org/wiki/Phabricator/Help/Forms#Pre-filling_forms. We want something similar like the web team did here: https://phabricator.w... [12:37:17] 06Release-Engineering-Team, 10MediaWiki-REST-API, 06MW-Interfaces-Team, 10MW-1.43-notes (1.43.0-wmf.5; 2024-05-14): Per-page graphite metrics created for MediaWiki.rest_api_latency / rest_api_errors - https://phabricator.wikimedia.org/T365111#9820828 (10daniel) The fix has been backported and deployed. It... [12:44:08] (03merge) 10jelto: create a group of instance wide WMCS runners [repos/releng/gitlab-settings] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-settings/-/merge_requests/61 (https://phabricator.wikimedia.org/T327949 https://phabricator.wikimedia.org/T362401) [12:45:35] 06Release-Engineering-Team, 10MediaWiki-REST-API, 06MW-Interfaces-Team, 10MW-1.43-notes (1.43.0-wmf.5; 2024-05-14): Per-page graphite metrics created for MediaWiki.rest_api_latency / rest_api_errors - https://phabricator.wikimedia.org/T365111#9820855 (10fgiunchedi) 05In progress→03Resolved a:03fgi... [13:05:58] 06Release-Engineering-Team, 10MediaWiki-REST-API, 06MW-Interfaces-Team, 10MW-1.43-notes (1.43.0-wmf.5; 2024-05-14): Per-page graphite metrics created for MediaWiki.rest_api_latency / rest_api_errors - https://phabricator.wikimedia.org/T365111#9820947 (10WMDE-leszek) hello, a silly question from an unin... [13:17:46] 06Release-Engineering-Team, 10MediaWiki-REST-API, 06MW-Interfaces-Team, 10MW-1.43-notes (1.43.0-wmf.5; 2024-05-14): Per-page graphite metrics created for MediaWiki.rest_api_latency / rest_api_errors - https://phabricator.wikimedia.org/T365111#9821004 (10fgiunchedi) >>! In T365111#9820947, @WMDE-leszek... [13:24:33] (03PS1) 10Hashar: Enable pipeline jobs for operations/software/bitu [integration/config] - 10https://gerrit.wikimedia.org/r/1034915 (https://phabricator.wikimedia.org/T362318) [13:25:28] (03CR) 10Slyngshede: [C:03+1] "LGTM" [integration/config] - 10https://gerrit.wikimedia.org/r/1034915 (https://phabricator.wikimedia.org/T362318) (owner: 10Hashar) [13:28:23] (03CR) 10Hashar: [C:03+2] Enable pipeline jobs for operations/software/bitu [integration/config] - 10https://gerrit.wikimedia.org/r/1034915 (https://phabricator.wikimedia.org/T362318) (owner: 10Hashar) [13:29:49] (03Merged) 10jenkins-bot: Enable pipeline jobs for operations/software/bitu [integration/config] - 10https://gerrit.wikimedia.org/r/1034915 (https://phabricator.wikimedia.org/T362318) (owner: 10Hashar) [14:12:36] releng, hashar: FYI, We're going to switch gerrit over to use the new postfix MTAs in a few minutes. Should be a no-op, jhathaway and I will be watching and testing after it rolls out. [14:18:02] (03open) 10gehel: build: Generate Javadoc by generating delombok sources. [repos/ci-tools/wmf-jvm-parent-pom] - 10https://gitlab.wikimedia.org/repos/ci-tools/wmf-jvm-parent-pom/-/merge_requests/12 (https://phabricator.wikimedia.org/T365600) [14:22:10] eoghan: ah great! My best wishes for the migration. Gerrit has some metrics regarding how it sends emails which might be helpful: https://grafana.wikimedia.org/d/tllYBrhGz/queues?orgId=1&var-instance=gerrit.wikimedia.org:443&var-queue=send_email [14:22:19] 06Release-Engineering-Team, 10MediaWiki-REST-API, 06MW-Interfaces-Team, 10MW-1.43-notes (1.43.0-wmf.5; 2024-05-14): Per-page graphite metrics created for MediaWiki.rest_api_latency / rest_api_errors - https://phabricator.wikimedia.org/T365111#9821538 (10WMDE-leszek) understood, thanks @fgiunchedi [14:22:22] specially the last one which is about completed emails [14:22:31] Ah nice, that's useful. Tahnks [14:22:43] and the n-2 graph "total scheduled tasks" would be the queue of emails to be send [14:23:14] eoghan: in which IRC channel do you sync up ? I can join and lurk :) [14:26:19] ah #wikimedia-sre-collab of course [14:32:51] (03merge) 10gehel: build: Generate Javadoc by generating delombok sources. [repos/ci-tools/wmf-jvm-parent-pom] - 10https://gitlab.wikimedia.org/repos/ci-tools/wmf-jvm-parent-pom/-/merge_requests/12 (https://phabricator.wikimedia.org/T365600) [14:49:59] (03merge) 10dancy: Operate using project paths, not project ids [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/80 [14:52:29] 10Release-Engineering-Team (Radar), 06collaboration-services: upgrade contint servers to bullseye - https://phabricator.wikimedia.org/T334517#9821744 (10Dzahn) >>! In T334517#9819898, @hashar wrote: > And we can also get rid of the Ganeti VM `contint1003.eqiad.wmnet` I wanted to ask you about this anyways. W... [14:55:30] (03merge) 10dancy: GitlabMentionsSink: Unsubscribe from MR after posting note [repos/releng/gitlab-webhooks] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-webhooks/-/merge_requests/35 (https://phabricator.wikimedia.org/T289712) [15:08:38] 10Release-Engineering-Team (Yakisfaction), 05Release, 05Train Deployments: 1.43.0-wmf.6 deployment blockers - https://phabricator.wikimedia.org/T361400#9821924 (10matmarex) [15:12:20] 10GitLab (Integrations), 10Release-Engineering-Team (Priority Backlog 📥), 07Code-Health, 07Developer Productivity, 13Patch-For-Review: Investigate whether we can/should integrate Git/Reviewers with GitLab - https://phabricator.wikimedia.org/T289712#9821944 (10dancy) 05Open→03Resolved [15:13:08] 10GitLab (Infrastructure): Allow access to Toolforge web services in Gitlab shared runners - https://phabricator.wikimedia.org/T365561#9821964 (10bd808) [15:16:21] 10Continuous-Integration-Infrastructure, 06collaboration-services, 06SRE, 10vm-requests, 13Patch-For-Review: Ganeti VM for contint migration - https://phabricator.wikimedia.org/T358237#9821997 (10ops-monitoring-bot) cookbooks.sre.hosts.decommission executed by dzahn@cumin1002 for hosts: `contint1003.... [15:16:23] 10GitLab (Infrastructure): Allow access to Toolforge web services from Gitlab shared runners - https://phabricator.wikimedia.org/T365561#9821983 (10bd808) [15:24:23] (03open) 10dancy: version: 0.23.0 [repos/releng/blubber] - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/87 [15:28:40] (03merge) 10dancy: version: 0.23.0 [repos/releng/blubber] - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/87 [15:31:43] 10GitLab, 14Release-Engineering-Team (GitLab IV: Mise En Place 🍱), 06collaboration-services: Convert runner-1030.gitlab-runners.eqiad1.wikimedia.cloud to an instance-wide shared runner - https://phabricator.wikimedia.org/T327949#9822057 (10Jelto) 05Open→03Resolved I was unable to add the private regi... [15:33:02] (03open) 10dancy: CHANGELOG.md: Fixed notes for 0.23.0 [repos/releng/blubber] - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/88 [15:33:04] (03update) 10dancy: CHANGELOG.md: Fixed notes for 0.23.0 [repos/releng/blubber] - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/88 [15:35:58] 10Release-Engineering-Team (Yakisfaction), 13Patch-For-Review: Blubber Python builder: Always use a virtualenv - https://phabricator.wikimedia.org/T357548#9822080 (10dancy) docker-registry.discovery.wmnet/repos/releng/blubber/buildkit:v0.23.0 has been published with the changes. [15:37:12] (03merge) 10dancy: CHANGELOG.md: Fixed notes for 0.23.0 [repos/releng/blubber] - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/88 [15:37:54] 06Release-Engineering-Team, 10Release Pipeline (Blubber): Blubber: Add workaround for Bookworm's Python pip restrictions - https://phabricator.wikimedia.org/T346090#9822093 (10dancy) [15:46:11] 10GitLab (Infrastructure), 13Patch-For-Review: Allow access to Toolforge web services from Gitlab shared runners - https://phabricator.wikimedia.org/T365561#9822143 (10bd808) p:05Triage→03Medium [16:01:54] (03CR) 10Hashar: [C:03+2] Add wbm1058 to CI allow list [integration/config] - 10https://gerrit.wikimedia.org/r/1034602 (owner: 10Bartosz Dziewoński) [16:03:27] (03Merged) 10jenkins-bot: Add wbm1058 to CI allow list [integration/config] - 10https://gerrit.wikimedia.org/r/1034602 (owner: 10Bartosz Dziewoński) [16:47:24] 10Release-Engineering-Team (Radar), 06collaboration-services, 13Patch-For-Review: upgrade contint servers to bullseye - https://phabricator.wikimedia.org/T334517#9822498 (10Dzahn) @hashar contint1003 is removed. What's next? Do we need a Hiera change to unmask any services? [16:49:27] 10GitLab, 10Release-Engineering-Team (Priority Backlog 📥), 10observability: Explore monitoring for the GitLab runners k8s cluster - https://phabricator.wikimedia.org/T363919#9822554 (10thcipriani) [16:56:18] Project beta-scap-sync-world build #156013: 04FAILURE in 59 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/156013/ [17:01:21] Does blubber support copying hidden dirs? It doesn't seem to be working with https://gitlab.wikimedia.org/repos/data-engineering/airflow/-/blob/main/blubber.yaml?ref_type=heads#L125 [17:06:29] Yippee, build fixed! [17:06:29] Project beta-scap-sync-world build #156014: 09FIXED in 1 min 10 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/156014/ [17:07:01] nm, I should have been using 'copies' [17:26:39] 10Release-Engineering-Team (Yakisfaction), 13Patch-For-Review: Blubber Python builder: Always use a virtualenv - https://phabricator.wikimedia.org/T357548#9822776 (10dancy) 05In progress→03Resolved [17:27:09] 10Release-Engineering-Team (Yakisfaction), 13Patch-For-Review: Blubber Python builder: Always use a virtualenv - https://phabricator.wikimedia.org/T357548#9822777 (10dancy) [17:31:27] 10Phabricator: Maniphest burnup report incorrectly counts "All Time" opened and closed tasks - https://phabricator.wikimedia.org/T119376#9822795 (10matmarex) This doesn't seem to have been fixed by the subtask. :( [17:36:27] (03open) 10dduvall: Update github.com/docker/docker dependency [repos/releng/blubber] - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/89 [17:36:28] (03update) 10dduvall: Update github.com/docker/docker dependency [repos/releng/blubber] - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/89 [17:36:40] (03update) 10dduvall: Update github.com/docker/docker dependency [repos/releng/blubber] - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/89 [18:00:34] (03update) 10dduvall: Update github.com/docker/docker dependency [repos/releng/blubber] - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/89 [18:06:43] (03open) 10dduvall: go: Upgrade to Go 1.21 [repos/releng/blubber] - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/90 [18:06:44] (03update) 10dduvall: go: Upgrade to Go 1.21 [repos/releng/blubber] - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/90 [18:07:05] (03update) 10dduvall: go: Upgrade to Go 1.21 [repos/releng/blubber] - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/90 [18:09:54] (03update) 10dduvall: go: Upgrade to Go 1.21 [repos/releng/blubber] - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/90 [18:09:55] (03update) 10dduvall: Update github.com/docker/docker dependency [repos/releng/blubber] (main-Iac559c421ea8fd54f2d50dfc3a34bd13ea21604a) - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/89 [18:09:59] (03update) 10dduvall: Update github.com/docker/docker dependency [repos/releng/blubber] (main-Iac559c421ea8fd54f2d50dfc3a34bd13ea21604a) - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/89 [18:21:15] (03approved) 10dancy: go: Upgrade to Go 1.21 [repos/releng/blubber] - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/90 (owner: 10dduvall) [19:48:32] 06Release-Engineering-Team, 10Temporary accounts, 06Trust and Safety Product Team, 10MW-1.43-notes (1.43.0-wmf.5; 2024-05-14), 13Patch-For-Review: [Epic] Make PHPUnit, Selenium, and API testing tests pass with temp account feature flag enabled - https://phabricator.wikimedia.org/T355879#9823113 (10kostajh) [19:48:57] 10Release-Engineering-Team (Radar), 10Temporary accounts, 13Patch-For-Review: Enable temp account creation in DevelopmentSettings.php - https://phabricator.wikimedia.org/T359043#9823116 (10kostajh) [19:48:58] 06Release-Engineering-Team, 10Temporary accounts, 06Trust and Safety Product Team, 10MW-1.43-notes (1.43.0-wmf.5; 2024-05-14), 13Patch-For-Review: [Epic] Make PHPUnit, Selenium, and API testing tests pass with temp account feature flag enabled - https://phabricator.wikimedia.org/T355879#9823115 (10kostajh) [19:51:47] 10Release-Engineering-Team (Radar), 10Temporary accounts, 13Patch-For-Review: Enable temp account creation in DevelopmentSettings.php - https://phabricator.wikimedia.org/T359043#9823131 (10kostajh) TSP team discussed this internally. We are proposing to do this in stages: 1. Add to DevelopmentSettings, but... [19:57:11] 10Phabricator, 10Release-Engineering-Team (Yakisfaction), 10wikimedia.biterg.io: Closed tickets in Bugzilla migrated without a closing date - https://phabricator.wikimedia.org/T107254#9823141 (10Aklapper) FYI, due to importing RT tickets into Phabricator on 2014-12-17 and due to some tickets originally c... [20:17:12] (03PS1) 10Kosta Harlan: local_settings: Load DevelopmentSettings.php after setting MW_QUIBBLE_CI [integration/quibble] - 10https://gerrit.wikimedia.org/r/1035035 (https://phabricator.wikimedia.org/T359043) [20:17:14] (03PS1) 10Kosta Harlan: Release: Quibble 1.8.0 [integration/quibble] - 10https://gerrit.wikimedia.org/r/1035036 [20:18:54] hashar: would appreciate if you can have a look at https://gerrit.wikimedia.org/r/c/integration/quibble/+/1035035. Sorry, I know it will be a bit of a pain to do the whole release process just for this change. [20:22:39] 06Release-Engineering-Team, 10Temporary accounts, 06Trust and Safety Product Team, 10MW-1.43-notes (1.43.0-wmf.5; 2024-05-14), and 2 others: [Epic] Make PHPUnit, Selenium, and API testing tests pass with temp account feature flag enabled - https://phabricator.wikimedia.org/T355879#9823289 (10kostajh) [20:59:57] 10Phabricator: Maniphest burnup report incorrectly counts "All Time" opened and closed tasks - https://phabricator.wikimedia.org/T119376#9823383 (10Aklapper) 05Stalled→03Open Hmm, right. https://phabricator.wikimedia.org/source/phabricator/browse/wmf%252Fstable/src/applications/maniphest/controller/Maniphest... [21:08:52] 10Phabricator: Maniphest burnup report incorrectly counts "All Time" opened and closed tasks (due to lack of transactions from Bugzilla/RT times) - https://phabricator.wikimedia.org/T119376#9823405 (10Aklapper) [22:04:15] 10Release-Engineering-Team (Yakisfaction), 05Release, 05Train Deployments: 1.43.0-wmf.6 deployment blockers - https://phabricator.wikimedia.org/T361400#9823600 (10Jdlrobson) [22:31:18] (03update) 10sandeeps: scap deploy: broken symlink warning log integration [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/322 (https://phabricator.wikimedia.org/T342162) [22:36:56] (03update) 10sandeeps: scap deploy: broken symlink warning log integration [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/322 (https://phabricator.wikimedia.org/T342162) [22:54:58] (03PS1) 10Jforrester: Zuul: Archive mediawiki/services/service-scaffold-{node,golang} [integration/config] - 10https://gerrit.wikimedia.org/r/1035057 (https://phabricator.wikimedia.org/T365512) [22:54:59] (03PS1) 10Jforrester: jjb: Drop {trigger-,}service-scaffold-node-test, unused [integration/config] - 10https://gerrit.wikimedia.org/r/1035058 (https://phabricator.wikimedia.org/T365512) [23:00:15] (03CR) 10Jforrester: [C:03+2] Zuul: Archive mediawiki/services/service-scaffold-{node,golang} [integration/config] - 10https://gerrit.wikimedia.org/r/1035057 (https://phabricator.wikimedia.org/T365512) (owner: 10Jforrester) [23:00:18] (03CR) 10Jforrester: [C:03+2] jjb: Drop {trigger-,}service-scaffold-node-test, unused [integration/config] - 10https://gerrit.wikimedia.org/r/1035058 (https://phabricator.wikimedia.org/T365512) (owner: 10Jforrester) [23:01:41] (03Merged) 10jenkins-bot: Zuul: Archive mediawiki/services/service-scaffold-{node,golang} [integration/config] - 10https://gerrit.wikimedia.org/r/1035057 (https://phabricator.wikimedia.org/T365512) (owner: 10Jforrester) [23:01:42] (03PS2) 10Jforrester: jjb: Drop {trigger-,}service-scaffold-node-pipeline-test, now unused [integration/config] - 10https://gerrit.wikimedia.org/r/1035058 (https://phabricator.wikimedia.org/T365512) [23:01:45] (03CR) 10Jforrester: "…" [integration/config] - 10https://gerrit.wikimedia.org/r/1035058 (https://phabricator.wikimedia.org/T365512) (owner: 10Jforrester) [23:02:36] !log Zuul: Archive mediawiki/services/service-scaffold-{node,golang} for T365512 [23:02:37] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:02:38] T365512: Archive the service-scaffold-node and service-scaffold-golang libraries - https://phabricator.wikimedia.org/T365512 [23:03:16] (03Merged) 10jenkins-bot: jjb: Drop {trigger-,}service-scaffold-node-pipeline-test, now unused [integration/config] - 10https://gerrit.wikimedia.org/r/1035058 (https://phabricator.wikimedia.org/T365512) (owner: 10Jforrester)