[00:12:42] Yippee, build fixed! [00:12:42] Project mediawiki-core-doxygen-docker build #47398: 09FIXED in 8 min 16 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/47398/ [00:41:05] 10GitLab (Account Approval), 10Release-Engineering-Team: Requesting GitLab account activation for abartov - https://phabricator.wikimedia.org/T349669 (10thcipriani) [00:43:17] 10GitLab (Account Approval), 10Release-Engineering-Team: Requesting GitLab account activation for abartov - https://phabricator.wikimedia.org/T349669 (10thcipriani) 05Open→03Resolved a:03thcipriani It looks like someone beat me to this one (or toolforge magic did). Looks like you should be able to login... [02:25:35] 10Phabricator: Some Phabricator tasks do not work with Google docs "Replace URL with this title" function - https://phabricator.wikimedia.org/T349675 (10Tgr) [04:17:22] (03PS1) 10Zoranzoki21: Zuul: Archive the AddThis extension [integration/config] - 10https://gerrit.wikimedia.org/r/968392 (https://phabricator.wikimedia.org/T335463) [06:11:48] Project mediawiki-core-doxygen-docker build #47405: 04FAILURE in 7 min 21 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/47405/ [07:12:50] Yippee, build fixed! [07:12:50] Project mediawiki-core-doxygen-docker build #47406: 09FIXED in 8 min 24 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/47406/ [07:45:26] 10Diffusion, 10Phabricator (Upstream), 10Upstream: Reduce the number of six default URIs in Diffusion - https://phabricator.wikimedia.org/T244907 (10valerio.bozzolan) Just to understand a bit better, could unnecessary URIs just be hidden by default? or, this is already a thing (I see that in the mentioned ex... [08:28:58] 10GitLab (Project Migration), 10collaboration-services, 10SRE Observability (FY2023/2024-Q2): Migrate SRE repositories to GitLab - operations/alerts - https://phabricator.wikimedia.org/T349626 (10LSobanski) p:05Triage→03High [08:31:13] 10Phabricator: Some Phabricator tasks do not work with Google docs "Replace URL with this title" function - https://phabricator.wikimedia.org/T349675 (10Aklapper) 05Open→03Stalled What is a task ID which triggers them problem? The link cannot be accessed [08:34:21] hi folks! what's the way of getting a gitlab repo mirrored on github? [08:35:27] vgutierrez: none, we do not mirror to GitHub [08:36:12] hashar: is that a temporary thing or the definitive state? [08:36:24] most certainly definitiv [08:36:25] e [08:36:40] ack [08:36:52] the reason we mirrored Gerrit repositories to GitHub was to "increase visibility" and "make it easier to contribute" [08:37:04] which arguably were also some of the reasons advanced to switch to GitLab [08:37:45] so with GitLab more or less providing the same visibility, discoverability and ease to contribute, there is little point in mirroring to GitHub [08:37:52] what is your use case? [08:39:26] hashar: got a new repo going on in gitlab and when that was the case in gerrit one of the final steps was mirroring to GH [08:39:35] but if that's not longer the case, no problem [08:43:58] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: Ssh / user issue with integration-agent-docker-1057.integration.eqiad1.wikimedia.cloud - https://phabricator.wikimedia.org/T349681 (10hashar) [08:44:30] !log integration: disconnected agent https://integration.wikimedia.org/ci/computer/integration%2Dagent%2Ddocker%2D1057/ due to T349681 [08:44:33] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [08:44:33] T349681: Ssh / user issue with integration-agent-docker-1057.integration.eqiad1.wikimedia.cloud - https://phabricator.wikimedia.org/T349681 [08:44:54] vgutierrez: ah yes true we had that extra step :) [08:45:17] vgutierrez: I guess it all boils down to our Gerrit not offering a pleasant experience to browse/search code [08:45:38] hashar: so.. what happens with migrated repos from gerrit to gitlab? we delete them on GH? [08:45:49] that's kinda sad :) [08:45:55] Gerrit code browser is Gitiles at https://gerrit.wikimedia.org/g/ which is humm not ideal [08:46:08] and for searching the workaround is https://codesearch.wmcloud.org/ [08:46:10] yeah... I usually browse our code on GH [08:46:11] anyway I digress [08:46:27] for the repositories migration it is a bit of a mess [08:46:56] since it is more or less self serves, each one does it slightly differently [08:47:14] but eventually the Gerrit repository should be: [08:47:45] 1) emptied up from code with a leftover ARCHIVED.md or README.md indicating the new canonical repository. That is to make it very bold to people pulling from it that is the repo is no more canonical [08:47:56] at least that is how we do it for mediawiki extensions/skins that get archived [08:48:35] 2) Adjust the repository description prefixing it with `[ARCHIVED ` and if there is any Phabricator task tracking the migrating add append it at the end of the description [08:48:40] 3) mark the repository read-only [08:48:46] 4) delete the Github mirror [08:50:35] cheers [08:50:57] I guess the steps should be described in a Phabricator task [08:51:05] s/task/task template/ [08:51:10] or maybe we should write a runbook [08:57:48] is step 4) really necessary? [08:58:07] I would’ve thought it’s enough to let the ARCHIVED.md get mirrored over there and then leave it at that [08:58:26] (though, just to put my cards on the table, my preference would be to continue mirroring, I think) [09:00:30] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: Ssh / user issue with integration-agent-docker-1057.integration.eqiad1.wikimedia.cloud - https://phabricator.wikimedia.org/T349681 (10hashar) From the Puppet log file: ` Oct 24 12:41:01 integration-agent-docker-1057 puppet-agent[1150814]: App... [09:05:55] 10Diffusion, 10Phabricator, 10Release-Engineering-Team (Seen), 10Patch-For-Review: Do not expose dead git-ssh.wikimedia.org service as repo Clone URLs (defined in diffusion.ssh-host setting) - https://phabricator.wikimedia.org/T347408 (10Aklapper) a:03Aklapper As a mitigation for the time being, I quickl... [09:06:42] 10Diffusion, 10Phabricator, 10Release-Engineering-Team (Seen), 10Patch-For-Review: Do not expose dead git-ssh.wikimedia.org service as repo Clone URLs (defined in diffusion.ssh-host setting) - https://phabricator.wikimedia.org/T347408 (10CodeReviewBot) aklapper opened https://gitlab.wikimedia.org/repos/pha... [09:10:41] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: Ssh / user issue with integration-agent-docker-1057.integration.eqiad1.wikimedia.cloud - https://phabricator.wikimedia.org/T349681 (10hashar) ` $ systemctl list-units --failed UNIT LOAD ACTIVE SUB DESCRIPTION... [09:17:24] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: Ssh / user issue with integration-agent-docker-1057.integration.eqiad1.wikimedia.cloud - https://phabricator.wikimedia.org/T349681 (10hashar) I have fixed it using: ` sudo cumin --force 'name:agent-docker-1057' 'systemctl start sssd-nss.socke... [09:27:44] 10Diffusion, 10Phabricator (Upstream), 10Upstream: Reduce the number of six default URIs in Diffusion - https://phabricator.wikimedia.org/T244907 (10Aklapper) [09:27:47] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: Ssh / user issue with integration-agent-docker-1057.integration.eqiad1.wikimedia.cloud - https://phabricator.wikimedia.org/T349681 (10hashar) 05Open→03Resolved a:03hashar From `/var/log/sssd/sssd_nss.log`: ` (2023-10-24 11:52:31): [nss]... [09:29:10] Lucas_WMDE: maybe, then there is probably not much point in keeping a mirror of a now empty repo [09:30:25] !log mediawiki-core-doxygen-docker had intermittent failure due to `integration-agent-docker-1057` refusing ssh connection since Oct 24th 13:00 # T349681 [09:30:29] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [09:30:30] T349681: Ssh / user issue with integration-agent-docker-1057.integration.eqiad1.wikimedia.cloud - https://phabricator.wikimedia.org/T349681 [09:32:40] hashar: it would keep old links to commits working, at least [09:33:03] assuming the ARCHIVED.md is added as a commit on top of the old history instead of forcibly deleting the old commits [09:35:56] and even if people follow old links to master instead of a specific commit, having the ARCHIVED.md file there makes it easier for them to find out where the code went [10:02:13] (03CR) 10Hashar: [C: 03+2] Zuul: Archive the AddThis extension [integration/config] - 10https://gerrit.wikimedia.org/r/968392 (https://phabricator.wikimedia.org/T335463) (owner: 10Zoranzoki21) [10:03:47] (03Merged) 10jenkins-bot: Zuul: Archive the AddThis extension [integration/config] - 10https://gerrit.wikimedia.org/r/968392 (https://phabricator.wikimedia.org/T335463) (owner: 10Zoranzoki21) [10:32:26] 10GitLab (CI & Job Runners), 10Release-Engineering-Team (Priority Backlog 📥), 10collaboration-services: Migrate to using new GitLab CI runner authentication scheme - https://phabricator.wikimedia.org/T344951 (10Jelto) I've done some tests and registered two new runners (two vms in Digital Ocean) to the GitLa... [11:30:24] 10Release-Engineering-Team, 10ChangeProp, 10WMF-JobQueue, 10User-greg: Delete the mediawiki/services/cp-jobqueue repo - https://phabricator.wikimedia.org/T217683 (10hashar) I have deleted the Gerrit repository `mediawiki/services/cp-jobqueue` which had no specifics beside what is in https://gerrit.wikimedi... [11:34:51] !log gerrit: deleted mediawiki/services/cp-jobqueue mediawiki/tools/bundle mediawiki/tools/secpm [11:34:53] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [12:22:40] 10GitLab (CI & Job Runners), 10Release-Engineering-Team (Priority Backlog 📥), 10collaboration-services: Migrate to using new GitLab CI runner authentication scheme - https://phabricator.wikimedia.org/T344951 (10Jelto) [12:25:40] hashar: FYI yuor jsoin patches are live now [12:25:44] *json [12:25:46] for pcc [12:46:22] jbond: AHH GREAT [12:46:29] and apparently nothing has completely exploded! [12:46:41] thank you for all the reviews :] [12:47:47] thanks for the contributions :) [12:48:13] (03PS1) 10Zoranzoki21: Zuul: Archive the TwitterLogin extension [integration/config] - 10https://gerrit.wikimedia.org/r/968655 (https://phabricator.wikimedia.org/T334546) [13:06:49] 10MediaWiki-Releasing, 10MW-1.41-release, 10Release: Release MediaWiki 1.41.0 - https://phabricator.wikimedia.org/T346919 (10Jdforrester-WMF) [13:07:15] 10MediaWiki-Releasing, 10MW-1.41-release, 10Patch-For-Review: Add REL1_41 to ExtensionDistributor as the development snapshot - https://phabricator.wikimedia.org/T346929 (10Jdforrester-WMF) 05Open→03Resolved a:03Jdforrester-WMF [13:14:36] (03CR) 10Hashar: [C: 03+2] Zuul: Archive the TwitterLogin extension [integration/config] - 10https://gerrit.wikimedia.org/r/968655 (https://phabricator.wikimedia.org/T334546) (owner: 10Zoranzoki21) [13:16:12] (03Merged) 10jenkins-bot: Zuul: Archive the TwitterLogin extension [integration/config] - 10https://gerrit.wikimedia.org/r/968655 (https://phabricator.wikimedia.org/T334546) (owner: 10Zoranzoki21) [13:29:09] https://puppet-compiler.wmflabs.org/output/968301/187/build.json and https://puppet-compiler.wmflabs.org/output/968301/187/cp1075.eqiad.wmnet/host.json [13:29:10] lovely [13:31:29] (03PS1) 10Zoranzoki21: Zuul: Archive the WikiArticleFeeds extension [integration/config] - 10https://gerrit.wikimedia.org/r/968670 (https://phabricator.wikimedia.org/T333512) [14:11:51] (03CR) 10Hashar: [C: 03+2] Zuul: Archive the WikiArticleFeeds extension [integration/config] - 10https://gerrit.wikimedia.org/r/968670 (https://phabricator.wikimedia.org/T333512) (owner: 10Zoranzoki21) [14:13:04] (03Merged) 10jenkins-bot: Zuul: Archive the WikiArticleFeeds extension [integration/config] - 10https://gerrit.wikimedia.org/r/968670 (https://phabricator.wikimedia.org/T333512) (owner: 10Zoranzoki21) [14:17:37] (03PS1) 10Pwangai: Zuul: [mediawiki/extensions/Thanks] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/968688 (https://phabricator.wikimedia.org/T321837) [14:19:14] (03PS1) 10Pwangai: Zuul: [mediawiki/extensions/UniversalLanguageSelector] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/968689 (https://phabricator.wikimedia.org/T321837) [14:20:25] (03PS1) 10Pwangai: Zuul: [mediawiki/extensions/SecurePoll] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/968690 (https://phabricator.wikimedia.org/T321837) [14:21:33] Peter_W: I am deploying them :) [14:21:52] (03CR) 10Hashar: [C: 03+2] Zuul: [mediawiki/extensions/SecurePoll] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/968690 (https://phabricator.wikimedia.org/T321837) (owner: 10Pwangai) [14:22:01] (03CR) 10Hashar: [C: 03+2] Zuul: [mediawiki/extensions/Thanks] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/968688 (https://phabricator.wikimedia.org/T321837) (owner: 10Pwangai) [14:22:03] (03PS1) 10Pwangai: Zuul: [mediawiki/skins/WikimediaApiPortal] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/968691 (https://phabricator.wikimedia.org/T321837) [14:22:06] (03CR) 10Hashar: [C: 03+2] Zuul: [mediawiki/extensions/UniversalLanguageSelector] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/968689 (https://phabricator.wikimedia.org/T321837) (owner: 10Pwangai) [14:22:21] (03CR) 10Hashar: [C: 03+2] Zuul: [mediawiki/skins/WikimediaApiPortal] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/968691 (https://phabricator.wikimedia.org/T321837) (owner: 10Pwangai) [14:23:22] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/SecurePoll] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/968690 (https://phabricator.wikimedia.org/T321837) (owner: 10Pwangai) [14:23:57] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/Thanks] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/968688 (https://phabricator.wikimedia.org/T321837) (owner: 10Pwangai) [14:23:59] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/UniversalLanguageSelector] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/968689 (https://phabricator.wikimedia.org/T321837) (owner: 10Pwangai) [14:24:15] (03Merged) 10jenkins-bot: Zuul: [mediawiki/skins/WikimediaApiPortal] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/968691 (https://phabricator.wikimedia.org/T321837) (owner: 10Pwangai) [14:24:41] (03PS1) 10Zoranzoki21: Zuul: Archive the UserThrottle extension [integration/config] - 10https://gerrit.wikimedia.org/r/968692 (https://phabricator.wikimedia.org/T334547) [14:25:22] !log Reloaded Zuul to enable Sonar Codehealth on SecurePoll, Thanks, UniversalLanguageSelector, WikimediaApiPortal # T321837 [14:25:25] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [14:25:25] T321837: Repositories integrated into Codehealth Pipeline (Production) - https://phabricator.wikimedia.org/T321837 [14:25:44] (03CR) 10Hashar: [C: 03+2] Zuul: Archive the UserThrottle extension [integration/config] - 10https://gerrit.wikimedia.org/r/968692 (https://phabricator.wikimedia.org/T334547) (owner: 10Zoranzoki21) [14:26:00] (03PS1) 10Zoranzoki21: Archive the Duplicator extension [integration/config] - 10https://gerrit.wikimedia.org/r/968693 (https://phabricator.wikimedia.org/T334319) [14:26:29] (03PS2) 10Zoranzoki21: Zuul: Archive the Duplicator extension [integration/config] - 10https://gerrit.wikimedia.org/r/968693 (https://phabricator.wikimedia.org/T334319) [14:28:05] (03Merged) 10jenkins-bot: Zuul: Archive the UserThrottle extension [integration/config] - 10https://gerrit.wikimedia.org/r/968692 (https://phabricator.wikimedia.org/T334547) (owner: 10Zoranzoki21) [14:28:09] (03PS1) 10Zoranzoki21: Zuul: Archive the GraphViz extension [integration/config] - 10https://gerrit.wikimedia.org/r/968695 (https://phabricator.wikimedia.org/T335254) [14:29:08] (03PS1) 10Zoranzoki21: Zuul: Archive the OpenID extension [integration/config] - 10https://gerrit.wikimedia.org/r/968697 (https://phabricator.wikimedia.org/T322820) [14:31:45] (03CR) 10Hashar: [C: 03+2] Zuul: Archive the Duplicator extension [integration/config] - 10https://gerrit.wikimedia.org/r/968693 (https://phabricator.wikimedia.org/T334319) (owner: 10Zoranzoki21) [14:31:49] (03CR) 10Hashar: [C: 03+2] Zuul: Archive the GraphViz extension [integration/config] - 10https://gerrit.wikimedia.org/r/968695 (https://phabricator.wikimedia.org/T335254) (owner: 10Zoranzoki21) [14:31:54] (03CR) 10Hashar: [C: 03+2] Zuul: Archive the OpenID extension [integration/config] - 10https://gerrit.wikimedia.org/r/968697 (https://phabricator.wikimedia.org/T322820) (owner: 10Zoranzoki21) [14:33:15] (03Merged) 10jenkins-bot: Zuul: Archive the Duplicator extension [integration/config] - 10https://gerrit.wikimedia.org/r/968693 (https://phabricator.wikimedia.org/T334319) (owner: 10Zoranzoki21) [14:33:19] (03Merged) 10jenkins-bot: Zuul: Archive the GraphViz extension [integration/config] - 10https://gerrit.wikimedia.org/r/968695 (https://phabricator.wikimedia.org/T335254) (owner: 10Zoranzoki21) [14:33:23] (03Merged) 10jenkins-bot: Zuul: Archive the OpenID extension [integration/config] - 10https://gerrit.wikimedia.org/r/968697 (https://phabricator.wikimedia.org/T322820) (owner: 10Zoranzoki21) [15:03:13] 10GitLab (CI & Job Runners), 10Release-Engineering-Team (Priority Backlog 📥), 10collaboration-services: Migrate to using new GitLab CI runner authentication scheme - https://phabricator.wikimedia.org/T344951 (10Jelto) Currently the python GitLab api client does not support the new runner creation API in vers... [15:44:18] 10Phabricator: The content of the "Activity" column on Phabricator's homepage is incorrectly set to display none. - https://phabricator.wikimedia.org/T346169 (10SunAfterRain) [15:56:00] !log doc: removed Graphviz MediaWiki extension code coverage following its archival: `sudo -u doc-uploader rm -fR /srv/doc/cover-extensions/GraphViz/` # T335254 [15:56:03] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:56:04] T335254: Archive the GraphViz extension - https://phabricator.wikimedia.org/T335254 [15:57:26] 10GitLab (Administration, Settings & Policy), 10Release-Engineering-Team: GitLab Kokkuri repos need: Protected branch: main; protected tags: * - https://phabricator.wikimedia.org/T349732 (10thcipriani) [16:18:03] hashar: FYI the check experimental just gave a -1 and im not sure why https://gerrit.wikimedia.org/r/c/operations/puppet/+/968716/ [16:18:10] as both reports where success [16:18:43] ah [16:19:06] OH YEAH [16:19:07] I know [16:19:18] it is a gerrit oddity [16:19:32] the pcc reported on patchset 6 which had a Verified +1 [16:19:49] err ps 6 has verified minus 1 [16:20:03] then patchset 7 got a verified plus 1 [16:20:13] and the pcc reports after that without voting verified [16:20:28] but since it reports on PS 6 which has a verified minus 1, that vote is carried over and override the previously set verified [16:20:37] essentially it can be ignored [16:20:42] ack thanks [16:20:53] the fix is https://gerrit.wikimedia.org/r/c/All-Projects/+/919831 [16:21:18] ahh thanks [16:21:36] but I have to test it out first. I have made that change on a dummy repo https://gerrit.wikimedia.org/r/c/test/gerrit-ping/+/931873 [16:21:49] so it is "all about" reproducing against that test/gerrit-ping to confirm the config change works [16:21:54] and if it does merge the All-Projects change :) [16:23:03] * hashar adds a self note to do that next week [16:26:21] jbond: also note the change itself still has Verified plus one since the latest patchset (7) has verified +1 [16:26:26] but the comment is confusing for sure [16:31:31] ack thanks [17:04:30] 10GitLab (Project Migration), 10SRE, 10Traffic, 10Patch-For-Review: Migrate Traffic repositories from Gerrit to Gitlab - https://phabricator.wikimedia.org/T347623 (10CodeReviewBot) brett merged https://gitlab.wikimedia.org/repos/sre/acme-chief/-/merge_requests/4 Implement Gitlab CI and Blubber config [17:40:21] 10Release-Engineering-Team (Social Piranhas 🐟), 10Quibble: Quibble support for repositories cloned with canonical name (no clonemapper) - https://phabricator.wikimedia.org/T349741 (10hashar) [17:45:05] 10Release-Engineering-Team (Social Piranhas 🐟), 10Quibble: Quibble support for repositories cloned with canonical name (no clonemapper) - https://phabricator.wikimedia.org/T349741 (10hashar) [19:33:33] 10Diffusion, 10Phabricator (Upstream), 10Upstream: Reduce the number of six default URIs in Diffusion - https://phabricator.wikimedia.org/T244907 (10valerio.bozzolan) (Please make F40005688 visible - and sorry if Phabricator is so messed up with these things ihih - https://we.phorge.it/T15106) [20:30:19] Project beta-update-databases-eqiad build #71222: 04FAILURE in 10 min: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/71222/ [20:52:17] 10Release-Engineering-Team (Social Piranhas 🐟), 10Patch-For-Review, 10Release, 10Train Deployments: 1.42.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T348355 (10Umherirrender) [20:52:39] 10Release-Engineering-Team (Social Piranhas 🐟), 10Patch-For-Review, 10Release, 10Train Deployments: 1.42.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T348355 (10Umherirrender) [21:30:28] Yippee, build fixed! [21:30:28] Project beta-update-databases-eqiad build #71223: 09FIXED in 10 min: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/71223/ [22:28:59] 10Release-Engineering-Team (Social Piranhas 🐟), 10Patch-For-Review, 10Release, 10Train Deployments: 1.42.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T348355 (10Jdforrester-WMF) [22:29:06] 10Release-Engineering-Team (Social Piranhas 🐟), 10Patch-For-Review, 10Release, 10Train Deployments: 1.42.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T348355 (10Jdforrester-WMF)