[01:00:02] 10Gerrit: Gerrit diffs almost completely lost the ability to highlight differences inline - https://phabricator.wikimedia.org/T338672 (10matmarex) That's true, but it's not really what I'm complaining about. I don't want it to ignore the whitespace (except when using that option, I indeed toggle it on and off oc... [07:01:30] !log contint: on both zuul merger: `sudo -u zuul rm -fR /srv/zuul/git/operations/homer/*` [07:01:31] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [07:03:21] !log contint: on both zuul merger: `sudo -u zuul rm -fR /srv/zuul/git/operations/homer [07:03:22] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [07:03:25] I need my coffee [08:38:33] 10GitLab (Auth & Access), 10Release-Engineering-Team (They Live 🕶️🧟), 10CAS-SSO, 10Infrastructure-Foundations, and 4 others: migrate gitlab away from the CAS protocol - https://phabricator.wikimedia.org/T320390 (10jbond) > After fixing the redirect_uri I'm able to login successfully to the admin interface... [08:48:07] 10Release-Engineering-Team (They Live 🕶️🧟), 10Patch-For-Review, 10Release, 10Train Deployments: 1.41.0-wmf.13 deployment blockers - https://phabricator.wikimedia.org/T337527 (10Winston_Sung) > **[ Risky patch ]** for backporting to wmf.13: since we've never removed a LC before > --https://gerrit.wikimedia... [08:49:21] (03CR) 10Hashar: [C: 03+2] Zuul: [mediawiki/extensions/Math] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/929353 (https://phabricator.wikimedia.org/T321837) (owner: 10Pwangai) [08:49:23] (03CR) 10Hashar: [C: 03+2] Zuul: [mediawiki/extensions/UploadWizard] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/929355 (https://phabricator.wikimedia.org/T321837) (owner: 10Pwangai) [08:49:25] (03CR) 10Hashar: [C: 03+2] Zuul: [mediawiki/extensions/MultimediaViewer] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/929358 (https://phabricator.wikimedia.org/T321837) (owner: 10Pwangai) [08:49:27] (03CR) 10Hashar: [C: 03+2] Zuul: [mediawiki/extensions/UploadsLink] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/929360 (https://phabricator.wikimedia.org/T321837) (owner: 10Pwangai) [08:50:48] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/Math] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/929353 (https://phabricator.wikimedia.org/T321837) (owner: 10Pwangai) [08:51:10] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/UploadWizard] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/929355 (https://phabricator.wikimedia.org/T321837) (owner: 10Pwangai) [08:51:25] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/MultimediaViewer] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/929358 (https://phabricator.wikimedia.org/T321837) (owner: 10Pwangai) [08:51:27] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/UploadsLink] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/929360 (https://phabricator.wikimedia.org/T321837) (owner: 10Pwangai) [08:52:26] !log Reloaded Zuul to enable Sonar Codehealth on Math, MultimediaViewer, UploadsLink and UploadWizard # T321837 [08:52:28] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [08:52:28] T321837: Repositories integrated into Codehealth Pipeline (Production) - https://phabricator.wikimedia.org/T321837 [09:03:59] 10Release-Engineering-Team (They Live 🕶️🧟), 10Patch-For-Review, 10Release, 10Train Deployments: 1.41.0-wmf.13 deployment blockers - https://phabricator.wikimedia.org/T337527 (10Func) >>! In T337527#8926625, @Winston_Sung wrote: >> **[ Risky patch ]** for backporting to wmf.13: since we've never removed a... [09:26:00] 10Beta-Cluster-Infrastructure, 10SRE, 10Traffic, 10HTTPS: upload.wikimedia.beta.wmflabs.org certificate expired (May 2023) - https://phabricator.wikimedia.org/T337642 (10AlexisJazz) 05Open→03Resolved a:03AlexisJazz Dunno how but it works again. [09:26:06] 10Beta-Cluster-Infrastructure, 10Quality-and-Test-Engineering-Team (QTE), 10SRE, 10Traffic, and 3 others: [epic] The SSL certificate for Beta cluster domains fails to properly renew & deploy - https://phabricator.wikimedia.org/T293585 (10AlexisJazz) [09:26:41] 10Beta-Cluster-Infrastructure, 10Quality-and-Test-Engineering-Team (QTE), 10SRE, 10Traffic, and 3 others: [epic] The SSL certificate for Beta cluster domains fails to properly renew & deploy - https://phabricator.wikimedia.org/T293585 (10AlexisJazz) 05Open→03Resolved [09:43:29] 10Phabricator, 10Release-Engineering-Team (They Live 🕶️🧟), 10serviceops-collab, 10Developer-Advocacy (Apr-Jun 2023): Automate SQL queries for annual end-of-year Phabricator statistics/metrics on wikitech-l@ - https://phabricator.wikimedia.org/T337388 (10Aklapper) [09:57:00] 10Phabricator, 10serviceops-collab, 10Developer-Advocacy (Apr-Jun 2023): Automate SQL queries for quarterly Phabricator statistics/metrics for Technical Community Newsletter - https://phabricator.wikimedia.org/T337387 (10Aklapper) 05Open→03Resolved @Dzahn: Thanks a lot for fiddling with the systemd/cron... [11:33:17] hello! gerrit appears to be a little unwell, getting "This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset" on most changes [11:33:38] :( [11:33:40] hashar: ^^ [11:34:08] Can confirm, at least on the puppet repo [11:34:20] I'd just seen it on a MW extension too [11:34:59] and mw config [11:35:17] so probably global then [11:35:43] Yeah... I bet it'll be pretty obvious from its logs [11:50:15] 10GitLab (Infrastructure), 10serviceops-collab: Locking for gitlab backups - https://phabricator.wikimedia.org/T338332 (10eoghan) 05Open→03In progress p:05Triage→03Medium a:03eoghan [12:14:44] hnowlan: Reedy: claime: yeah that is an infamous bug in Zuul (T309376). I restarted it so changes can be `recheck` and should no more show the issue [12:14:45] T309376: Zuul jenkins-bot user holding open SSH sessions - https://phabricator.wikimedia.org/T309376 [12:42:52] hashar: thanks! [13:13:30] thanks hashar <3 [13:15:24] 10GitLab (Project Migration), 10Release-Engineering-Team (They Live 🕶️🧟), 10serviceops-collab: New LDAP user to trigger Jenkins downstream jobs - https://phabricator.wikimedia.org/T338950 (10jnuche) [14:07:36] 10Phabricator, 10Release-Engineering-Team (They Live 🕶️🧟), 10serviceops-collab, 10Developer-Advocacy (Apr-Jun 2023): Make quarterly and yearly Phabricator statistics / metrics emails have a second recipient - https://phabricator.wikimedia.org/T338955 (10Aklapper) p:05Triage→03Low [14:08:13] 10Phabricator, 10Release-Engineering-Team (They Live 🕶️🧟), 10serviceops-collab, 10Developer-Advocacy (Apr-Jun 2023): Automate SQL queries for annual end-of-year Phabricator statistics/metrics on wikitech-l@ - https://phabricator.wikimedia.org/T337388 (10Aklapper) >>! In T337388#8924672, @Dzahn wrote: > In... [14:40:00] (03PS1) 10Hashar: Review access change [eventgate-wikimedia] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/929653 [14:40:40] (03PS2) 10Hashar: Allow CI to submit changes [eventgate-wikimedia] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/929653 [14:43:04] (03CR) 10Hashar: "The permissions can be seen at https://gerrit.wikimedia.org/r/admin/repos/eventgate-wikimedia,access . Zuul/CI requires the Submit permiss" [eventgate-wikimedia] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/929653 (owner: 10Hashar) [14:43:40] (03CR) 10Hashar: "Promote persons in CC to reviewers :]" [eventgate-wikimedia] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/929653 (owner: 10Hashar) [14:54:20] (03CR) 10Ottomata: [C: 03+1] Allow CI to submit changes [eventgate-wikimedia] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/929653 (owner: 10Hashar) [15:18:33] 10Phabricator, 10Release-Engineering-Team (They Live 🕶️🧟), 10serviceops-collab, 10Developer-Advocacy (Apr-Jun 2023): Automate SQL queries for annual end-of-year Phabricator statistics/metrics on wikitech-l@ - https://phabricator.wikimedia.org/T337388 (10Dzahn) >>! In T337388#8924674, @Dzahn wrote: > Was th... [15:18:38] 10Phabricator, 10Release-Engineering-Team (They Live 🕶️🧟), 10serviceops-collab, 10Developer-Advocacy (Apr-Jun 2023), 10Patch-For-Review: Make quarterly and yearly Phabricator statistics / metrics emails have a second recipient - https://phabricator.wikimedia.org/T338955 (10Dzahn) Is this a duplicate of T... [15:22:20] (03CR) 10Hashar: [V: 03+2 C: 03+2] Allow CI to submit changes [eventgate-wikimedia] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/929653 (owner: 10Hashar) [15:35:04] 10GitLab (Project Migration), 10Release-Engineering-Team (They Live 🕶️🧟), 10serviceops-collab: New LDAP user to trigger Jenkins downstream jobs - https://phabricator.wikimedia.org/T338950 (10Dzahn) to start with, let's see what existing (jenkins-related) users we have an LDAP right now: ` [mwmaint1002:~] $... [15:36:41] 10GitLab (Project Migration), 10Release-Engineering-Team (They Live 🕶️🧟), 10serviceops-collab: New LDAP user to trigger Jenkins downstream jobs - https://phabricator.wikimedia.org/T338950 (10Dzahn) Hey @jBond_WMF @Muehlenhoff are system (non-human) users in LDAP a problem? Also see existing users above. Is c... [16:07:31] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 10Zuul, 10Patch-For-Review: Zuul jenkins-bot user holding open SSH sessions - https://phabricator.wikimedia.org/T309376 (10hashar) The Zuul scheduler configuration now has a `gerrit-reporter` connection defined. Since it is not ref... [16:13:59] (03CR) 10Hashar: "Puppet patch got merged so Zuul config now has two connections:" [integration/config] - 10https://gerrit.wikimedia.org/r/924900 (https://phabricator.wikimedia.org/T309376) (owner: 10Hashar) [16:28:23] 10GitLab (Project Migration), 10Release-Engineering-Team (They Live 🕶️🧟), 10serviceops-collab: New LDAP user to trigger Jenkins downstream jobs - https://phabricator.wikimedia.org/T338950 (10hashar) to start with, let's see what existing (jenkins-related) users we have in LDAP right now: | jenkins-bot | Tha... [16:33:04] 10GitLab (Project Migration), 10Release-Engineering-Team (They Live 🕶️🧟), 10serviceops-collab: New LDAP user to trigger Jenkins downstream jobs - https://phabricator.wikimedia.org/T338950 (10Dzahn) jenkins-17 looks like a human user totally unrelated to the software called jenkins. nevermind about that. tha... [16:37:09] 10GitLab (Project Migration), 10Release-Engineering-Team (They Live 🕶️🧟), 10serviceops-collab: New LDAP user to trigger Jenkins downstream jobs - https://phabricator.wikimedia.org/T338950 (10jnuche) > I don't think an anonymous user can trigger a build, can they? They need to get a token issued (via /crumbIs... [17:08:38] 10Phabricator, 10Release-Engineering-Team (They Live 🕶️🧟), 10serviceops-collab, 10Developer-Advocacy (Apr-Jun 2023), 10Patch-For-Review: Make quarterly and yearly Phabricator statistics / metrics emails have a second recipient - https://phabricator.wikimedia.org/T338955 (10Aklapper) >>! In T338955#892805... [17:12:19] 10Phabricator, 10Release-Engineering-Team (They Live 🕶️🧟), 10serviceops-collab, 10Developer-Advocacy (Apr-Jun 2023): Make quarterly and yearly Phabricator statistics / metrics emails have a second recipient - https://phabricator.wikimedia.org/T338955 (10Aklapper) >>! In T338955#8928408, @gerritbot wrote: >... [18:14:29] 10GitLab (Project Migration), 10Release-Engineering-Team (They Live 🕶️🧟): Make a tool to convert .pipeline/config.yaml to .gitlab-ci.yaml - https://phabricator.wikimedia.org/T327332 (10thcipriani) Is this still a good idea? Still in progress? [18:15:11] 10Release-Engineering-Team (They Live 🕶️🧟), 10Scap: `scap backport` dependency check for cherry-picked mediawiki commits - https://phabricator.wikimedia.org/T334984 (10jeena) We plan to release and deploy the changes that address this on June 15 [18:28:55] 10Release-Engineering-Team (They Live 🕶️🧟), 10Scap: scap backport applied +2 to wrong patchset when run shortly after a rebase and hung waiting for merge - https://phabricator.wikimedia.org/T333241 (10thcipriani) [18:29:21] 10Release-Engineering-Team (They Live 🕶️🧟), 10Scap: scap backport applied +2 to wrong patchset when run shortly after a rebase and hung waiting for merge - https://phabricator.wikimedia.org/T333241 (10jeena) a:03jeena [19:42:54] 10Phabricator: Update H333 to allow Inuka Team (next) - https://phabricator.wikimedia.org/T339024 (10SBisson) [19:44:00] 10Phabricator: Update H333 to allow Inuka Team (next) - https://phabricator.wikimedia.org/T339024 (10taavi) 05Open→03Resolved a:03taavi Done. [19:44:59] 10Phabricator: Update H333 to allow Inuka Team (next) - https://phabricator.wikimedia.org/T339024 (10Dzahn) done! {F37102769} [19:54:44] 10Release-Engineering-Team (They Live 🕶️🧟), 10Patch-For-Review, 10Release, 10Train Deployments: 1.41.0-wmf.13 deployment blockers - https://phabricator.wikimedia.org/T337527 (10matmarex) [20:03:37] 10Release-Engineering-Team (They Live 🕶️🧟), 10Anti-Harassment, 10Projects-Cleanup, 10Security-Team, 10iPoid-Service: Use Gitlab Security Pipeline for ipoid - https://phabricator.wikimedia.org/T338238 (10Mstyles) @kostajh I created [a merge request](https://gitlab.wikimedia.org/repos/mediawiki/services/ip... [20:10:51] 10Release-Engineering-Team (They Live 🕶️🧟), 10Anti-Harassment, 10Security-Team, 10iPoid-Service: Use Gitlab Security Pipeline for ipoid - https://phabricator.wikimedia.org/T338238 (10Pppery) [20:23:26] 10Release-Engineering-Team (They Live 🕶️🧟), 10Anti-Harassment, 10Security-Team, 10iPoid-Service: Use Gitlab Security Pipeline for ipoid - https://phabricator.wikimedia.org/T338238 (10CodeReviewBot) mstyles updated https://gitlab.wikimedia.org/repos/mediawiki/services/ipoid/-/merge_requests/8 Update .gitla... [20:34:34] (03CR) 10Cwhite: [C: 03+2] bump dashboards version to 2.7.0 [releng/phatality] - 10https://gerrit.wikimedia.org/r/928906 (https://phabricator.wikimedia.org/T333732) (owner: 10Cwhite) [20:35:03] (03Merged) 10jenkins-bot: bump dashboards version to 2.7.0 [releng/phatality] - 10https://gerrit.wikimedia.org/r/928906 (https://phabricator.wikimedia.org/T333732) (owner: 10Cwhite) [20:43:34] 10Release-Engineering-Team (They Live 🕶️🧟), 10Scap: scap backport applied +2 to wrong patchset when run shortly after a rebase and hung waiting for merge - https://phabricator.wikimedia.org/T333241 (10jeena) 05Open→03In progress [21:23:44] 10Release-Engineering-Team (They Live 🕶️🧟), 10Scap, 10Patch-For-Review: scap backport applied +2 to wrong patchset when run shortly after a rebase and hung waiting for merge - https://phabricator.wikimedia.org/T333241 (10CodeReviewBot) jhuneidi opened https://gitlab.wikimedia.org/repos/releng/scap/-/merge_re... [21:23:54] 10Release-Engineering-Team (They Live 🕶️🧟), 10Scap, 10Patch-For-Review: scap backport applied +2 to wrong patchset when run shortly after a rebase and hung waiting for merge - https://phabricator.wikimedia.org/T333241 (10CodeReviewBot)