[00:07:25] (03PS1) 10Umherirrender: Zuul: [mediawiki/libs/Bcp47Code] Add basic test [integration/config] - 10https://gerrit.wikimedia.org/r/863056 [01:00:07] (03CR) 10Jforrester: [C: 03+2] Zuul: [mediawiki/libs/Bcp47Code] Add basic test [integration/config] - 10https://gerrit.wikimedia.org/r/863056 (owner: 10Umherirrender) [01:01:50] (03Merged) 10jenkins-bot: Zuul: [mediawiki/libs/Bcp47Code] Add basic test [integration/config] - 10https://gerrit.wikimedia.org/r/863056 (owner: 10Umherirrender) [01:02:25] (03CR) 10Jforrester: [C: 03+2] zuul: [mediawiki/extensions/ImageSuggestions] Add phan dependency [integration/config] - 10https://gerrit.wikimedia.org/r/863015 (owner: 10Umherirrender) [01:02:57] !log Zuul: [mediawiki/libs/Bcp47Code] Add basic test [01:02:58] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [01:04:07] (03Merged) 10jenkins-bot: zuul: [mediawiki/extensions/ImageSuggestions] Add phan dependency [integration/config] - 10https://gerrit.wikimedia.org/r/863015 (owner: 10Umherirrender) [01:04:48] !log Zuul: [mediawiki/extensions/ImageSuggestions] Add phan CirrusSearch and Elastica dependencies for T302711 [01:04:50] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [01:04:51] T302711: [M] Deploy ImageSuggestions - https://phabricator.wikimedia.org/T302711 [01:22:45] (03PS1) 10Umherirrender: Zuul: [mediawiki/extensions/WatchAnalytics] Fix name [integration/config] - 10https://gerrit.wikimedia.org/r/863058 [01:24:29] (03CR) 10CI reject: [V: 04-1] Zuul: [mediawiki/extensions/WatchAnalytics] Fix name [integration/config] - 10https://gerrit.wikimedia.org/r/863058 (owner: 10Umherirrender) [01:29:22] (03CR) 10Umherirrender: "recheck" [integration/config] - 10https://gerrit.wikimedia.org/r/863058 (owner: 10Umherirrender) [01:31:39] (03PS2) 10Umherirrender: Zuul: [mediawiki/extensions/WatchAnalytics] Fix name [integration/config] - 10https://gerrit.wikimedia.org/r/863058 [01:33:40] (03CR) 10Jforrester: [C: 03+2] "Doh." [integration/config] - 10https://gerrit.wikimedia.org/r/863058 (owner: 10Umherirrender) [01:35:26] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/WatchAnalytics] Fix name [integration/config] - 10https://gerrit.wikimedia.org/r/863058 (owner: 10Umherirrender) [01:51:44] !log Zuul: [mediawiki/extensions/WatchAnalytics] Fix name [01:51:46] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [06:58:33] 10Phabricator, 10Trust-and-Safety, 10cloud-services-team (Kanban): Reset Phab & Wikitech 2FA for Zhuyifei1999 - https://phabricator.wikimedia.org/T324309 (10Zhuyifei1999_test) [08:00:38] 10Diffusion, 10Release-Engineering-Team, 10serviceops-collab: reconsider https://git.wikimedia.org link - https://phabricator.wikimedia.org/T323073 (10Aklapper) (Note that https://www.mediawiki.org/wiki/Talk:Phabricator/Diffusion states that `git.wikimedia.org` was also used in some commit messages.) [08:14:20] 10GitLab (Project Migration), 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥), 10serviceops-collab, and 3 others: Migrate active repositories in Phabricator Differential to GitLab - https://phabricator.wikimedia.org/T191182 (10Aklapper) FYI, I updated related docs in https://www.mediawiki.or... [08:31:08] 10Diffusion, 10Phabricator, 10serviceops-collab: Redirect https://phabricator.wikimedia.org/r/ to https://gerrit.wikimedia.org/g/ - https://phabricator.wikimedia.org/T324311 (10Aklapper) p:05Triage→03Low [09:40:55] (03PS1) 10David Caro: Remove Bug header order restriction [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/863238 (https://phabricator.wikimedia.org/T324316) [09:58:11] 10Gerrit, 10Patch-For-Review: Gerrit 3.2 upgrade - https://phabricator.wikimedia.org/T254158 (10hashar) [09:58:37] 10Continuous-Integration-Config, 10Gerrit, 10Release-Engineering-Team (Seen), 10Developer Productivity: Jenkins comments no longer formatted on Gerrit 3 - https://phabricator.wikimedia.org/T256575 (10hashar) 05Open→03Declined I gave it a try using a JavaScript plugin that would crawl the DOM for report... [09:59:42] (03Abandoned) 10Hashar: Plugin to customize Zuul reports [software/gerrit] (deploy/wmf/stable-3.5) - 10https://gerrit.wikimedia.org/r/858598 (owner: 10Hashar) [10:00:58] 10Project-Admins, 10User-Slst2020: Create project tag for Outreachy internship project - https://phabricator.wikimedia.org/T324317 (10Slst2020) [10:51:45] 10Phabricator, 10Release-Engineering-Team, 10Wikimedia-Phabricator-Extensions: Custom Phab Reports: "Unhandled Exception ("RuntimeException") Division by zero" - https://phabricator.wikimedia.org/T324319 (10Aklapper) [10:51:49] 10Phabricator, 10Release-Engineering-Team, 10Wikimedia-Phabricator-Extensions: Custom Phab Reports: "Unhandled Exception ("RuntimeException") count(): Parameter must be an array or an object that implements Countable" - https://phabricator.wikimedia.org/T324320 (10Aklapper) [10:59:16] !log Updating development images bullseye-openldap and puppet-debugger for merge request 22 and 24 [10:59:17] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [11:37:26] 10Project-Admins: Create three Phab Projects for Machine Learning: Lift Wing, Pilot Flag, Test Grounds - https://phabricator.wikimedia.org/T264774 (10Aklapper) @calbon: In two years, #Test-Grounds has seen one task (`T265068`) which got declined. Is #Test-Grounds still a thing or should it be archived? [12:44:05] jnuche: do you know what happens to scap3-dev when the containers are not run with privileged: true? Does it explodes completely? :\ [12:45:09] hashar: yes, the ones with systemd fail to start altogether [12:45:58] nice [12:49:43] I need to unlearn my Gerrit now [12:50:13] I have chained a bunch of commits and I guess I now have to rebase each of them on tip of the branhc to forge multiple merge requests [12:50:23] I am learning it the hard way :-\ [12:54:13] 10Phabricator, 10Developer-Advocacy (Oct-Dec 2022): Review / clean up Phabricator project page side bar menu items (misc) - https://phabricator.wikimedia.org/T324330 (10Aklapper) 05Open→03Resolved p:05Triage→03Low [13:02:45] 10Phabricator: Creating account with alias causes sending verify link to incorrect email address - https://phabricator.wikimedia.org/T324324 (10Aklapper) 05Open→03Invalid If you enter an incorrect email address when registering a Phabricator account, then Phabricator will try to send its verification email a... [13:02:55] 10Phabricator: Creating account with alias causes sending verify link to incorrect email address - https://phabricator.wikimedia.org/T324324 (10Aklapper) [13:03:01] 10Phabricator: Creating account with alias causes sending verify link to incorrect email address - https://phabricator.wikimedia.org/T324324 (10Aklapper) (This task is now public and world readable.) [13:05:41] 10Phabricator, 10Developer-Advocacy (Oct-Dec 2022): Review / clean up Phabricator project page side bar menu items (misc) - https://phabricator.wikimedia.org/T324330 (10Aklapper) [13:07:13] 10Phabricator: Creating account with alias causes sending verify link to incorrect email address - https://phabricator.wikimedia.org/T324324 (10Aklapper) >>! In T324324#8438807, @Emicbie22 wrote: > it took me a long time to note that was the REASON why I didn't received my confirmation link - it was sending to i... [13:14:03] hashar: I'm going to continue working on adding CasC configuration, do you want to pair on that? [13:14:30] 10Project-Admins, 10User-Slst2020: Create project tag for Outreachy internship project - https://phabricator.wikimedia.org/T324317 (10Aklapper) a:03Aklapper > A repository for the project will be created in the next week or two. Once it's existing, please feel encouraged to fill in the "Source Repo" field at... [13:15:11] 10Project-Admins, 10User-Slst2020: Create project tag for Outreachy internship project - https://phabricator.wikimedia.org/T324317 (10Aklapper) 05Open→03Resolved p:05Triage→03Medium Requested public project #Toolhunt has been created: https://phabricator.wikimedia.org/project/view/6283/ (In case you n... [13:51:12] jnuche: sure [13:51:29] I am giving up on using gitlab for today ;) [13:55:18] hashar: I'm on spoffice [14:12:05] PROBLEM - Work requests waiting in Zuul Gearman server on contint2001 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [400.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/d/000000322/zuul-gearman?orgId=1&viewPanel=10 [14:25:37] PROBLEM - Host contint1001 is DOWN: PING CRITICAL - Packet loss = 100% [14:41:40] (Queue (Jenkins jobs + Zuul functions) alert) firing: Queue (Jenkins jobs + Zuul functions) alert - https://alerts.wikimedia.org/?q=alertname%3DQueue+%28Jenkins+jobs+%2B+Zuul+functions%29+alert [14:46:39] (Queue (Jenkins jobs + Zuul functions) alert) firing: (2) Queue (Jenkins jobs + Zuul functions) alert - https://alerts.wikimedia.org/?q=alertname%3DQueue+%28Jenkins+jobs+%2B+Zuul+functions%29+alert [15:09:35] (03PS1) 10Umherirrender: Zuul: [mediawiki/extensions/PagePermissions] Add basic tests [integration/config] - 10https://gerrit.wikimedia.org/r/863341 [15:39:52] How do we handle Phabricator "Audit" spam? https://phabricator.wikimedia.org/rMSFSa19a2e30654f43e9ea270399685849586e3dd682 [15:53:20] (03CR) 10Jforrester: [C: 03+2] Zuul: [mediawiki/extensions/PagePermissions] Add basic tests [integration/config] - 10https://gerrit.wikimedia.org/r/863341 (owner: 10Umherirrender) [15:55:40] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/PagePermissions] Add basic tests [integration/config] - 10https://gerrit.wikimedia.org/r/863341 (owner: 10Umherirrender) [15:55:53] 10Project-Admins, 10User-Slst2020: Create project tag for Outreachy internship project - https://phabricator.wikimedia.org/T324317 (10Slst2020) Thank you @Aklapper ! [15:56:50] James_F: we can probably phab-ban the user for a start? [15:56:56] the task they created also makes no sense imho [15:57:33] 10Phabricator, 10Trust-and-Safety, 10cloud-services-team (Kanban): Reset Phab & Wikitech 2FA for Zhuyifei1999 - https://phabricator.wikimedia.org/T324309 (10bd808) Please do follow the ssh based verification process. [15:58:35] Lucas_WMDE: Ack, but for tasks we have the "trash" project but I don't even know how to untag the commit for audit. I mean, I could just unpublish the mirror of the repo… [15:59:17] I think I remember hearing about some way to “rollback” all of a user’s actions in Phabricator, but I don’t know where that would be or what kind of access it needs [15:59:30] this thing https://wikitech.wikimedia.org/wiki/Phabricator#Revert_all_activity_of_a_given_user [16:00:10] might be a heavy hammer for this case though [16:00:23] Eh. [16:00:25] Yeah. [16:00:35] Also I'm not sure if I have the rights for that. [16:00:42] I’m pretty sure I don’t [16:02:17] I’ve phab-banned and mw-blocked the user at least [16:02:22] Thanks. [16:03:25] I'll just ignore it. [16:04:36] ok, apparently removing them from the auditors was enough to trigger the “no longer requires audit” magic [16:06:02] Aha. Nice. [16:07:26] !log Zuul: [mediawiki/extensions/PagePermissions] Add basic tests [16:07:27] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:17:03] RECOVERY - Work requests waiting in Zuul Gearman server on contint2001 is OK: OK: Less than 100.00% above the threshold [200.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/d/000000322/zuul-gearman?orgId=1&viewPanel=10 [17:21:39] (Queue (Jenkins jobs + Zuul functions) alert) resolved: Queue (Jenkins jobs + Zuul functions) alert - https://alerts.wikimedia.org/?q=alertname%3DQueue+%28Jenkins+jobs+%2B+Zuul+functions%29+alert [17:36:12] 10Gerrit, 10Release-Engineering-Team (Seen), 10SRE, 10serviceops-collab: Create Gerrit Administrator right policy - https://phabricator.wikimedia.org/T218686 (10LSobanski) [17:38:59] 10Release-Engineering-Team (Priority Backlog 📥), 10Patch-For-Review, 10Release, 10Train Deployments: 1.40.0-wmf.12 deployment blockers - https://phabricator.wikimedia.org/T320517 (10dancy) 05Open→03Resolved [19:38:27] 10Release-Engineering-Team (GitLab III: GitLab in LA 🪃): Provision Ingress with TLS for reggie to allow image reuse in pipelines - https://phabricator.wikimedia.org/T324361 (10dduvall) [19:38:38] 10Release-Engineering-Team (GitLab III: GitLab in LA 🪃): Provision Ingress with TLS for reggie to allow image reuse in pipelines - https://phabricator.wikimedia.org/T324361 (10dduvall) a:03dduvall [19:47:21] (03PS16) 10Hashar: Replace CI results table by Gerrit Check API [software/gerrit] (deploy/wmf/stable-3.5) - 10https://gerrit.wikimedia.org/r/859083 (https://phabricator.wikimedia.org/T214068) [19:47:25] (03PS6) 10Hashar: Boilerplate for QUnit testing [software/gerrit] (deploy/wmf/stable-3.5) - 10https://gerrit.wikimedia.org/r/861486 [20:38:04] 10Gerrit, 10Release-Engineering-Team (Seen), 10SRE, 10serviceops-collab: Create Gerrit Administrator right policy - https://phabricator.wikimedia.org/T218686 (10Dzahn) Probably needs input from Tim because he wrote the access policy after this came up in the past and there was a lot of discussion about it... [20:58:37] 10Phabricator, 10Release-Engineering-Team, 10VPS-project-Phabricator, 10User-brennen: phab.wmflabs.org down: Unable to load the "Arcanist" library - https://phabricator.wikimedia.org/T314460 (10Dylsss) 05Open→03Resolved Error is gone. I guess someone has fixed it. [21:16:06] 10GitLab (Project Migration), 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥), 10serviceops-collab, and 3 others: Migrate active repositories in Phabricator Differential to GitLab - https://phabricator.wikimedia.org/T191182 (10Dzahn) Thank you, @Aklapper :) [21:19:21] 10Phabricator, 10Diffusion-Repository-Administrators, 10serviceops-collab: Audit Diffusion-Repository-Administrators group membership and rights - https://phabricator.wikimedia.org/T324171 (10Dzahn) [21:25:45] 10Diffusion, 10Release-Engineering-Team, 10serviceops-collab: reconsider https://git.wikimedia.org link - https://phabricator.wikimedia.org/T323073 (10Dzahn) >>! In T323073#8407610, @Aklapper wrote: > Do we know how many hits https://git.wikimedia.org gets? {F35829042} [21:55:04] 10Phabricator, 10Trust-and-Safety, 10cloud-services-team (Kanban): Reset Phab & Wikitech 2FA for Zhuyifei1999 - https://phabricator.wikimedia.org/T324309 (10zhuyifei1999) Done. bastion-eqiad1-03.bastion.eqiad1.wikimedia.cloud:/home/zhuyifei1999/2fa-reset-request.txt [23:02:58] 10Phabricator: Remove "Active Differential users" stats from monthly community_metrics.sh email - https://phabricator.wikimedia.org/T324131 (10Dzahn) [23:03:00] 10GitLab (Project Migration), 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥), 10serviceops-collab, and 3 others: Migrate active repositories in Phabricator Differential to GitLab - https://phabricator.wikimedia.org/T191182 (10Dzahn) 05Open→03In progress [23:07:57] 10Phabricator, 10Release-Engineering-Team, 10translatewiki.net, 10Language-Team (Language-2022-October-December), 10Unplanned-Sprint-Work: Phabricator translations can no more be received due to decommissioning of git-ssh - https://phabricator.wikimedia.org/T321350 (10Dzahn) [23:08:04] 10GitLab (Project Migration), 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥), 10serviceops-collab, and 3 others: Migrate active repositories in Phabricator Differential to GitLab - https://phabricator.wikimedia.org/T191182 (10Dzahn)