[00:02:01] (03Abandoned) 10Jforrester: Review access change [tools/release] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/632822 (owner: 10Jforrester) [00:15:37] (03PS2) 10Jforrester: Zuul: [mediawiki/extensions/WikimediaMaintenance] +AbuseFilter phan dep [integration/config] - 10https://gerrit.wikimedia.org/r/880540 (owner: 10Zabe) [00:15:45] (03CR) 10Jforrester: [C: 03+2] Zuul: [mediawiki/extensions/WikimediaMaintenance] +AbuseFilter phan dep [integration/config] - 10https://gerrit.wikimedia.org/r/880540 (owner: 10Zabe) [00:16:55] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/WikimediaMaintenance] +AbuseFilter phan dep [integration/config] - 10https://gerrit.wikimedia.org/r/880540 (owner: 10Zabe) [00:17:17] (03PS2) 10Jforrester: Zuul: [operations/debs/cadvisor] Add debian-glue-unstable CI [integration/config] - 10https://gerrit.wikimedia.org/r/880529 (https://phabricator.wikimedia.org/T325557) (owner: 10Ssingh) [00:19:23] (03CR) 10Jforrester: [C: 03+2] Zuul: [operations/debs/cadvisor] Add debian-glue-unstable CI [integration/config] - 10https://gerrit.wikimedia.org/r/880529 (https://phabricator.wikimedia.org/T325557) (owner: 10Ssingh) [00:21:15] (03Merged) 10jenkins-bot: Zuul: [operations/debs/cadvisor] Add debian-glue-unstable CI [integration/config] - 10https://gerrit.wikimedia.org/r/880529 (https://phabricator.wikimedia.org/T325557) (owner: 10Ssingh) [00:21:46] !log Zuul: [mediawiki/extensions/WikimediaMaintenance] +AbuseFilter phan dep [00:21:47] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [00:21:52] !log Zuul: [operations/debs/cadvisor] Add debian-glue-unstable CI for T325557 [00:21:54] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [00:21:55] T325557: Review cp2041 and cp2042 running bullseye - https://phabricator.wikimedia.org/T325557 [00:26:39] (03CR) 10Jforrester: "In my version of this (Ib2267eecee) I left it as WIP because I couldn't see how to add Python 2.7, and couldn't get someone to confirm tha" [integration/config] - 10https://gerrit.wikimedia.org/r/868624 (owner: 10David Caro) [00:31:12] (03CR) 10Jforrester: [C: 04-1] "Also should this go out after the bullseye image lands touching that set too?" [integration/config] - 10https://gerrit.wikimedia.org/r/871178 (owner: 10David Caro) [00:47:15] (03PS3) 10Jforrester: Zuul: [mediawiki/services/jobrunner] Use lib template (add doxygen and coverage) [integration/config] - 10https://gerrit.wikimedia.org/r/860062 [00:47:24] (03CR) 10CI reject: [V: 04-1] Zuul: [mediawiki/services/jobrunner] Use lib template (add doxygen and coverage) [integration/config] - 10https://gerrit.wikimedia.org/r/860062 (owner: 10Jforrester) [00:47:32] (03PS4) 10Jforrester: Zuul: [mediawiki/services/jobrunner] Use lib template (add doxygen and coverage) [integration/config] - 10https://gerrit.wikimedia.org/r/860062 [08:28:07] (03CR) 10Hashar: [C: 03+2] "+ Isabelle who maintains Parsoid and is in Europe timezone." [integration/config] - 10https://gerrit.wikimedia.org/r/880503 (https://phabricator.wikimedia.org/T321837) (owner: 10Pwangai) [08:29:17] (03Merged) 10jenkins-bot: Zuul: [mediawiki/services/parsoid] Add service to codehealth pipeline [integration/config] - 10https://gerrit.wikimedia.org/r/880503 (https://phabricator.wikimedia.org/T321837) (owner: 10Pwangai) [08:32:50] !log Reloading Zuul for https://gerrit.wikimedia.org/r/c/integration/config/+/880503 | T321837 [08:32:52] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [08:32:52] T321837: Repositories integrated into Codehealth Pipeline (Production) - https://phabricator.wikimedia.org/T321837 [08:58:00] (03PS1) 10Hashar: Add support for Parsoid service to codehealth jobs [integration/config] - 10https://gerrit.wikimedia.org/r/880885 (https://phabricator.wikimedia.org/T321837) [09:24:16] 10Phabricator: Make sure anti-vandalism features are up to snuff - https://phabricator.wikimedia.org/T84 (10Aklapper) [09:27:41] (03PS2) 10Hashar: Add support for Parsoid service to codehealth jobs [integration/config] - 10https://gerrit.wikimedia.org/r/880885 (https://phabricator.wikimedia.org/T321837) [09:27:43] (03PS1) 10Hashar: dockerfiles: sonar-scanner support Parsoid service and skins [integration/config] - 10https://gerrit.wikimedia.org/r/880892 (https://phabricator.wikimedia.org/T321837) [09:27:45] (03PS1) 10Hashar: jjb: update jobs for sonar-scanner [integration/config] - 10https://gerrit.wikimedia.org/r/880893 (https://phabricator.wikimedia.org/T321837) [09:30:25] (03CR) 10Hashar: [C: 03+2] dockerfiles: sonar-scanner support Parsoid service and skins [integration/config] - 10https://gerrit.wikimedia.org/r/880892 (https://phabricator.wikimedia.org/T321837) (owner: 10Hashar) [09:31:31] (03Merged) 10jenkins-bot: dockerfiles: sonar-scanner support Parsoid service and skins [integration/config] - 10https://gerrit.wikimedia.org/r/880892 (https://phabricator.wikimedia.org/T321837) (owner: 10Hashar) [09:52:13] !log Built image docker-registry.discovery.wmnet/releng/sonar-scanner:4.6.0.2311-4 [09:52:14] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [09:52:24] (03CR) 10Hashar: [C: 03+2] "Built image docker-registry.discovery.wmnet/releng/sonar-scanner:4.6.0.2311-4" [integration/config] - 10https://gerrit.wikimedia.org/r/880892 (https://phabricator.wikimedia.org/T321837) (owner: 10Hashar) [09:58:16] 10GitLab (Infrastructure), 10Release-Engineering-Team, 10serviceops-collab, 10Patch-For-Review: GitLab minor version upgrade 15.7.3 - https://phabricator.wikimedia.org/T326815 (10Jelto) I updated Debian package `gitlab-ce` to `15.7.3` and `gitlab-runner` to `15.7.2` on apt host. I'm going to update `gitla... [10:05:10] !log Updating sonar-scanner jobs https://gerrit.wikimedia.org/r/c/integration/config/+/880893/ [10:05:11] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [10:11:39] (03CR) 10Hashar: [C: 03+2] jjb: update jobs for sonar-scanner [integration/config] - 10https://gerrit.wikimedia.org/r/880893 (https://phabricator.wikimedia.org/T321837) (owner: 10Hashar) [10:11:49] (03CR) 10Hashar: [C: 03+2] Add support for Parsoid service to codehealth jobs [integration/config] - 10https://gerrit.wikimedia.org/r/880885 (https://phabricator.wikimedia.org/T321837) (owner: 10Hashar) [10:13:15] (03Merged) 10jenkins-bot: jjb: update jobs for sonar-scanner [integration/config] - 10https://gerrit.wikimedia.org/r/880893 (https://phabricator.wikimedia.org/T321837) (owner: 10Hashar) [10:13:18] (03Merged) 10jenkins-bot: Add support for Parsoid service to codehealth jobs [integration/config] - 10https://gerrit.wikimedia.org/r/880885 (https://phabricator.wikimedia.org/T321837) (owner: 10Hashar) [10:34:52] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10Patch-For-Review, 10Release, 10Train Deployments: 1.40.0-wmf.19 deployment blockers - https://phabricator.wikimedia.org/T325582 (10Zabe) [11:06:43] 10GitLab (Infrastructure), 10Release-Engineering-Team, 10serviceops-collab, 10Patch-For-Review: GitLab minor version upgrade 15.7.3 - https://phabricator.wikimedia.org/T326815 (10Jelto) I updated test instance to `gitlab-ce` to `15.7.3` and `gitlab-runner` to `15.7.2`. Test instance: [x] gitlab-prod-1001.... [11:19:51] (03PS1) 10Krinkle: doc: Update less.php links [integration/docroot] - 10https://gerrit.wikimedia.org/r/880927 [11:20:09] (03CR) 10Krinkle: [C: 03+2] doc: Update less.php links [integration/docroot] - 10https://gerrit.wikimedia.org/r/880927 (owner: 10Krinkle) [11:21:01] (03Merged) 10jenkins-bot: doc: Update less.php links [integration/docroot] - 10https://gerrit.wikimedia.org/r/880927 (owner: 10Krinkle) [11:33:52] 10Diffusion, 10Phabricator: Incorrectly attributed as the author of a random commit I have nothing to do with in a repository I've never touched - https://phabricator.wikimedia.org/T327105 (10ashley) Thanks for the quick fix & explanation, @Dylsss, much appreciated! πŸ‘ Never in a million years would I have thou... [12:49:47] (03PS1) 10Robert Vogel: Add VisualEditor as dependency to DrawioEditor [integration/config] - 10https://gerrit.wikimedia.org/r/880951 [12:56:58] (03CR) 10Hashar: [C: 03+2] Add VisualEditor as dependency to DrawioEditor [integration/config] - 10https://gerrit.wikimedia.org/r/880951 (owner: 10Robert Vogel) [12:58:06] (03Merged) 10jenkins-bot: Add VisualEditor as dependency to DrawioEditor [integration/config] - 10https://gerrit.wikimedia.org/r/880951 (owner: 10Robert Vogel) [12:58:32] !log Reloaded Zuul for https://gerrit.wikimedia.org/r/c/integration/config/+/880951/ [12:58:33] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [12:58:38] (03CR) 10Hashar: [C: 03+2] "deployed ;)" [integration/config] - 10https://gerrit.wikimedia.org/r/880951 (owner: 10Robert Vogel) [15:08:03] (03Abandoned) 10Hashar: Show CI status while waiting for change to merge [tools/release] - 10https://gerrit.wikimedia.org/r/826675 (https://phabricator.wikimedia.org/T315452) (owner: 10Hashar) [15:53:10] 10Release-Engineering-Team, 10SRE, 10SRE-OnFire, 10serviceops-collab, 10Sustainability: Remove old scap repositories from deploy1002 - https://phabricator.wikimedia.org/T309162 (10LSobanski) p:05Mediumβ†’03Low [15:57:28] 10Phabricator, 10serviceops-collab: create aphlict2001 (Phabricator realtime notifications codfw) - https://phabricator.wikimedia.org/T322369 (10LSobanski) p:05Triageβ†’03Medium [16:01:07] 10GitLab (Infrastructure), 10serviceops-collab: Setup alerting for GitLab projects size limits - https://phabricator.wikimedia.org/T316553 (10LSobanski) p:05Triageβ†’03Low [16:05:34] 10GitLab (CI & Job Runners), 10serviceops, 10serviceops-collab: Standardize Debian package builds on GitLab CI - https://phabricator.wikimedia.org/T304491 (10LSobanski) p:05Triageβ†’03Low [16:07:50] 10GitLab (Infrastructure), 10serviceops-collab: Create GitLab mirror - https://phabricator.wikimedia.org/T291322 (10LSobanski) [16:09:25] 10GitLab (Infrastructure), 10serviceops-collab: Create GitLab mirror - https://phabricator.wikimedia.org/T291322 (10LSobanski) p:05Triageβ†’03Low [16:11:13] 10GitLab (CI & Job Runners), 10serviceops-collab, 10Patch-For-Review: Evaluate if Docker can be executed as non-root on Trusted Runners - https://phabricator.wikimedia.org/T320411 (10LSobanski) p:05Triageβ†’03Low [16:13:48] 10GitLab (Infrastructure), 10Toolforge, 10serviceops-collab, 10Kubernetes, 10cloud-services-team (Kanban): gitlab: enable agent server for kubernetes (KAS) - https://phabricator.wikimedia.org/T320483 (10LSobanski) p:05Triageβ†’03Lowest [16:14:56] 10Gerrit, 10Release-Engineering-Team (Holiday Leftovers πŸ₯‘), 10SRE-OnFire, 10serviceops-collab, and 2 others: gerrit1001 running out of space on / - https://phabricator.wikimedia.org/T323262 (10LSobanski) a:05LSobanskiβ†’03None [16:15:49] 10Gerrit, 10Release-Engineering-Team (Holiday Leftovers πŸ₯‘), 10SRE-OnFire, 10serviceops-collab, and 2 others: gerrit1001 running out of space on / - https://phabricator.wikimedia.org/T323262 (10LSobanski) p:05Highβ†’03Medium [16:17:09] 10Release-Engineering-Team (Radar), 10serviceops-collab, 10serviceops-radar: give releng access to logs to debug buildkit-to-wmf-registry publishing - https://phabricator.wikimedia.org/T322579 (10LSobanski) p:05Highβ†’03Medium [16:19:04] 10Phabricator, 10Release-Engineering-Team (Radar), 10serviceops-collab, 10serviceops-radar, 10Sustainability (Incident Followup): Phabricator: Unable to view tasks in DB read-only mode - https://phabricator.wikimedia.org/T313879 (10LSobanski) 05Openβ†’03Stalled To be investigated during the Phorge PoC. [16:20:04] 10Phabricator, 10Release-Engineering-Team (Radar), 10serviceops-collab, 10serviceops-radar, 10Sustainability (Incident Followup): Phabricator: Unable to view tasks in DB read-only mode - https://phabricator.wikimedia.org/T313879 (10LSobanski) p:05Highβ†’03Low [16:24:22] 10GitLab (Infrastructure), 10serviceops-collab: Migrate gitlab-test instance to bullseye - https://phabricator.wikimedia.org/T318521 (10LSobanski) a:03Dzahn [16:25:04] 10GitLab (Infrastructure), 10serviceops-collab: Migrate gitlab-test instance to bullseye - https://phabricator.wikimedia.org/T318521 (10LSobanski) p:05Triageβ†’03High [16:27:08] 10Phabricator, 10serviceops-collab, 10Patch-For-Review: Phabricator phd.service refuses to start on machine boot - https://phabricator.wikimedia.org/T326146 (10LSobanski) p:05Triageβ†’03Low [16:27:39] 10GitLab (Infrastructure), 10serviceops-collab, 10Patch-For-Review: Align and refactor GitLab restore scripts - https://phabricator.wikimedia.org/T326315 (10LSobanski) p:05Mediumβ†’03High [16:34:57] 10GitLab, 10serviceops-collab: Investigate incremental backups for GitLab - https://phabricator.wikimedia.org/T324506 (10LSobanski) [16:40:18] 10Phabricator, 10Release-Engineering-Team (Radar), 10serviceops-collab, 10serviceops-radar, 10Sustainability (Incident Followup): Phabricator: Unable to view tasks in DB read-only mode - https://phabricator.wikimedia.org/T313879 (10LSobanski) [16:40:33] 10Phabricator, 10serviceops-collab: make Phabricator work with readonly DB - https://phabricator.wikimedia.org/T323312 (10LSobanski) [16:41:23] 10Phabricator, 10Release-Engineering-Team (Radar), 10serviceops-collab, 10serviceops-radar, 10Sustainability (Incident Followup): Phabricator: Unable to view tasks in DB read-only mode - https://phabricator.wikimedia.org/T313879 (10LSobanski) [16:42:06] (03CR) 10Ahmon Dancy: [C: 03+1] Migrate release repo to gitlab [tools/release] - 10https://gerrit.wikimedia.org/r/879906 (https://phabricator.wikimedia.org/T290260) (owner: 10Jeena Huneidi) [16:43:31] (03Abandoned) 10Ahmon Dancy: make-container-image: push -q to all docker-pusher calls [tools/release] - 10https://gerrit.wikimedia.org/r/802648 (owner: 10Ahmon Dancy) [16:45:41] 10GitLab, 10serviceops, 10serviceops-collab, 10Kubernetes: Trusted gitlab runner containers need access to staging k8s cluster - https://phabricator.wikimedia.org/T325385 (10LSobanski) 05Openβ†’03Resolved a:03Dzahn Resolving as it seems like the original request was addressed. If there's follow up disc... [16:45:54] 10Continuous-Integration-Infrastructure, 10SRE, 10serviceops-collab, 10Patch-For-Review: contint2002 service implementation tracking - https://phabricator.wikimedia.org/T324659 (10LSobanski) a:03Dzahn [18:59:01] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10Patch-For-Review, 10Release, 10Train Deployments: 1.40.0-wmf.19 deployment blockers - https://phabricator.wikimedia.org/T325582 (10Zabe) [19:11:27] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10Patch-For-Review, 10Release, 10Train Deployments: 1.40.0-wmf.19 deployment blockers - https://phabricator.wikimedia.org/T325582 (10Jdlrobson) **RISKY CHANGE** Just noting that tomorrow the default skin for English Wikipedia will be changing. If any issues... [19:33:48] 10Scap, 10Parsoid, 10SRE, 10serviceops: scap groups on bastions still needed? - https://phabricator.wikimedia.org/T327066 (10Dzahn) +1, I also think those are not relevant anymore. If something like this is needed it should be done from deployment servers or maybe mwmaint but not bastions. I would say rem... [21:02:34] (03Abandoned) 10Thcipriani: logstash checker: port to python, magic task update [tools/release] - 10https://gerrit.wikimedia.org/r/870944 (owner: 10Thcipriani) [21:18:11] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10Fundraising-Backlog, 10MediaWiki-extensions-DonationInterface, and 3 others: Update CI config for new fundraising/REL1_39 branch of mediawiki/core under php7.4 - https://phabricator.wikimedia.org/T326372 (10XenoRyet) [21:19:21] (03CR) 10Jeena Huneidi: "We are getting ready to archive this repo, so I've moved the change request to gitlab: https://gitlab.wikimedia.org/repos/releng/release/-" [tools/release] - 10https://gerrit.wikimedia.org/r/868398 (owner: 10ClΓ©ment Goubert) [21:47:32] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10Patch-For-Review, 10Release, 10Train Deployments: 1.40.0-wmf.19 deployment blockers - https://phabricator.wikimedia.org/T325582 (10Urbanecm) [21:52:51] 10GitLab (CI & Job Runners), 10Release-Engineering-Team, 10mwcli: Add registry.gitlab.com/dependabot-gitlab/dependabot to list of allowed images for gitlab runners - https://phabricator.wikimedia.org/T326507 (10Addshore) [22:11:07] 10Beta-Cluster-Infrastructure, 10Cloud-VPS (Debian Stretch Deprecation): Cloud VPS "deployment-prep" project Stretch deprecation - https://phabricator.wikimedia.org/T306068 (10Andrew) We have several Stretch VMs that have been shutoff for quite some time: # deployment-echostore01.deployment-prep.eqiad1.wik... [22:14:08] 10Beta-Cluster-Infrastructure, 10Performance-Team: Upgrade deployment-mdb01 to Buster/Bullseye - https://phabricator.wikimedia.org/T301637 (10Andrew) 05Openβ†’03Resolved a:03Andrew done. [22:14:11] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Radar): Migrate deployment-prep away from Debian Stretch to Buster/Bullseye - https://phabricator.wikimedia.org/T278641 (10Andrew) [22:14:59] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Doing), 10Patch-For-Review: Replace deployment-imagescaler03 (stretch) with deployment-imagescaler04 (buster) - https://phabricator.wikimedia.org/T294148 (10Andrew) 05Openβ†’03Resolved done [22:15:01] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Radar): Migrate deployment-prep away from Debian Stretch to Buster/Bullseye - https://phabricator.wikimedia.org/T278641 (10Andrew) [22:16:32] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Radar): Migrate deployment-prep away from Debian Stretch to Buster/Bullseye - https://phabricator.wikimedia.org/T278641 (10Andrew) [22:16:37] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Radar): Migrate deployment-prep away from Debian Stretch to Buster/Bullseye - https://phabricator.wikimedia.org/T278641 (10Andrew) [22:16:39] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Doing), 10Patch-For-Review: Replace deployment-imagescaler03 (stretch) with deployment-imagescaler04 (buster) - https://phabricator.wikimedia.org/T294148 (10Andrew) 05Resolvedβ†’03Open sorry, wrong window! This still needs to be replaced. I may sh... [22:16:42] 10Beta-Cluster-Infrastructure, 10Wikifeeds: Upgrade deployment-wikifeeds01 host to Buster - https://phabricator.wikimedia.org/T306055 (10Andrew) 05Openβ†’03Resolved a:03Andrew [22:18:56] dancy: deployment-imagescaler03 is extremely, extremely overdue for replacement with a Bullseye VM. Can you give that some attention, or suggest who else I should ask? https://phabricator.wikimedia.org/T294148 [22:40:49] (03PS3) 10Jeena Huneidi: layout.yaml: remove release repo [integration/config] - 10https://gerrit.wikimedia.org/r/879907 (https://phabricator.wikimedia.org/T290260) [22:41:36] (03CR) 10Jeena Huneidi: "I added the depends-on so we can make sure production is using the new repo before stopping tests/archiving the gerrit one." [integration/config] - 10https://gerrit.wikimedia.org/r/879907 (https://phabricator.wikimedia.org/T290260) (owner: 10Jeena Huneidi) [23:14:38] andrewbogott: Unfortunately I don't think I can be helpful on that matter. I'm in favor of the suggestion to turn it off and see what breaks. [23:17:03] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Doing), 10Patch-For-Review: Replace deployment-imagescaler03 (stretch) with deployment-imagescaler04 (buster) - https://phabricator.wikimedia.org/T294148 (10dancy) >>! In T294148#8533034, @Andrew wrote: > I may shut down the stretch VM soon in order... [23:17:57] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Doing), 10Patch-For-Review: Replace deployment-imagescaler03 (stretch) with deployment-imagescaler04 (buster) - https://phabricator.wikimedia.org/T294148 (10dancy) a:05dancyβ†’03None Unassigning myself since I don't have any actions on this. [23:20:55] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10Patch-For-Review, 10Release, 10Train Deployments: 1.40.0-wmf.19 deployment blockers - https://phabricator.wikimedia.org/T325582 (10Zabe)