[04:15:33] 10Continuous-Integration-Config, 10Release-Engineering-Team (Priority Backlog šŸ“„), 10Pywikibot, 10Upstream: tox v4 with skipsdist=true does not recognize use_develop=true - https://phabricator.wikimedia.org/T346238 (10hashar) I filed this one solely for tracking purpose. I intend to do all the debug/code/so... [07:16:34] 10Release-Engineering-Team (Deployment Training Requests): Deployment training request for mabualruz - https://phabricator.wikimedia.org/T345186 (10hashar) [07:58:58] apergos: I did the training with Mohd :) [09:15:50] 10GitLab (Pipeline Services MigrationšŸ¤), 10collaboration-services: move static-codereview.wikimedia.org to kubernetes - https://phabricator.wikimedia.org/T346309 (10Jelto) [09:16:11] 10GitLab (Pipeline Services MigrationšŸ¤), 10collaboration-services: move static-codereview.wikimedia.org to kubernetes - https://phabricator.wikimedia.org/T346309 (10Jelto) [09:16:18] 10GitLab (Pipeline Services MigrationšŸ¤), 10collaboration-services, 10Patch-For-Review: Move micro sites from Ganeti to Kubernetes and from Gerrit to GitLab - https://phabricator.wikimedia.org/T300171 (10Jelto) [09:16:45] 10GitLab (Pipeline Services MigrationšŸ¤), 10collaboration-services, 10Patch-For-Review: Move micro sites from Ganeti to Kubernetes and from Gerrit to GitLab - https://phabricator.wikimedia.org/T300171 (10Jelto) [09:19:29] 10GitLab (Pipeline Services MigrationšŸ¤), 10collaboration-services, 10Patch-For-Review: move static-codereview.wikimedia.org to kubernetes - https://phabricator.wikimedia.org/T346309 (10CodeReviewBot) jelto opened https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/45 add miscweb... [09:20:37] 10GitLab (Pipeline Services MigrationšŸ¤), 10collaboration-services, 10Patch-For-Review: move static-codereview.wikimedia.org to kubernetes - https://phabricator.wikimedia.org/T346309 (10CodeReviewBot) jelto merged https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/45 add miscweb... [09:30:45] 10GitLab (Pipeline Services MigrationšŸ¤), 10collaboration-services: move static-codereview.wikimedia.org to kubernetes - https://phabricator.wikimedia.org/T346309 (10Jelto) p:05Triageā†’03Medium [10:54:12] 10Phabricator: Move notification servers config to local files - https://phabricator.wikimedia.org/T346321 (10jnuche) [10:55:29] 10Phabricator: Move notification servers config to local files - https://phabricator.wikimedia.org/T346321 (10jnuche) @eoghan would you happen to know why the config value was added manually to the DB instead of keeping it in Puppet? [11:08:01] 10Release-Engineering-Team (Seen), 10MW-on-K8s, 10SRE, 10Traffic, 10serviceops: Serve production traffic via Kubernetes - https://phabricator.wikimedia.org/T290536 (10Clement_Goubert) [11:09:13] 10Release-Engineering-Team (Seen), 10MW-on-K8s, 10SRE, 10Traffic, and 2 others: Direct 5% of all traffic to mw-on-k8s - https://phabricator.wikimedia.org/T341780 (10Clement_Goubert) 05Openā†’03Resolved We are now serving 5% of global traffic from mw-on-k8s. Resolving. [11:09:59] 10Phabricator: Move notification servers config to local files - https://phabricator.wikimedia.org/T346321 (10Peachey88) Note there is/was some work to move to the DB model for more config compared to the files, {T239355} [11:23:00] 10Phabricator: Move notification servers config to local files - https://phabricator.wikimedia.org/T346321 (10jnuche) @Peachey88 Thanks for pointing to that. I'm not sure it applies anymore though, given we are more recently creating most of the Phab config with Puppet and writing it to files: https://gerrit.wik... [11:26:13] 10Continuous-Integration-Infrastructure, 10Jenkins, 10Release-Engineering-Team, 10Security: Jenkins core and plugins security advisory 2023-09-20 - https://phabricator.wikimedia.org/T346324 (10jnuche) [11:27:19] 10Continuous-Integration-Infrastructure, 10Jenkins, 10Release-Engineering-Team, 10Security: Jenkins core and plugins security advisory 2023-09-20 - https://phabricator.wikimedia.org/T346324 (10jnuche) [13:39:27] 10Release-Engineering-Team: Replace Go templates with Common Expression Language expressions in the JWT authorizer - https://phabricator.wikimedia.org/T346331 (10Aklapper) [13:41:16] 10GitLab (Pipeline Services MigrationšŸ¤), 10Metrics Platform Backlog, 10Data Products (Sprint 01): Migrate metrics-platform repo to GitLab - https://phabricator.wikimedia.org/T344733 (10VirginiaPoundstone) [14:06:18] (03PS1) 10Pwangai: Zuul: [mediawiki/extensions/Interwiki] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/957740 (https://phabricator.wikimedia.org/T321837) [14:07:30] (03PS1) 10Pwangai: Zuul: [mediawiki/extensions/ParserFunctions] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/957741 (https://phabricator.wikimedia.org/T321837) [14:09:02] (03PS1) 10Pwangai: Zuul: [mediawiki/skins/Timeless] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/957742 (https://phabricator.wikimedia.org/T321837) [14:10:18] (03PS1) 10Pwangai: Zuul: [mediawiki/skins/CologneBlue] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/957743 (https://phabricator.wikimedia.org/T321837) [14:23:56] 10GitLab (Pipeline Services MigrationšŸ¤), 10Math, 10Mathoid, 10MW-1.41-notes (1.41.0-wmf.27; 2023-09-19), 10Patch-For-Review: Migrate Mathoid to GitLab - https://phabricator.wikimedia.org/T344747 (10Jdforrester-WMF) [14:24:52] 10GitLab (Pipeline Services MigrationšŸ¤), 10Math, 10Mathoid, 10MW-1.41-notes (1.41.0-wmf.27; 2023-09-19), 10Patch-For-Review: Migrate Mathoid to GitLab - https://phabricator.wikimedia.org/T344747 (10Jdforrester-WMF) 05Openā†’03Resolved a:03Jdforrester-WMF OK, I've deployed the change switching the sou... [14:34:09] 10Release-Engineering-Team (Seen), 10MW-on-K8s, 10SRE, 10Traffic, 10serviceops: Serve production traffic via Kubernetes - https://phabricator.wikimedia.org/T290536 (10Jdforrester-WMF) [14:41:03] 10Continuous-Integration-Config, 10Release-Engineering-Team (Priority Backlog šŸ“„), 10Patch-For-Review: Deal with tox 4 upgrading - https://phabricator.wikimedia.org/T345695 (10hashar) [15:24:17] 10Scap, 10serviceops: restbase deploys via scap lead to all hosts being disabled in conftool - https://phabricator.wikimedia.org/T346354 (10hnowlan) [15:24:27] 10Scap, 10serviceops: restbase deploys via scap lead to all hosts being disabled in conftool - https://phabricator.wikimedia.org/T346354 (10hnowlan) p:05Triageā†’03High [15:32:18] 10Release-Engineering-Team, 10Scap, 10serviceops: restbase deploys via scap lead to all hosts being disabled in conftool - https://phabricator.wikimedia.org/T346354 (10hnowlan) [15:49:53] 10GitLab (Project Migration), 10SRE, 10Traffic: Move purged repository from Gerrit to GitLab - https://phabricator.wikimedia.org/T346305 (10Aklapper) + #gitlab-migration [15:50:53] 10GitLab (Project Migration), 10collaboration-services: Migrate SRE repositories to GitLab - operations/software - https://phabricator.wikimedia.org/T341504 (10Aklapper) [15:51:06] 10GitLab (Project Migration), 10collaboration-services: Migrate SRE repositories to GitLab - operations/software - https://phabricator.wikimedia.org/T341504 (10Aklapper) [15:51:12] 10GitLab (Project Migration), 10SRE, 10Traffic: Move purged repository from Gerrit to GitLab - https://phabricator.wikimedia.org/T346305 (10Aklapper) [15:58:12] 10Release-Engineering-Team (Priority Backlog šŸ“„), 10Patch-For-Review, 10Release, 10Train Deployments: 1.41.0-wmf.26 deployment blockers - https://phabricator.wikimedia.org/T343728 (10jnuche) 05Openā†’03Resolved [20:05:27] 10Release-Engineering-Team (Priority Backlog šŸ“„), 10Security-Team: Determine a better way to track Wikimedia production security patches - https://phabricator.wikimedia.org/T295925 (10sbassett) >>! In T295925#9160334, @jnuche wrote: > @sbassett from the point of view of the #security-team , the only change in t... [21:52:27] 10GitLab (Pipeline Services MigrationšŸ¤), 10Math, 10Mathoid, 10MW-1.41-notes (1.41.0-wmf.27; 2023-09-19), 10Patch-For-Review: Migrate Mathoid to GitLab - https://phabricator.wikimedia.org/T344747 (10thcipriani) Thank you @Jdforrester-WMF Archived Gerrit repo, too: https://gerrit.wikimedia.org/g/mediawiki/... [21:53:09] 10Beta-Cluster-Infrastructure, 10Observability-Logging: Beta logstash filled with kafka errors - https://phabricator.wikimedia.org/T346402 (10Daimona) [21:55:35] 10Release-Engineering-Team, 10Scap, 10Data-Platform-SRE: "scap deploy"'s config-deploy should check for broken symlinks - https://phabricator.wikimedia.org/T342162 (10RKemper) [22:16:29] 10Beta-Cluster-Infrastructure, 10Data-Engineering: Beta logstash filled with kafka errors - https://phabricator.wikimedia.org/T346402 (10colewhite) It appears there is some problem with the kafka-jumbo nodes in deployment prep. Logstash appears functional. Adding Data Engineering for the kafka-jumbo nodes. [22:17:31] 10Beta-Cluster-Infrastructure, 10Data-Engineering: Many kafka errors in beta/deployment-prep - https://phabricator.wikimedia.org/T346402 (10colewhite)