[04:10:45] (03PS1) 10Krinkle: zuul: Enable CI jobs for mediawiki/tools/grunt-cssjanus [integration/config] - 10https://gerrit.wikimedia.org/r/902897 [04:12:57] (03PS2) 10Krinkle: zuul: Enable CI jobs for mediawiki/tools/grunt-cssjanus [integration/config] - 10https://gerrit.wikimedia.org/r/902897 [04:17:20] (03CR) 10Krinkle: [C: 03+2] zuul: Enable CI jobs for mediawiki/tools/grunt-cssjanus [integration/config] - 10https://gerrit.wikimedia.org/r/902897 (owner: 10Krinkle) [04:18:31] (03Merged) 10jenkins-bot: zuul: Enable CI jobs for mediawiki/tools/grunt-cssjanus [integration/config] - 10https://gerrit.wikimedia.org/r/902897 (owner: 10Krinkle) [07:33:09] 10Release-Engineering-Team (Seen), 10MW-on-K8s, 10SRE, 10Traffic, and 3 others: Migrate internal traffic to k8s - https://phabricator.wikimedia.org/T333120 (10Joe) [08:23:27] 10Release-Engineering-Team (GitLab V: Event Horizon 🌄), 10Release, 10Train Deployments: 1.41.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T330208 (10Urbanecm_WMF) FTR: I've backported the patch I mentioned as T330208#8717262. [08:25:38] 10Gerrit: Error while fetching results for wm-zuul-status: TypeError: can't access property "pipeline", status.jobs[0] is undefined - https://phabricator.wikimedia.org/T333126 (10kostajh) [08:45:16] (03PS2) 10Hashar: wm-zuul-status: filter out non-live item [software/gerrit] (deploy/wmf/stable-3.5) - 10https://gerrit.wikimedia.org/r/902705 (https://phabricator.wikimedia.org/T214068) [08:54:08] 10Gerrit: Error while fetching results for wm-zuul-status: TypeError: can't access property "pipeline", status.jobs[0] is undefined - https://phabricator.wikimedia.org/T333126 (10hashar) Thanks! The issue got reported on the task tracking the plugin implementation and I have did the analysis there. The name of... [10:06:10] 10Release-Engineering-Team (Seen), 10MW-on-K8s, 10SRE, 10Traffic, and 3 others: Serve production traffic via Kubernetes - https://phabricator.wikimedia.org/T290536 (10Clement_Goubert) [10:06:20] 10Release-Engineering-Team (Seen), 10MW-on-K8s, 10SRE, 10Traffic, and 4 others: Migrate internal traffic to k8s - https://phabricator.wikimedia.org/T333120 (10Clement_Goubert) 05Open→03In progress p:05Triage→03Medium a:03Clement_Goubert [10:16:45] 10Phabricator, 10Release-Engineering-Team, 10Security-Team: Make it more difficult to unintentionally paste secrets, passwords, and keys in Phabricator task descriptions, comments and pastes - https://phabricator.wikimedia.org/T332639 (10kostajh) >>! In T332639#8712071, @sbassett wrote: > So there are at lea... [11:05:12] 10Release-Engineering-Team (GitLab V: Event Horizon 🌄), 10Release, 10Train Deployments: 1.41.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T330208 (10matej_suchanek) ##### Risky Patch! 🚂🔥 * **Change**: https://gerrit.wikimedia.org/r/445185 * **Summary**: ** !!Why is it risky?!! Large pat... [11:19:53] (03PS2) 10Samtar: Add my other email address to CI allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/902804 (owner: 10Bartosz Dziewoński) [11:47:58] (03CR) 10Hashar: [C: 03+2] Add my other email address to CI allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/902804 (owner: 10Bartosz Dziewoński) [11:49:03] (03Merged) 10jenkins-bot: Add my other email address to CI allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/902804 (owner: 10Bartosz Dziewoński) [11:49:30] (03CR) 10Hashar: [C: 03+2] "Deployed" [integration/config] - 10https://gerrit.wikimedia.org/r/902804 (owner: 10Bartosz Dziewoński) [12:02:07] 10Gerrit, 10Release-Engineering-Team, 10serviceops-collab, 10Sustainability (Incident Followup): Move Gerrit data out of root partition - https://phabricator.wikimedia.org/T333143 (10hashar) [12:06:33] 10Gerrit, 10Release-Engineering-Team (Priority Backlog 📥), 10SRE-OnFire, 10SRE-Sprint-Week-Sustainability-March2023, and 3 others: gerrit1001 running out of space on / - https://phabricator.wikimedia.org/T323262 (10hashar) 05Open→03Resolved a:03Clement_Goubert I have finally filled the follow up task... [12:14:44] 10Phabricator-Bot-Requests, 10Infrastructure-Foundations: update permissions for offboarding user - https://phabricator.wikimedia.org/T333146 (10jbond) [12:58:33] 10GitLab, 10mwcli: Gitlab-Github mirror broken - https://phabricator.wikimedia.org/T333029 (10Addshore) >>! In T333029#8726985, @hashar wrote: > Alternative is to have Gitlab replication to use it is own based after the GitLab repository name instead of the Gerrit one. Eg: > > | Source | GitHub > |--|-- > | G... [13:25:52] 10GitLab (CI & Job Runners), 10serviceops-collab: self-build/import registry.gitlab.com/gitlab-org/release-cli image - https://phabricator.wikimedia.org/T333161 (10Jelto) [13:27:21] 10GitLab (CI & Job Runners), 10Release-Engineering-Team, 10serviceops-collab: Define access to external resources for GitLab CI Runners - https://phabricator.wikimedia.org/T320730 (10Jelto) >>! In T320730#8420142, @MatthewVernon wrote: > I think policy is now that Trusted runners only use internal images; wo... [13:55:56] 10Release-Engineering-Team (Seen), 10MW-on-K8s, 10SRE, 10Traffic, and 3 others: Migrate internal traffic to k8s - https://phabricator.wikimedia.org/T333120 (10JJMC89) [13:59:13] (03CR) 10Bartosz Dziewoński: Add my other email address to CI allowlist (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/902804 (owner: 10Bartosz Dziewoński) [14:13:10] 10GitLab (CI & Job Runners), 10Release-Engineering-Team: Gitlab CI failure: ContainersNotReady: "containers with unready status: [build helper istio-proxy]" - https://phabricator.wikimedia.org/T333047 (10brennen) @dduvall, @dancy I feel like this might duplicate an existing issue? [14:59:04] 10GitLab (CI & Job Runners), 10Release-Engineering-Team: Gitlab CI failure: ContainersNotReady: "containers with unready status: [build helper istio-proxy]" - https://phabricator.wikimedia.org/T333047 (10dancy) >>! In T333047#8729219, @brennen wrote: > @dduvall, @dancy I feel like this might duplicate an exist... [15:00:39] 10GitLab (CI & Job Runners), 10Release-Engineering-Team: Gitlab CI failure: ContainersNotReady: "containers with unready status: [build helper istio-proxy]" - https://phabricator.wikimedia.org/T333047 (10dancy) p:05Triage→03Medium a:03dancy [15:54:02] 10Release-Engineering-Team (Seen), 10MW-on-K8s, 10SRE, 10Traffic, and 3 others: Migrate internal traffic to k8s - https://phabricator.wikimedia.org/T333120 (10Clement_Goubert) [16:01:51] 10Phabricator, 10Release-Engineering-Team, 10Security-Team: Make it more difficult to unintentionally paste secrets, passwords, and keys in Phabricator task descriptions, comments and pastes - https://phabricator.wikimedia.org/T332639 (10sbassett) It might not be terrible to port Whispers' [[ https://github.... [16:20:14] 10Phabricator, 10Release-Engineering-Team, 10SecTeam-Processed: Make it more difficult to unintentionally paste secrets, passwords, and keys in Phabricator task descriptions, comments and pastes - https://phabricator.wikimedia.org/T332639 (10sbassett) [16:23:13] 10GitLab (Infrastructure), 10serviceops-collab: Investigate incremental backups for GitLab - https://phabricator.wikimedia.org/T324506 (10Arnoldokoth) @Jelto I tested this on gitlab1004. ` aokoth@gitlab1004:~$ time sudo /usr/bin/gitlab-backup create CRON=1 STRATEGY=copy GZIP_RSYNCABLE=yes \ SKIP=db,uploads,bu... [16:34:54] 10GitLab (CI & Job Runners), 10Release-Engineering-Team: Gitlab CI failure: image pull failed: Back-off pulling image "docker-hub-mirror.staging.cloud.releng.team/library/rust:latest" - https://phabricator.wikimedia.org/T333047 (10dduvall) [16:56:36] 10Release-Engineering-Team (Priority Backlog 📥), 10Release, 10Train Deployments: 1.41.0-wmf.3 deployment blockers - https://phabricator.wikimedia.org/T330209 (10Jdlrobson) [17:16:53] 10GitLab (CI & Job Runners), 10Release-Engineering-Team: Gitlab CI failure: image pull failed: Back-off pulling image "docker-hub-mirror.staging.cloud.releng.team/library/rust:latest" - https://phabricator.wikimedia.org/T333047 (10dancy) 05Open→03Resolved Cleaned up. Should be working now. [17:30:54] 10Phabricator: Uninstall Differential - https://phabricator.wikimedia.org/T330797 (10valerio.bozzolan) 05Stalled→03Declined I landed here again and I admit that this makes so sad, frustrated and confused. Apologies for my emotional context but I see this Task like, someone wants to shoot a minority, because... [17:51:02] 10Phabricator: Uninstall Differential - https://phabricator.wikimedia.org/T330797 (10Dzahn) fyi, regardless of how one feels about differential, there are already no more active repositories on Phabricator. And just in general we don't think it's good to have too many different places for storing git repos. If... [17:51:28] 10Phabricator: Uninstall Differential - https://phabricator.wikimedia.org/T330797 (10Aklapper) 05Declined→03Open @valerio.bozzolan: For which code repositories hosted on Wikimedia Phabricator do you use currently Differential for code review? I may sound like a broken record: We shall not maintain, host, an... [18:29:44] 10Phabricator: Less restrictive l image policy for drag and drop images for users or anons - https://phabricator.wikimedia.org/T333213 (10Jdlrobson) [18:30:34] 10Phabricator: Less restrictive l image policy for drag and drop images for users or anons - https://phabricator.wikimedia.org/T333213 (10Jdlrobson) [21:32:30] 10Continuous-Integration-Infrastructure, 10serviceops-collab, 10Patch-For-Review: Migrate doc hosts to Bullseye - https://phabricator.wikimedia.org/T319477 (10andrea.denisse) [21:34:02] 10Continuous-Integration-Infrastructure, 10serviceops-collab, 10Patch-For-Review: Migrate doc hosts to Bullseye - https://phabricator.wikimedia.org/T319477 (10andrea.denisse) [21:35:24] 10Continuous-Integration-Infrastructure, 10serviceops-collab, 10Patch-For-Review: Migrate doc hosts to Bullseye - https://phabricator.wikimedia.org/T319477 (10andrea.denisse) [21:35:36] 10Scap: scap backport applied +2 to wrong changeset when run shortly after a rebase and hung waiting for merge - https://phabricator.wikimedia.org/T333241 (10SDunlap) [21:39:41] 10Scap: scap backport applied +2 to wrong changeset when run shortly after a rebase and hung waiting for merge - https://phabricator.wikimedia.org/T333241 (10taavi) I suspect the order of events here might actually be # Run scap backport, which queries the patchset number # Rebase # Approve the backport, which t... [21:40:26] this needs update: https://wikitech.wikimedia.org/wiki/Deployments/Training#Useful_information :D [21:43:55] or a redirect to https://wikitech.wikimedia.org/wiki/Backport_windows/Deployers [21:51:40] 10Continuous-Integration-Infrastructure, 10serviceops-collab, 10Patch-For-Review: Migrate doc hosts to Bullseye - https://phabricator.wikimedia.org/T319477 (10andrea.denisse) [22:12:04] 10GitLab (Administration, Settings & Policy), 10Release-Engineering-Team (Priority Backlog 📥), 10Privacy Engineering, 10Product-Analytics, and 2 others: Request for Private repos to be enabled - https://phabricator.wikimedia.org/T305082 (10thcipriani) There are a lot of good use cases for private repositor... [22:12:15] Project mediawiki-core-doxygen-docker build #42088: 04FAILURE in 7 min 49 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/42088/ [23:11:58] Project mediawiki-core-doxygen-docker build #42089: 04STILL FAILING in 7 min 32 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/42089/