[10:36:54] 10GitLab, 10Release-Engineering-Team, 10serviceops-collab: Align the GitLab runner tags - https://phabricator.wikimedia.org/T325069 (10Jelto) Shared and Trusted Runners have a additional tag `wmcs` and `trusted` now. The next step is to create MR to move from `protected` tag to `trusted` tag and adjust the [... [10:37:50] 10GitLab, 10Release-Engineering-Team, 10serviceops-collab: Align the GitLab runner tags - https://phabricator.wikimedia.org/T325069 (10Jelto) [11:33:12] 10GitLab, 10serviceops-collab: Optimize Gitlab Backups - https://phabricator.wikimedia.org/T324506 (10jcrespo) Alternatively, as discussed in the past, one thing that we could do is incremental backups in a raw way- if we do a filesystem snapshot (e.g. with LVM) and perform hourly incrementals, we could go bac... [13:41:21] kudos for the GitLab -> phabricator integration. I just noticed a comment in phab task from a GitLab merge request :) [14:23:14] 10GitLab, 10Release-Engineering-Team, 10serviceops-collab: Align the GitLab runner tags - https://phabricator.wikimedia.org/T325069 (10Jelto) >>! In T325069#8468648, @bd808 wrote: > I know that I am a leading contributor to this problem, but "cloud" is pretty closely tied to [[https://wikitech.wikimedia.org/... [15:53:18] Where are we at with official package builds / uploads to apt.wm.org from gitlab CI? I have built rclone myself (in a chroot) and can just upload that myself, but I do now have it building for bullseye-wikimedia [I want the version from bookworm available on our bullseye systems] in CI - https://gitlab.wikimedia.org/repos/data_persistence/rclone/-/blob/bullseye-wikimedia/debian/.wmf-gitlab-ci.yml obviously at some point it might be nice [15:53:18] to use these artifacts instead of relying on my laptops' chroots :) [15:54:09] example build log is https://gitlab.wikimedia.org/repos/data_persistence/rclone/-/jobs/41909 [16:02:28] [happy to email someone / update phab or whatever, just not sure where we are, what/when next steps might be] [16:06:40] Emperor: a bunch of things would nee to be automated. I've been interested in the past, but never got time to look at this properly [16:06:42] for example: T215812 [16:06:42] T215812: reprepro: automate incoming processing - https://phabricator.wikimedia.org/T215812 [18:53:32] 10GitLab, 10Release-Engineering-Team, 10serviceops-collab: Align the GitLab runner tags - https://phabricator.wikimedia.org/T325069 (10dduvall) >>! In T325069#8470932, @Jelto wrote: >>>! In T325069#8468648, @bd808 wrote: >> I know that I am a leading contributor to this problem, but "cloud" is pretty closely... [18:56:02] 10GitLab, 10Release-Engineering-Team, 10serviceops-collab: Align the GitLab runner tags - https://phabricator.wikimedia.org/T325069 (10dduvall) One more thought: Since the DigitalOcean runners will be instance wide starting in the new year and will be configured to run untagged jobs, I'm not sure their tags... [19:06:51] 10GitLab (CI & Job Runners), 10serviceops-collab, 10Release-Engineering-Team (Priority Backlog 📥): Move cloud runner CI jobs to trusted runners - https://phabricator.wikimedia.org/T322344 (10dduvall) We're now seeing errors during our [[ https://gitlab.wikimedia.org/repos/releng/gitlab-terraform-images/-/job... [19:08:14] 10GitLab (CI & Job Runners), 10serviceops-collab, 10Release-Engineering-Team (Priority Backlog 📥): Move cloud runner CI jobs to trusted runners - https://phabricator.wikimedia.org/T322344 (10dduvall) I'm thinking the best course for us at this point might be to rely on binaries verified by checksum for now a... [19:08:38] 10GitLab, 10Release-Engineering-Team, 10serviceops-collab: Align the GitLab runner tags - https://phabricator.wikimedia.org/T325069 (10brennen) > maybe let's be explicit using domain names and stop saying just "cloud" I think this or something like it is probably the best solution for @bd808's original conc... [19:10:39] what's going on here https://gitlab.wikimedia.org/repos/cloud/cloud-vps/terraform-cloudvps/-/jobs/42073? [19:11:14] That happened to me too today. When I re-ran the pipeline it worked.