[00:14:01] how often should I expect to get logged out of GitLab? [00:37:05] I think some of the runners don't have enough free space? runner-1026.gitlab-runners.eqiad1.wikimedia.cloud - https://gitlab.wikimedia.org/repos/mwbot-rs/mwbot/-/jobs/44689 ran out of disk space [00:37:58] https://gitlab.wikimedia.org/repos/mwbot-rs/mwbot/-/jobs/44688 on the same host, probably running concurrently failed too [00:46:35] 10GitLab: runner-1026.gitlab-runners.eqiad1.wikimedia.cloud ran out of disk space - https://phabricator.wikimedia.org/T326378 (10Legoktm) [00:58:47] 10GitLab: runner-1026.gitlab-runners.eqiad1.wikimedia.cloud ran out of disk space - https://phabricator.wikimedia.org/T326378 (10Legoktm) [01:03:15] still failing on retry, will leave it for tomorrow [10:09:34] 10GitLab (CI & Job Runners), 10serviceops-collab, 10Patch-For-Review: runner-1026.gitlab-runners.eqiad1.wikimedia.cloud ran out of disk space - https://phabricator.wikimedia.org/T326378 (10Jelto) Thanks for opening the task. `runner-1026` should be unblocked because the daily cleanup happened some hours afte... [16:26:44] are gitlab CI/CD schedule triggers disabled on our install? [16:27:20] oh no, im just lookin in the wrong CI/CD menu.... [18:25:37] 10GitLab (CI & Job Runners), 10serviceops-collab, 10Patch-For-Review: runner-1026.gitlab-runners.eqiad1.wikimedia.cloud ran out of disk space - https://phabricator.wikimedia.org/T326378 (10Dzahn) on runnner-1026: ` +ExecStart=/usr/bin/docker run --rm -v /var/run/docker.sock:/var/run/docker.sock... [18:59:17] 10GitLab, 10serviceops, 10serviceops-collab, 10Kubernetes: Trusted gitlab runner containers need access to staging k8s cluster - https://phabricator.wikimedia.org/T325385 (10dduvall) (Orthogonal but worth a discussion in our next meeting.) Perhaps this is a good time to start looking at something like [[... [19:42:01] 10GitLab (CI & Job Runners), 10serviceops-collab, 10Release-Engineering-Team (GitLab III: GitLab in LA 🪃), 10User-brennen: Provision untrusted instance-wide GitLab job runners to handle user-level projects and merge requests from forks - https://phabricator.wikimedia.org/T297426 (10dduvall) [19:42:20] 10GitLab (CI & Job Runners), 10serviceops-collab, 10Patch-For-Review, 10Release-Engineering-Team (Priority Backlog 📥): Move cloud runner CI jobs to trusted runners - https://phabricator.wikimedia.org/T322344 (10dduvall) 05Open→03Resolved a:03dduvall >>! In T322344#8484468, @MoritzMuehlenhoff wrote: >...