[17:15:08] 10GitLab (CI & Job Runners), 10Security Team AppSec, 10Security-Team, 10Security, 10user-sbassett: Support conda environments with a new gitlab appsec ci template - https://phabricator.wikimedia.org/T305732 (10sbassett) I need to review the [[ https://ossindex.sonatype.org/tos | OSSI TOS ]] with WMF Lega... [18:09:32] 10GitLab (Infrastructure), 10DC-Ops, 10SRE, 10ops-eqiad, 10serviceops: Q3:(Need By: TBD) rack/setup/install gitlab100[3|4] and gitlab-runner100[2|3|4] - https://phabricator.wikimedia.org/T301177 (10Jclark-ctr) Name Rack U Port Cableid gitlab1003... [18:16:27] 10GitLab (Infrastructure), 10DC-Ops, 10SRE, 10ops-eqiad, 10serviceops: Q3:(Need By: TBD) rack/setup/install gitlab100[3|4] and gitlab-runner100[2|3|4] - https://phabricator.wikimedia.org/T301177 (10Jclark-ctr) [18:20:06] 10GitLab (Infrastructure), 10DC-Ops, 10SRE, 10ops-eqiad, 10serviceops: Q3:(Need By: TBD) rack/setup/install gitlab100[3|4] and gitlab-runner100[2|3|4] - https://phabricator.wikimedia.org/T301177 (10Jclark-ctr) a:05Jclark-ctr→03Cmjohnson [18:51:14] 10GitLab (Infrastructure), 10DC-Ops, 10SRE, 10ops-eqiad, 10serviceops: Q3:(Need By: TBD) rack/setup/install gitlab100[3|4] and gitlab-runner100[2|3|4] - https://phabricator.wikimedia.org/T301177 (10Dzahn) It's sufficient if you put the "insetup" role on this and hand it over to us. Let us apply the actua... [19:03:01] the gitlab-prod-1001 instance in devtools is reporting as "failed puppet run" [19:03:09] but then when I try to ssh to it to check why [19:03:14] I can't even do that [19:03:19] 10GitLab (Administration, Settings & Policy), 10Horizon, 10SRE, 10wikitech.wikimedia.org, 10Security: Take some pointers from GitHub security updates - https://phabricator.wikimedia.org/T304231 (10brennen) [19:03:26] 10GitLab (Administration, Settings & Policy), 10cloud-services-team (Kanban): gitlab: consider enabling docker container registry - https://phabricator.wikimedia.org/T304845 (10brennen) [19:04:47] console shows nothing special. this is one of those things where stuff randomly stops working [19:05:02] hrm [19:09:04] it's back after soft rebooting the instance. uptime 1 minute [19:09:15] logged to https://phabricator.wikimedia.org/T297411 [19:09:35] Notice: /Stage[main]/Profile::Gitlab/Interface::Alias[gitlab service IP]/Interface::Ip[gitlab service IP ipv6]/Exec[ip addr add ::/128 preferred_lft 0 dev eth0]/returns: executed successfully (corrective) [19:09:39] this is very suspicious [19:10:57] well, maybe not. dunno [19:11:04] but normally IPv6 wasnt a thing in cloud [19:11:11] this is only loopback though [19:11:59] 10GitLab (Infrastructure), 10serviceops, 10Patch-For-Review: Migrate gitlab-test instance to puppet - https://phabricator.wikimedia.org/T297411 (10Dzahn) running puppet after it came back showed: ` Notice: /Stage[main]/Profile::Gitlab/Interface::Alias[gitlab service IP]/Interface::Ip[gitlab service IP ipv6... [19:13:03] just managed to login after updating bastion host key [19:13:09] 10GitLab (CI & Job Runners), 10serviceops: upgrade gitlab-runners to bullseye - https://phabricator.wikimedia.org/T297659 (10Dzahn) @Jelto All the (non-protected) prod runners are upgraded. Now I was just wondering about the 2 protected runners. They are paused. Should I try upgrading those as well? [19:13:31] brennen: yep, now it seems normal again [19:14:27] brennen: looking at https://gitlab.wikimedia.org/admin/runners are there any runners that are offline/stalled/paused that you once used for testing but could be deleted? [19:14:40] just wanted to clean up a bit if it makes sense [19:14:51] notice there are 2 pages and the oldest ones are on page 2 [19:22:02] lemme look [19:22:18] i don't _think_ i have anything laying around, but i'll check [19:26:12] thanks [19:27:21] mutante: good call, i did go ahead and find a number of stale ones it looks safe to remove [19:27:33] (got rid of those) [19:27:42] great! I wasn't sure if I should. thank you