[10:58:45] 10GitLab (CI & Job Runners), 10Security-Team, 10serviceops, 10Patch-For-Review, and 2 others: Setup GitLab Runner in trusted environment - https://phabricator.wikimedia.org/T295481 (10Jelto) [14:19:37] 10GitLab (CI & Job Runners), 10Security-Team, 10serviceops, 10Patch-For-Review, and 2 others: Setup GitLab Runner in trusted environment - https://phabricator.wikimedia.org/T295481 (10Jelto) [15:52:36] 10GitLab (Project Migration), 10Product-Analytics, 10Release-Engineering-Team (Doing): Create new GitLab people & project groups for Product Analytics - https://phabricator.wikimedia.org/T302727 (10mpopov) Thank you very much @brennen! [22:31:06] 10GitLab (CI & Job Runners), 10Security Team AppSec, 10Security-Team, 10Security: Create initial proof of concept application security pipeline repository - https://phabricator.wikimedia.org/T289293 (10sbassett) [22:32:00] 10GitLab (CI & Job Runners), 10Security Team AppSec, 10Security-Team, 10Security: Create initial proof of concept application security pipeline repository - https://phabricator.wikimedia.org/T289293 (10sbassett) [22:34:17] 10GitLab (CI & Job Runners), 10Security Team AppSec, 10Security-Team, 10Security: Create initial proof of concept application security pipeline repository - https://phabricator.wikimedia.org/T289293 (10sbassett) [22:37:47] 10GitLab (CI & Job Runners), 10Security Team AppSec, 10Security-Team, 10Security: Merge templates that are part of initial release to production branch - https://phabricator.wikimedia.org/T303482 (10sbassett) [23:00:41] Is there a way to run PipelineLib things from GitLab yet? I haven't found any obvious docs but semi-surprisingly I also haven't found a ticket to follow on making it possible. [23:01:33] ah. T287211 [23:01:33] T287211: Figure out the future of (or replacements for) PipelineLib in a GitLab world - https://phabricator.wikimedia.org/T287211 [23:11:20] bd808: yeah, also dan demoed some work this morning on T301169 [23:11:21] T301169: Implement buildkit frontend support in blubber for use on GitLab runners - https://phabricator.wikimedia.org/T301169 [23:12:14] ah, nice. I wondered why the buildkit stuff was being dusted off after so long. :) [23:19:49] o/ [23:20:14] heya dduvall, was just chatting a bit about pipelinelib stuff in gitlab-land and realized you weren't in here. [23:21:11] * dduvall nods [23:21:14] ( backscroll: https://wm-bot.wmcloud.org/logs/%23wikimedia-gitlab/20220309.txt ) [23:22:07] oh, bd808. figures. always trying to do cool things [23:22:32] :) I get to build green field things, so I poke at edges [23:23:10] lucky! seriously though, i always appreciate the edge poking [23:23:32] re: pipelinelib, the short of it is that migration of pipelinelib to gitlab is a bit of an unknown right now [23:23:41] and there are a few possible routes to take [23:24:25] I found the buildpack investigation ticket too [23:25:06] for me, blubber turns out to be more fun than I had imagined it would be and in ways I'm not sure that buildpacks would easily replace [23:25:08] we could lean almost entirely on gitlab's workflow stuff, but there's the question of dev/operator image conformance [23:25:14] yeah, buildpacks could help us there [23:25:30] re: blubber, interesting. say more [23:26:04] Mostly it is the pattern that I have invented for using blubber + docker-compose to make a loca dev environment [23:26:46] Off the top of my head I'm not sure that buildpacks would have a similar workflow possibility [23:27:33] the frontend pattern that buildkit has introduced, and corresponding `syntax` feature of docker, is pretty powerful [23:27:35] I mean yes a buildpack stack should be able to run locally, but I don't know if it would be simple to integrate with docker-compose [23:28:16] and the fact that buildkit frontends can Just Work with `docker build` and related toolchains makes it an attractive route to take [23:28:24] blubber + docker-compose is basically just a curl call to blubberoid and then the normal docker builder [23:28:38] (i.e. continuing to improve blubber and release it exclusively as a buildkit/docker frontend) [23:29:31] well, if https://gerrit.wikimedia.org/r/c/blubber/+/504651 lands, you should be able to just do `dockerfile: blubber.yaml` in your compose file [23:29:54] so long as blubber.yaml has `# syntax=docker-registry.wikimedia.org/wikimedia/blubber-buildkit` at the top [23:30:03] (that ref doesn't exist yet) [23:30:13] how would that pick the variant? [23:30:38] you have to specify the variant as the `target` (i.e. `docker build --target foo -f blubber.yaml .`) [23:30:47] is that in docker compose? [23:30:56] ah, it is [23:31:01] `target: foo` [23:31:36] nice! [23:37:30] I guess part of the pipelinelib question is how much of it is really recreating the .gitlab-ci.yaml system. [23:38:47] that's what i'm trying to figure out right now. i'm trying to migrate the blubber project over to gitlab so i can start poking at `.gitlab-ci.yaml` and see how much of pipelinelib's functionality it provides [23:38:54] my guess is most of it [23:39:46] the big question for me is: can we trust an image generated via a gitlib pipeline enough that we let it be published to our registry? [23:40:14] and how do we provide for publishing without exposing the registry credentials? [23:40:51] high quality questions [23:40:55] or do we need to rethink our registry authn/authz to accommodate gitlab (ditch the single cred approach and move to auth-ing individual groups) [23:42:01] there are other shared secrets in jenkins too (like the github creds for composer pulls) [23:42:39] so there probably is a reason to think about how that could work generally (shared creds exposed to CI) [23:43:11] * dduvall nods [23:47:33] "Use Vault secrets in a CI job (premium)" barf barf barf [23:47:43] https://docs.gitlab.com/ee/ci/secrets/#use-vault-secrets-in-a-ci-job [23:49:48] :| [23:51:20] so long as we use a shared credential for registry pushes, we can't expose it to a job, regardless of whether it's kept secure between jobs [23:51:43] maybe we just need to rethink the authz model we're using for the registry