[09:02:47] can I avoid kokkuri and blubber? [10:24:25] what would you replace them with? [12:06:36] the same Containerfile/Dockerfile I'm using to build the image locally? [12:07:21] alternatively, can I run kokkuri or blubber locally? [12:26:38] you can use blubber locally via docker build: https://doc.wikimedia.org/releng/blubber/#usage [12:36:23] I use podman locally [12:42:20] if I have to keep using kokkuri and blubber I'd rather have a setup that reproduces the gitlab CI runner environment. Can I pull docker-registry.wikimedia.org/repos/releng/kokkuri:v2.4.0 and run kokkuri image build ... inside? [13:49:10] the reason that the Containerfile/Dockerfile approach isn't supported is because frankly, Dockerfiles are a mess. They make it extremely easy to create an insecure, inefficient, non deterministic layer of containers. I 've been writing Dockerfiles for years and if you ask me know to write one, it will not be a good one for sure. There are so many [13:49:10] ways to cut yourself writing them. Blubber was build very early on when adopting Kubernetes and containers as a way to codify and apply a number of sane rules so that developers could avoid having to reinvent them one by one after having long discussions with SRE. At the beginning it was nothing more than a transpilation from YAML to Dockerfile [13:49:10] syntax, now it is a buildkit frontend. [13:50:14] that's for the context. I don't think we tried to see how all of this would work with podman. You can definitely do it with docker [13:55:51] hmm look like it is possible, let me double check on something [13:59:58] federico3: If you're having trouble with Kokkuri or Blubber in general, #wikimedia-releng on IRC or #developer-experience on Slack are good places to ask for help. [13:59:58] If I wanted to build your container image locally, I would do: [13:59:59] `docker build -f .pipeline/blubber.yaml --target production .` [13:59:59] I assume podman has an equivalent invocation. [14:01:36] dancy: podman doesn't have buildkit capabilities built-in so it doesn't understand blubber's (or any of the other buildkit frontends, except Dockerfiles) syntax [14:01:44] so that -f .pipeline/blubber.yaml which just error out [14:01:48] will* [14:01:59] Bummer. [14:02:14] which is kinda expected, we 've never stated any goal of supporting podman [14:02:20] Nod. [14:03:04] doesn't podman have some buildkit capabilities like secrets and cached mounts? [14:03:36] yes, but IIRC they are directly implemented, not abstracted via some intermediary language. [14:06:15] I'd rather reproduce the same setup that exists inside the gitlab CI runner to avoid more glitches so I'd like to run the same container that gitlab CI runs [14:07:04] e.g. if I start docker-registry.wikimedia.org/repos/releng/kokkuri:v2.4.0 and run kokkuri image build ... inside so that it runs kokkuri in a controlled environment... would that be ok? [14:07:24] If you do that, kokkuri will eventually try to run buildctl to contact a buildkitd daemon which you are not running. [14:07:51] If you're willing to fire up a working buildkitd, you might as well install Docker. [14:07:58] which can be run though under podman [14:08:18] podman run --rm --name buildkitd docker.io/moby/buildkit:latest [14:08:38] and I see I have a buildkitd running, or so at least buildkitd's logs try to convince that it happens [14:08:44] hmm.. yeah that would work too. Needs --privileged mode. [14:08:53] now ofc the difficult part is having the 2 things talk to each other [14:10:04] ah yes, otherwise it doesn't have access to various things, e.g. readonly fs for /sys/fs/cgroup [14:10:44] Nod. Anyway, none of that is simpler than "docker build -f ...." [14:11:05] yup [14:11:57] What I can say (I also run podman) is that if you are on the move a lot, running dockerd (and thus containerd) can be more taxing than needed on the battery [14:12:15] which was my personal drive for this. Others are very happy with rootless containers [14:12:58] sigh. Alternatively, can the CI print out the generated Dockerfile/Containerfile perhaps? [14:13:17] It does not generate a Dockerfile. [14:13:18] there isn't one, that's what I 've been trying above to explain. [14:13:33] what you might be able to get is a dump of the LLB DAG [14:13:40] but that is probably useless? [14:14:26] federico3: I'm happy to work with you to help you wrap up whatever you're trying to do. I just need to know what problem you're running into. [14:16:32] I just wanted to build a pretty simple container for https://gitlab.wikimedia.org/repos/data_persistence/zarcillo/-/jobs ideally with the ability to generate the same locally for development and testing [14:17:25] (for context I was using https://gitlab.wikimedia.org/repos/data_persistence/zarcillo/-/blob/devel/zarcillo/Containerfile?ref_type=heads with podman for local tests) [14:18:10] federico3: if you don't want to install docker locally, maybe you could use build2001.codfw.wmnet ? [14:18:54] I'll install docker locally or in a vm... [14:19:36] that's probably the easiest and faster way out. You miss the chance of a very deep rabbithole though :P [14:20:27] jokes aside, there are so many times I wish for podman to be closer to docker's functionalities. But that's mostly a fools errand I fear. [14:20:43] ^ [14:21:05] e.g. if you try and use devcontainers and try to use one of the "features", podman, at least up to 4.9.3 (which is what I got) will error out with some very cryptic messages [14:22:09] and I 've tested up to 5.4 IIRC. Makes it a bit harder to work with podman and devcontainers. It used to be impossible 2 years ago.