[06:13:35] morning [08:43:45] T395542 [08:43:45] T395542: Keystone in Epoxy does not support shell names with underscores - https://phabricator.wikimedia.org/T395542 [09:02:12] there's enough users with those names that i suspect we'll need to patch the validation logic in keystone [09:04:23] ouch [10:14:17] * taavi afk for a bit [11:44:42] taavi: I will start working on T395542 if you are not already [11:44:43] T395542: Keystone in Epoxy does not support shell names with underscores - https://phabricator.wikimedia.org/T395542 [11:55:46] andrewbogott: please do [12:37:37] * andrewbogott is giving upstream devs the business but that is not actually going to help in the short term [14:05:25] dhinus I have an MR that's ready to be looked at: https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-api/-/merge_requests/135 [14:06:56] chuckonwu: looking! [14:37:43] hmm I'm trying to test that MR in lima-kilo but my lima-kilo is kaput. restarting the VM made it even worse :D [14:39:04] debugging what's the problem seems a big rabbit hole so I'll just rebuild it [16:09:56] andrewbogott: at the risk of bring back horrible memories, in today's zuul workgroup meeting we talked a bit about the problems we saw with nodepool in the past. We have basically 3 options for exec node management in the next zuul build: static pool, nodepool, and kubernetes. [16:09:58] There is some concern that static pool instances might be corruptible by untrusted jobs. The working group may come asking soon for permission to test nodepool-like workload pressure to see if things are different 6-7 years after we last shut that down. I think we decided to test k8s first though. [16:10:30] Oh yeah, I talked to hashar about nodepool a bit. [16:10:59] I would be willing to give nodepool another chance, but I thought they were talking about having nodepool manage containers within a k8s cluster? [16:11:06] James Blair did acknowledge that nodepool is a stress test for the public clouds it runs against, so not 100% our issue in in the past either. [16:11:07] Or otherwise manage containers rather than full VMs [16:12:31] Mostly it seems like if zuul can do its thing with containers then the overhead of full VMs is surely not worth the trouble. [16:13:04] I was in the meeting but haven't caught up on all the irc discussion that I missed while being out earlier this week. I think just yesterday folks got to a point of wondering how robust the containment of bubblewrapp + docker on static nodes will be. [16:13:59] It does seem like Kubernetes is a better long term target [16:16:21] boy, I would really hope that we can trust the containment of a docker container! [16:22:00] When the theoretical attack discussions start there is often reasonably wide speculation about threat vectors during the gathering phase. :) [16:25:14] heh, yeah [17:29:04] * dhinus offline and out tomorrow + Monday, back Tuesday!