[10:58:12] I have a question about namespace-level access in k8s. If I understand correctly, helmfile.d/admin_ng/helmfile_rbac.yaml together with e.g. helmfile.d/admin_ng/values/ml-serve.yaml grants extra privileges to kserve, i.e. that it can manipulate pods etc. in all namespaces of that cluster. [10:59:25] Now in the context of T354516, I want something similar: the members of our team that aren't SREs (and thus can't use `kube_env admin ...`) should have access to do admin-level stuff with things in the experimental namespace of the staging cluster. [11:00:19] Is that the right approach? What else might be needed on the deployment server itself? Obviously kube_env doesn't know about namespaces. [11:08:08] You understanding of the structure is correct, but please define what "admin-level" stuff means. Kubernetes's RBAC system is very very granular, so technically we aren't constrained by anything. [11:08:54] e.g. you can even create a namespace and give an account the ability to get create/delete (but e.g. not edit) pods/services/ingresses etc [11:21:17] Basically stuff like taking down pods, attaching to them etc. So mostly affecting objects inside that namespace, but e.g. not other users. [11:22:39] I do understand that there are all manners of interesting isolation questions (like consuming resources, crashlooping hard etc) that make access to one namespace something that can affect other namespaces as well, but we'r enot talking adversarial use. [11:24:57] AIUI, the k8s rbac system thinks in terms of roles (people), verbs/actions and affected objects. Naturally just the latter two make for a wonderful combinatorial matrix. I mostly wanted to ask here about things I may have overlooked before diving into the sea of that matrix. [12:19:08] The usecase is to avoid the whole round trip through CI etc when trying to make something work. Sometimes, startup memory usage is a problem, and so being able to search for a RAM setting that works is much easier without the whole Gerrit dance. Similar for debugging by attaching to a service (network issues can be easier to debug by attaching and running a Python interpreter. [12:20:58] experimental-* file permissions are btw already deploy-ml-service [12:21:08] e.g. -rw-r----- 1 root deploy-ml-service 490 May 31 2023 experimental-ml-staging-codfw.config [12:21:18] so at least that part is solved ? [12:25:19] Yes, I wasn't sure about how kube_env worked, but I've now read it's rather short code :D [12:26:04] (I was thrown by _me_ not being a member of that group, but naturally I don't need to be) [12:26:28] I shall do some reading up on verbs and objects etc.