[07:59:09] morning [07:59:33] o/ [08:00:04] morning [08:06:51] morning [08:31:04] o/ [09:11:03] dhinus: I added a very speculative example of user flow here https://phabricator.wikimedia.org/T362051, let me know if it helps, otherwise I'll remove it [09:11:51] awesome thanks! that's exactly what I had in mind :) [09:24:06] I'm thinking on how to allow users to authenticate as themselves, and manage tools without having to `become` and use the tool certificate, I know we have idp.wmcloud.org, would that be a good alternative to using the certificates? [09:24:37] (we would have to create endpoints like `/tool/mytool/*` on all the apis though) [09:26:14] I don't think I have enough knowledge about the whole authz/authn field to have a strong opinion [09:28:32] maybe bitu will be replacing it? (I have some experience with keycloak, and we have keystone too, though for toolforge we only need ldap auth + ldap groups) [09:29:54] keycloack had several sessions and mentions in the latest kubecon EU [09:30:12] it's kind of common in k8s environments yes [09:58:36] looking for a review of https://gerrit.wikimedia.org/r/c/operations/puppet/+/1011174 (the pcc looks a bit weird as the hiera was already moved to the new keys, so the old secret_env is empty) [10:08:52] taavi: LGTM [10:16:14] re: managing tools from the CLI, I've seen several commercial products where the CLI opens a web browser, lets you authenticate from the web (in our case it would be at idp.wmcloud.org) then stores the auth token for API requests [10:16:56] an alternative to modifying the API endpoints could be to create a new endpoint /tool/token that lets you generate a separate token for a specific tool (if your user token matches the admin list for that tool) [10:16:57] that's from the user's laptop right? [10:17:11] yes [10:17:57] bitu as far as I know will not replace Apereo CAS (which is what powers idp.wmcloud.org). there's also idp.wikimedia.org seems to be the same? [10:18:26] bitu is just a layer on top to manage users and other things [10:18:33] ack [10:18:54] correct. and idp.wmcloud.org delegates the actual authentication to idp.wikimedia.org, the .wmcloud.org one exists purely to avoid having to configure all the wmcloud services on the main idp also used for wikiprod services [10:24:56] it seems then that idp.wmcloud.org might be the way to go then, will try to give it a look, if it does not seem too complicated, we might want to do it sooner than later [10:25:10] hmm, for some reason inccloud told me I was offline for a bit [10:50:49] patch to set up a new striker instance for toolsbeta: https://gerrit.wikimedia.org/r/c/operations/puppet/+/1018664 [12:05:14] for some reason our LDAP directory has an account named 'cn=toolsbeta.toolsbeta.test5,ou=servicegroups,dc=wikimedia,dc=org' :/ (note the double 'toolsbeta.') [12:05:57] doesn't ring any bell, maybe you can just delete it? [12:07:32] i'm trying to rename it to the normal version, but running ldapvi with --rename results in a segfault [12:11:07] sounds like a copy-paste gone wrong or using some variable like `toosbeta.$ACCOUNTNAME` kind of thing and the variable having already the prefix [12:11:52] yeah [12:12:11] I don't see that account in use (it doesn't have a home directory), so I will just try to delete it [12:22:44] moritzm: remind me how instant is LDAP replication supposed to be? I just deleted two stale broken tool account entries via ldapvi, they disappeared from seaborgium and serpens instantly but none of the replicas seem to have caught up on that [12:34:49] hmmh, I think it attempts to resync every minute? [14:15:34] quick review? https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-cli/-/merge_requests/26 Raymond_Ndibe maybe? [14:20:39] dcaro: LGTM [15:18:07] taavi: metricsinfra-meta-monitor-1 says 'Could not find class role::wmcs::metricsinfra::meta_monitor for metricsinfra-meta-monitor-1.metricsinfra.eqiad1.wikimedia.cloud' -- is that you? [15:18:32] andrewbogott: that class should currently be locally cherry-picked on the puppetserver [15:18:48] hm, that's concerning [15:19:55] yeah, the git sync misfired somehow, those patches are still there but on a set aside branch [15:19:59] I'll figure it out [15:24:14] I set up a k8s-status instance for toolsbeta -- https://k8s-status.beta.toolforge.org/ [15:27:47] that reminds me, what should I do if I want to transfer the `toolsbeta.org` to the foundation? [15:28:06] anyone knows/ [15:28:07] ? [15:29:05] I'd start by asking dzahn/mutante or robh [15:34:39] Feel free to add any details/ask stuff :) T362253 [15:34:39] T362253: [toolforge] transfer/adopt toolsbeta.org domain to the foundation - https://phabricator.wikimedia.org/T362253 [15:38:26] taavi: it's so fast again with hardly any namespaces to crawl. :) [15:39:05] the main one being horribly slow is a good sign that redis caching doesn't actually work well anymore [15:45:35] minifix https://gitlab.wikimedia.org/toolforge-repos/k8s-status/-/merge_requests/3 [15:47:00] dcaro: is the double `&&` intentional? [15:47:09] oh no! [15:49:31] fixed :) [15:51:14] thanks, merged [15:51:35] thanks! [15:54:42] hmm, that grafana board is missing some data, looking [15:55:17] maybe not, the namespace just has no pods xd [15:57:34] * arturo offline [16:00:12] * dcaro off [16:00:18] cya tomorrow [16:29:53] taavi: I feel like this is the same issue we had last week, returning... [16:30:07] https://www.irccloud.com/pastebin/LqEX9IBl/ [16:31:53] CFP is open for KubeCon NA and Chris Aniszczyk (CNCF CTO) said he was interested in blog posts & keynotes about how we replaced grid engine with kubernetes over 8+ years. https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/program/cfp/ [16:32:12] dcaro, arturo, taavi: ^ [17:01:15] slyngs: which parts of T360795 would you like me to do vs. which are you planning to do? I opened a ganeti request ticket but haven't actually allocated the host yet. [17:01:16] T360795: Set up a bitu instance for codfw1dev - https://phabricator.wikimedia.org/T360795 [18:11:46] * bd808 lunch