[16:48:32] we had a good gitlab sync meeting! [18:19:33] 10GitLab (Infrastructure), 10DC-Ops, 10SRE, 10ops-eqiad, 10serviceops: Q3:(Need By: TBD) rack/setup/install gitlab100[2|3] and gitlab-runner100[2|3|4] - https://phabricator.wikimedia.org/T301177 (10RobH) [18:21:41] 10GitLab (Infrastructure), 10DC-Ops, 10SRE, 10ops-codfw: Q3:(Need By: TBD) rack/setup/install gitlab200[2|3] and gitlab-runner200[2|3|4] - https://phabricator.wikimedia.org/T301183 (10RobH) [18:25:06] 10GitLab (Infrastructure), 10DC-Ops, 10SRE, 10ops-eqiad, 10serviceops: Q3:(Need By: TBD) rack/setup/install gitlab100[2|3] and gitlab-runner100[2|3|4] - https://phabricator.wikimedia.org/T301177 (10Dzahn) @RobH (cc: @Jelto ) gitlab1002 has existed as a VM in the past, when contractors used it but the... [18:35:57] 10GitLab (Infrastructure), 10DC-Ops, 10SRE, 10ops-eqiad, 10serviceops: Q3:(Need By: TBD) rack/setup/install gitlab100[2|3] and gitlab-runner100[2|3|4] - https://phabricator.wikimedia.org/T301177 (10RobH) a:05Jclark-ctr→03LSobanski @lsobanski: Is it ok to shift these hostnames from gitlab100[23] to gi... [19:40:50] hey guys, I was able to import a repo from phabricator to gitlab [19:40:57] https://gitlab.wikimedia.org/dzahn/tool-cr-grants-team-metasync [19:41:08] came from https://phabricator.wikimedia.org/source/tool-cr-grants-team-metasync/ [19:41:19] except that the user name in the URL should not be there [19:41:20] oh nice - builtin import tool supports phab, or just using the git remote? [19:41:37] it supports importing _tickets_ from Phab [19:41:44] but for repos it is just "any git URL" [19:41:55] and then using https, http wont be recognized because proto redirect [19:42:07] yeah, makes sense. i've used it a couple of times but didn't remember if it had any special support. [19:42:21] "repo by URL" [19:42:32] as far as the username, people with permissions on a group under /repos should be able to import to that namespace [19:42:39] that worked when combining it with all those URI features in phab [19:43:03] hmm, I should not have clicked "new project" ? [19:43:30] "New Project" -> Import project is what this was [19:43:49] it does not let me edit the project URL int he import dialog [19:45:14] i think if you're a member of other groups, it should allow you to pick one. [19:45:46] * brennen looks at repo [19:46:03] probably I am not in any groups yet [19:46:06] yeah [19:46:08] brb, food on the stove [19:47:37] I do see a choice of the groups I'm in or my own user space when using the import_project tool [19:49:20] i don't think we have a namespace for toolforge tools yet under https://gitlab.wikimedia.org/repos but ot [19:49:25] er, but it's pretty easy to define one. [19:49:44] (there's https://gitlab.wikimedia.org/repos/cloud/toolforge but that's internals / infra stuff i think.) [19:50:15] yeah, so far that is Toolforge infra stuff [19:51:15] wanna create "tools" ? [19:51:29] offhand thoughts on: /repos/tools, /repos/cloud/tools, /repos/toolforge ? [19:52:02] "tools" is way way generic :) [19:52:04] since they all start with "tool-" now, I would say /repos/tools/ and I drop the "tool-" prefix from the actual repo name [19:52:19] well, ok, or toolforge [19:52:41] also, I should delete that again from my personal namespace now [19:52:47] let's see if I can [19:53:09] it's also totally fine to have copies of things in your personal namespace; the forking model here will tend to encourage it [19:53:11] brennen: let's add naming to our discussion today. [19:53:16] bd808: yeah, sounds good. [19:54:25] (not to say people should leave tons of random things lying around, but it's expected that it'll happen.) [19:54:39] export / archive / transfer / delete .. I can do all of these. anything bad about _really deleting_ ? [19:54:48] nope [19:54:51] kk [19:55:26] nice, it has a "are you sure, type this stuff" sanity check [19:55:36] Project 'Dzahn / Tool Cr Grants Team Metasync' is in the process of being deleted. [19:56:37] I was about to put this in /repos and then tell the guys "here you go". Because that repo is the one where they actively said "you can move us any time, tell us when done" [19:56:48] but I will step back and wait for naming decision. [19:57:10] realizing it's just one of the striker tools like all the others [19:57:27] yeah, we can probably come up with something today and i'll create it after bd808 and i talk, but there might be reasons to wait until we migrate all striker repos. [19:57:49] ACK, makes a lot of sense [20:00:59] To not break clones of the repos on Toolforge I think I would like to see some thing like 1) import to gitlab, 2) disable push to diffusion repo, 3) switch diffusion repo to mirror from gitlab. [20:01:10] brennen: how exactly do we have ACL set up on repos/cloud? I am listed on https://gitlab.wikimedia.org/people/volunteer-group-cloud-admin but don't see any way to create new repos on repos/cloud for example [20:01:19] which hopefully is something we can script [20:01:50] also, remind me why we have separate groups for the cloud services team and volunteers with similar access on the cloud realm? [20:02:59] taavi: one sec and i'll look - from memory, i think we might need to make the volunteer group an owner there [20:03:00] I was already planning to do 2) after importing, dont know the 3) yet [20:03:12] https://phabricator.wikimedia.org/T293741#7515694 [20:03:25] one thing re: diffusion repos is if we're turning off ssh there, we're likely to break some clones regardless [20:03:44] brennen: agreed, but we can break fewer than all :) [20:04:04] taavi: re: separate groups, WMF team membership or the org chart or who knows what might change [20:04:39] we're modeling people that way to keep org structure type stuff one layer of indirection away from who is a member of what project. [20:05:58] the truth of the matter might be that this just creates more complexity than benefit and we can scrap it if we need to. i'm kind of on the fence about it. i hoped to make all of this more rational in the transition from gerrit, because gerrit ACLs and project layout are a giant confusing mess, but i guess honestly i'm starting to care less and less about that. [20:06:02] * bd808 has feelings about the "org" vs "volunteer" things [20:06:14] Mmm. [20:07:21] i have feelings too, but i don't want to manage a giant deluge of gitlab acl changes in the inevitable-on-a-long-enough-timeframe event of the wmf's reporting structure getting thrown in a blender. [20:08:00] volunteer groups aren't intended to model some form of 2nd-class citizenship; they're just different sets of people. [20:08:10] ok, question: where do people in other wmf teams (for example R.eedy) but who have admin rights on WMCS (i.e. are members of the "admin" Cloud VPS project) belong in? [20:08:33] my feelings are maybe different... I really dislike the idea that where you get you paycheck is strongly relevant to your rights in the FOSS community [20:08:41] when we documented this plan originally, we said that one-off memberships would be modeled by direct membership in the group or project. [20:09:47] it should not matter whether you get paid, is my feeling too [20:09:48] part of the question is are user groups supposed to be ACLs, or are they supposed to actually matter for organizational/process reasons? [20:10:45] i guess my thinking is that the organizational/process reasons they matter for are the ones where somebody has to manage onboarding for new team members. [20:10:48] the thing that we (or at least I) want is that people who have admin rights on WMCS (as defined by having access on openstack) have access to related GitLab repos, and having to split the people with pretty much exact same access in two groups isn't really giving us any value [20:12:02] yeah, that's fair. would it be better if there was something like just a self-managed people/team-cloud? [20:12:11] or, similarly, people with the ability to deploy code (as defined by being a member of some unix group) need to have similar access to deployment branches regardless of where they are employed [20:12:30] yea, wmf has trouble with orgcharts, but does it matter for your gitlab privileges who is reporting to who? maybe more like "you are in this group because you work on topic area X" [20:13:03] yeah, i guess at that point: _is_ there any gain from modeling groups of humans separately from groups of repos? [20:13:30] i _think_ upstream broadly recommends separating these concerns, although we got the idea from kde [20:14:01] ideally I'd just have a script syncing it from a single source of truth (as opposed to things like https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin#What_makes_a_root) [20:14:27] if groups of humans are defined by "working on the same thing" they seem to make way more sense than "volunteer/contractor/employee" [20:15:11] especially given how often WMF folks move between projects/groups but maintain (and use) access to previous projects [20:15:25] what WMF calls their teams ..we better don't even try to follow, imho [20:16:26] ^ [20:16:28] also Okta is killing corp-ldap (where we could check who is staff vs contractor) [20:16:58] s/killing/moving to a SaaS host/ I think? [20:17:15] I mean Okta is really a user directory of some sort [20:17:31] so i've already been getting kind of nervous about this scheme in practice, and i find the stuff mentioned here fairly compelling. [20:17:33] "maybe later but for now we still have 2 sources of truth, employee here, contractors there etc" [20:17:36] it's complicated [20:17:55] but it's not even going to be the one source of truth for now [20:18:01] still namely and the other etc [20:18:10] syncing from a single source of truth is... well, i'm skeptical it's going to happen, but it seems like we need to revisit the why/how of modeling things by team. [20:19:35] yes, the closest we have is corp-ldap, hence my concern that it's replaced with 2 [20:21:02] i think the main practical thing i want from modeling groups of people is to be able to say: you've just started work (be that on a wmf/wmde/whatever org team or in an area of volunteer concern), we put you in a group, now you have access to all the things you need. [20:21:11] well... we have the developer account + cloud LDAP still and that's actually where Gerrit and other things get membership today [20:21:20] bd808: that is only sort of true. [20:21:29] (as i understand it) [20:21:35] a few things in gerrit correspond to ldap group [20:21:40] the corp ldap is maybe used to guess about granting the wmf group [20:21:43] many... don't [20:21:59] i think the list is wmf/wmde/nda? [20:22:11] and everything else is modeled in gerrit [20:22:19] and so much "inherited from" in the privileges there [20:22:27] it is... gnarly. [20:22:39] you have way too much stuff with "wmf" probably [20:22:41] gnarly is what i want to avoid here. [20:23:13] https://gerrit.wikimedia.org/r/admin/groups/4cdcb3a1ef2e19d73bc9a97f1d0f109d2e0209cd,members -- "included groups: ldap/wmf" [20:23:15] yea, some LDAP and some not [20:24:15] I have +2 in many many repos because I'm in the wmf ldap group. Other folks get rights in other ways. :) [20:24:37] ah, yeah, and i was forgetting ldap/ops [20:24:38] brennen: I almost see an advantage in "they have to know/say which repos/projects/topic areas they want" vs "you are new, here is all the stuff" and dumping it on them [20:24:51] also that mediawiki members list is like a who's who of doesn't work here anymore [20:25:48] mutante: some advantage, but onboarding here is also kind of a hellscape of having to know what you should ask for, followed by figuring out who to ask for it [20:26:00] and the whole "asking for gerrit privileges" process has been redesigned and now there is an open ticket how to hand that over and whatnot..fwiw [20:26:23] so probably most of it is again happening by pinging on IRC "as needed" [20:26:29] which we once wanted to avoid [20:26:38] has it? [20:26:53] I do think modeling groups of people can be useful in some situations -- outside contractors needing access to something for a limited time, for example [20:26:57] another use case i have for people groups is just something like: there's a new area of work, i know that everybody in releng and serviceops needs access to it, or something along those lines. [20:28:05] brennen: but then maintain a list of "which wmf team owns which repo"? it seems like that ends in "move these repos around please" tickets [20:29:03] brennen: we can solve the "who and how to ask" and make it easy..with a form [20:29:36] but keeping a list of "all the needed things for each team" seems a lot [20:29:40] the idea was more like: these groups of people own a chunk of the namespace and mostly manage it themselves [20:29:53] maybe let the teams do the work and list the things they need on their own onboarding ticket template [20:30:02] doesnt mean the new person has to know it [20:30:07] just their onboarding buddy [20:31:39] this comes down to giving each repo an official owner [20:31:48] so far we have not even managed to do that with services [20:31:55] and with tickets we think it can be counterproductive [20:33:54] for example the puppet repo is used by all SRE subteams and cloud. You _could_ force it on infra-foundations to officially own it..yea.. but only advantage I see is if we don't want to handle the access requests for everything. yea, I see that part. [20:35:05] https://phabricator.wikimedia.org/tag/gerrit-privilege-requests/ https://www.mediawiki.org/wiki/Gerrit/Privilege_policy [20:37:09] I'm not sure if puppet is a good example.. you probably want everyone in the 'ops' unix group (but no-one else) to have merge access there, so you should automate that sync instead of forcing people through another process [20:40:28] maybe, though once we get into the general onboarding process and that gerrit privilege RFC I will refer to management and Tim [20:55:22] hey taavi - re: volunteer-group-cloud-admin specifically, somehow you were marked as "Guest" in that - which isn't even the default, i must just have mis-clicked when creating it. i also bumped that group from "maintainer" to "owner" on /repos/cloud. [20:55:46] (you're now an owner on the volunteer group.) [20:57:08] thanks! [20:57:25] yeah, sorry 'bout that [20:58:17] whether we re-think the current access scheme or not, or can use ldap for very much or not, automating this all and putting it in something declarative so manual clicking isn't a factor is probably going to be a good idea. [21:00:48] I think manual clicking is also ok, as long as we do the request itself on tickets and not by poking on IRC, fwiw. [21:40:03] if I look only at "active" and "hosted" repos..and then subtract all called tool-*, then subtract Phabricator itself and phabricator-translations.. and now releng-secrets as well.. there are really only like 2 things left. wow, nice. though of course.. whatever the definition of "active" is [21:40:49] sounds like "activeuser" in MW where people also keep asking [21:41:33] oh..and it's not true. it's just "Query Overheated" so it truncated results.. awwww [21:45:48] searching "-tool -extension" works a bit better [21:47:25] finds obscure repos but could be worse :) [22:08:06] and here is one more funny thing for today: [22:08:08] https://phabricator.wikimedia.org/diffusion/TSVN/ [22:08:20] this is an SVN repo on phabricator... no comment :p [22:08:25] laters [22:08:35] (no mercurial though :) [22:08:55] toolserver-svn, ahhaa [22:11:13] and we need to fix https://svn.wikimedia.org redirect .. omg [22:22:39] mutante: for rTSVN the reason seems to be https://phabricator.wikimedia.org/T60801 [22:29:46] which is archived at https://archive.org/details/toolserver-svn [22:30:51] hashar: ah, thank you! [22:31:29] "svn->git migration is not completely trivial, due to the free-form nature of SVN repos:" :p [22:31:38] quip [22:42:01] mutante: in phab terms, "active" === "not disabled" [22:42:36] taavi: ACK, makes sense. so nothing about how many commits. just disabled an empty but active one [22:44:44] I think everything that is active but has 0 commits I'll take the liberty to disable (but not delete) [22:44:57] overall this list isn't too bad [22:46:23] well, I would treat it different based on how long ago it was created, 2018 [22:46:33] "Hamishcn renamed this repository from tool-listunpatrolledpages to xtools-H." [22:46:47] well this explains why it was created by striker but did not look like it was [22:48:25] "Dangerous changes allowed: yes/no" is also such a lovely sounding thing [22:49:19] seems to mean "--force push" and delete branches