[09:05:05] morning! [09:08:17] I'm catching up on the last two weeks, let me know if there's anything specific I should focus on first [09:21:14] welcome back [10:45:51] arturo: fyi I am adding ospf to the cloudsw1-d5 <-> cloudsw2-d5 link [10:46:00] topranks: ack [10:46:07] the only thing it does is announce the loopback IP for our icinga checks, not expecting any issue [10:46:10] just fyi [10:52:07] ok [10:52:24] topranks: now that you mention icinga, this popped [10:52:25] https://usercontent.irccloud-cdn.com/file/BtDqRZbf/image.png [10:52:37] yeah that's fine, it'll disappear now [10:52:48] just when it was coming up one side before other had been added [10:52:49] thansk [10:52:51] *thanks [11:04:18] 👍 [11:28:40] dcaro: FYI during the last couple of weeks there was some work on this T371391 [11:28:40] T371391: Cloud VPS: extend tofu-infra to cover quotas - https://phabricator.wikimedia.org/T371391 [11:28:48] so now we can have quotas on tofu-infra [11:29:04] more info here: https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Projects_lifecycle#Tracking_project_quotas_on_tofu-infra_repository [11:47:02] so, when a user requests an increase of quota, what's the way to add it? [11:51:18] I would say the cookbook is fine, unless there is a quota set in tofu-infra, which will then "win" and override whatever is set via other methods [11:51:35] we would need to decide what we want to do moving forward [11:52:23] should we discourage then the tofu one until we decide? to avoid confusion? [11:53:42] the other option is to discourage the cookbook, though if I understand correctly not all quotas are supported on tofu, and the cookbook is the current entry point for clinic duty actions [11:55:45] mmm [11:55:57] I prefer the tofu-infra repo, even if it cannot cover trove quotas [11:56:17] we can change the cookbook to create the tofu patch then (like with projects) [12:00:00] I want to avoid having two conflicting ways of doing the same thing, and having many different starting points for clinic duty/operational tasks [12:13:40] sure [12:13:57] the project patch I guess is broken because the repo changed format BTW [12:24:47] oh, that's not good [12:25:01] is there a task for it, are you working on it already? [12:30:28] I'm not working on it [12:30:32] I'm busy with other stuff and I don't plan to work on it [12:33:25] do you have a task for it then? project creation flow being broken is quite a relevant thing [12:33:48] no, no ticket either [12:36:37] we have been flying like that for a month more or less, so maybe not as relevant as it might seen [12:39:19] we have been without ceph HA for years... xd [13:45:25] reopened T375283 to keep track there given the lack of a new one, will update soon, it's fairly easy to add to the new structure to the cookbook I think (yay for modularity \o/ ) [13:45:26] T375283: tofu-infra: refactor repo structure - https://phabricator.wikimedia.org/T375283 [14:10:09] 👍 [14:15:07] I'm getting 404s when the cookbook tries to create a new project, is that the same thing that you're talking about? [14:15:11] https://www.irccloud.com/pastebin/Wa7bjncL/ [14:15:19] (404s trying to reach gitlab) [14:15:32] most likely the cookbook doesn't work, as the file structure changed [14:16:26] ok. Is that something I should fix? (Missing that cookbook is currently at the top of a tall stack of issues between me and the actual thing I want to work on) [14:17:45] (specifically, when I create a project the old way it doesn't have any security groups, which I think is because that's now done by the cookbook vs. by keystone hooks) [14:21:43] andrewbogott: I started working on it [14:21:44] no [14:21:58] ok! If I create the project with tofu instead will tofu take care of security groups? [14:22:11] andrewbogott: the default security group is a different thing, and it has a couple of angles. I would be happy to elaborate more on a video call [14:22:13] not the users, or the checks and such [14:22:34] ok, I will see you at the checkin shortly anyway [14:22:38] the short answer is tofu-infra can track the default security group [14:22:47] ok! [14:22:52] that's all I need I think [14:22:57] ok [14:23:35] but also there is this thing about "default security group rules" which is another neutron API extension, which tofu cannot track because there is no support in the go library [14:23:43] happy to chat about all that [15:27:36] arturo: is this missing pieces? https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/185 [15:32:45] andrewbogott: I'm almost there if you wait 5 min, the quotas though are not handled yet, I'd wait until we decide if tracking them or not [15:32:59] (as that'd mean that we have to re-import all) [15:33:09] dcaro: almost there with the cookbook you mean? sounds good! [15:33:31] andrewbogott: do you know if we are supposed to get root@wmcloud.org emails? [15:33:38] we as in WMCS humans [15:33:54] andrewbogott: yep, the cookbook, mr example (failing the fmt xd) [15:33:55] https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/187 [15:34:08] arturo: I don't know about 'supposed to' but I do [15:34:21] and I sometimes use that as the email for service accounts, expecting to get them [15:34:39] andrewbogott: can you check in real time if you just got a confirmation email for a `toolsbeta-tofu-bot` account? [15:35:33] arturo: not yet. And actually the most recent one I got was in 08 of 2024 [15:36:02] which I was maybe bcc'd for? So maybe we don't get them [15:36:20] * andrewbogott wonders how e.g. root@wm.org works [15:36:36] it should be in private puppet [15:36:57] although i think the mx exim->postfix migration changed something [15:37:15] ok, I wont fall into this rabbit hole today --- I'll just use a different email for now [15:37:19] andrewbogott: for the MR, you need `sudo cookbook wmcs.openstack.tofu --plan --no-dologmsg --gitlab-mr 185` [15:37:55] you mean, to test? [15:38:00] puppetserver1001:/srv/git/private/hieradata/common/profile/postfix/mx.yaml apparently [15:39:52] andrewbogott: https://gerrit.wikimedia.org/r/c/cloud/wmcs-cookbooks/+/1136402 for the cookbook, working on https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/189, feel free to try with your new project see if it works [15:40:02] thx [15:43:57] I'm adding dcaro and arturo to root@wmcloud.org. taavi, do you want in? [15:45:15] ta [15:47:19] thanks [15:47:21] yes [15:49:52] dcaro: I tried the cookbook, it's missing the line in codfw1dev.resources, otherwise seems to behave well up to the point of asking me to merge the patch [15:50:13] nice, let me fix that [15:52:40] andrewbogott: yes, to run tofu plan and verify the change does what you think it will do in terms of resource creation [15:53:10] * andrewbogott nods [15:56:00] andrewbogott: Do you still have some time at hand to look into the PAWS deployment on codfw1dev later today? [15:58:44] atrawog: yes, I have time. Right now I'm stuck in some other issues that need fixed before we can get anywhere with magnum but I might have things sorted in a few hours [15:58:47] maybe :/ [16:03:42] That's okay. I will try to catch up with you later today or give things another try tommorow. [16:06:47] * arturo offline [16:24:16] Hm, with manage_default_secgroup = false we don't get any rules, with = true tofu errors out with a 403 [16:24:38] What user does tofu run as when creating openstack resources? [16:31:09] andrewbogott: fixed the cookbook, should work now [16:31:18] it has its own service account [16:39:02] andrewbogott: ready for review https://gerrit.wikimedia.org/r/c/cloud/wmcs-cookbooks/+/1136402 [16:48:42] hmmm taavi in codfw1dev at least, tofuadmin has inherited 'admin' role on all projects but it doesn't have 'member' -- I think I'm going to add it as member as well since that's what policies usually check for [16:50:54] yeah, that fixed my 403 [17:12:12] * dcaro off [17:12:14] cya tomorrow!