[07:15:22] greetings [07:39:24] morning! [08:11:49] morning [10:37:31] * godog lunch [10:39:34] if anyone ever complains that our systems are not user-friendly, just show them the finnish university course registration system :/ [10:48:21] andrewbogott: what was the codfw1dev URL that you saw those certificate errors yesterday on? [10:51:15] taavi: LOL, Italian uni websites also are good challengers :D [12:32:31] got a small MR https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/202 enabling debug logging for the tests [12:33:36] and this small refactor https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/197 and following that one https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/201 simplifying the tests for runtime [12:33:49] I'll take a look [12:38:17] ok only one I'm confident enough to actually approve [12:38:21] bbiab [12:38:30] thanks :) [13:39:52] taavi: I couldn't open the proxy panel in labtesthorizon. Did you fix it already or did it fix itself? [13:43:18] andrewbogott: I did have to kick the acme-chief instance to get https://gerrit.wikimedia.org/r/c/operations/puppet/+/1179756 to apply, that might or might not have at the same time fixed the certificates themselves [13:43:55] huh, when I last looked I could not get puppet to run on the acme-chief host at all [13:44:06] but anyway, it seems fixed! [13:45:02] ok, then I am taking 100% credit of that unless someone finds evidence to the contrary [13:47:02] works for me [14:51:39] taavi: I'm trying to track down some dns record leaks; you deleted the terraform project recently, right? [14:51:49] did you manually delete VMs first? And/or does the cookbook do that? [14:53:59] I think this must have to do with old VMs again; the logs show it deleting a ptr record and yet a different ptr is leaked [15:34:32] andrewbogott: godog I think that as you don't have a filter in the commit it's running on all the nodes https://integration.wikimedia.org/ci/job/operations-puppet-catalog-compiler-test/7258/console [15:34:57] oh crap, you're right -- we had hosts specified in an earlier version [15:35:07] but because I did a live gerrit edit they were lost in the next patch :( [15:35:13] xd [15:35:16] been there [15:35:17] lol yeah that would do it, good find dcaro [15:35:21] godog: live migration looks like this... [15:35:39] oh, wait, hang on... I'll make a big paste [15:39:33] https://www.irccloud.com/pastebin/rpBO2FN6/ [15:39:51] godog: want to try ^ and confirm that it works/makes sense for you? [15:40:47] andrewbogott: thank you, makes total sense, tomorrow I'll give it a try after merging my patch and report back [15:41:13] also of course in hindsight it also makes sense you don't have to specify a target host for live migration, it gets picked automatically [15:41:43] godog: one thing to know is that the 'migrate' workflow and the 'resize' workflow use shared code in the backend. So sometimes you'll be migrating a thing and it will show with a 'resize' status (or maybe the other way around?) So don't be alarmed at nova using those two terms interchangeably. [15:42:04] you /can/ specify a target host but you don't have to, since it's most often used for e.g. evacuation. [15:42:15] * godog nods [15:43:05] * andrewbogott out for the rest of the day, probably [15:43:14] take care, ttyl [15:43:34] take care! [17:00:20] * dcaro off [17:00:22] cya! [21:40:15] andrewbogott: yeah, I deleted the VMs by hand first and then deleted the project itself [22:19:22] Ok, so it's probably not specific to deleting a project then