[09:40:59] !log admin [codfw1dev] rebooting bastion-codfw1dev-02 (no IP address in the main interface) T336963 [09:41:06] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [09:41:06] T336963: cloudcontrol2001-dev can't reach cloud-vps public IPs - https://phabricator.wikimedia.org/T336963 [14:56:10] I looked around but can't find my Cloud VPS project's GID so I can finish my puppet patch here: https://gerrit.wikimedia.org/r/c/operations/puppet/+/926599 [14:56:17] can anyone help? [15:00:40] milimetric: I can, let's see... [15:02:25] https://www.irccloud.com/pastebin/IcDUtKOj/ [15:02:32] milimetric: ^ [15:03:42] Have you tried https://wikitech.wikimedia.org/wiki/Portal:Data_Services/Admin/Runbooks/Enable_NFS_for_a_project#Find_out_the_GID_for_the_project ? [15:03:50] (we might have to update the page) [15:04:28] andrewbogott: thanks! dcaro: I searched like "find gid Cloud VPS" and a combination of those and didn't find it [15:04:36] didn't think to look at the NFS instructions, thx [15:06:07] tjere [15:06:17] there's also https://ldap.toolforge.org/ which probably would answer the question [18:30:14] Hi all! Does anyone know who owns quarry.wmcloud.org? I sent an email to the admin listed on the website and they said that they haven't been involved for a while [18:31:00] HTriedman: owns is too strong of a word. But I work on it [18:31:10] HTriedman, the WMCS team supports that site. You can chat with us here, or would you like a better email? [18:34:31] I'll send an email — it's a bit in-depth :) [18:34:38] where can I reach you all? [18:36:14] cloud-admin@lists.wikimedia.org is the public list for all wmcs roots. If it's sensitive or personal, feel free to DM for an alternative [18:36:21] !log clouddb-services add btullis to the project [18:36:23] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Clouddb-services/SAL [18:42:46] !log integration - access to old gerrit service IP (gerrit-old.wikimedia.org) for cloud IPs was removed with gerrit:927246 (homer deploy), T336427 [18:42:49] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Integration/SAL [18:42:50] T336427: decom gerrit1001 - https://phabricator.wikimedia.org/T336427 [18:43:53] balloons: just sent an email to the list [18:46:04] is commonswiki.analytics.db.svc.wikimedia.cloud down? [18:46:47] yes, T338172 [18:46:47] T338172: Can't connect to analytics replicas from Toolforge - https://phabricator.wikimedia.org/T338172 [18:47:02] ok, thank you [19:24:24] I'm back at keyboard now, looking at the wikireplicas issue. [19:24:58] Could it be to do with the alert on lvs1018 ? [19:29:10] btullis: I think the *.analytics replicas are reachable again, but I don't know what if anything was done to fix them following your comments on the private task where you were attempting to make things work again. [19:31:36] bd808: Thanks for the update. I haven't done anything myself to fix them. I'm looking into the cause of `Services known to PyBal but not to IPVS` from https://phabricator.wikimedia.org/T338172#8903867 [19:43:49] there also seemed to be some ongoing lvs/pybal maintenance, I wonder if it was just a case of bad luck and timing [21:13:37] I understand this might be a little early (*at least* 5 days too early!), but what's the plan for getting Debian Bookworm images available on WMCS? [21:13:59] ("no plan yet" is also fine, I'm just curious :D ) [21:14:52] "As soon as Bookworm is released we should start the drive to move users off of Buster and on to Bookworm." -- found on https://phabricator.wikimedia.org/T331738 [21:15:19] So, Saturday? ;) [21:16:13] I'm assuming it'll take a week or two to build a base image [21:16:22] maybe ask on that ticket, by quoting it [21:16:35] or even bolder, create ticket to make the images [21:17:01] seems like it's going to be needed anyways [21:38:56] stw: T335507 is currently blocked on T335560. Once there is a WMF "official" bookworm base image we can start working on adding new Toolforge containers based on it. I would expect this to be something that t.aavi and I are interested in working on, but don't expect that the new images will happen immediately. [21:38:56] T335560: Publish Wikimedia bookworm base Docker image - https://phabricator.wikimedia.org/T335560 [21:38:56] T335507: Build Bookworm based Toolforge Kubernetes images - https://phabricator.wikimedia.org/T335507 [21:40:30] bd808: sorry, I'm referring to the nova images, not docker images :) [21:40:41] (the images used to build VMs with, not containers) [21:43:50] stw: ah. right. I can't find any bookwork tasks made by andrewbogott other than T331738 which is actually about starting the march to get rid of buster once we have bookworm. [21:43:51] T331738: Cloud-vps Buster deprecation - https://phabricator.wikimedia.org/T331738 [21:44:11] *bookworm tasks [21:44:46] yeah, I'm planning on creating a ticket for it later tonight once I've finished eating :) [21:45:11] I think the answer for nova is similar... I'd like it to be released (or almost released) before we start creating VMs since we tend to have to support those VMs forever once they're out there. [21:45:19] Was there a release and I missed it? [21:45:27] figuring out what needs to change in ops/puppet.git to support Bookworm is the general prerequisite. [21:45:39] andrewbogott: it's planned for Saturday (June 10) [21:45:53] Great, I'll plan to build something on Monday then! [21:46:13] ..................................................................... [21:46:31] (argh, sorry, leant on keyboard) [21:47:09] that sounds awesome, thanks :) [21:47:10] very cool, I will also try those out for devtools and more [21:47:28] it makes sense to do that before prod servers [21:47:31] T330495 seems to be getting active work [21:47:31] T330495: Prepare our custom installer for Bookworm - https://phabricator.wikimedia.org/T330495 [21:47:45] ack, thx [21:47:59] subscribes [22:02:19] andrewbogott: I've created https://phabricator.wikimedia.org/T338188 for the bookworm image :) ( cc: mutante ) [22:02:39] ok! [22:25:07] thanks [23:29:13] stw: Just in case for some reason you have an answer for this... https://phabricator.wikimedia.org/T338192 [23:31:15] That, unfortunately, I do not :( My experience with systemd-resolved (and systemd-networkd) has been entirely through configuration through Netplan... which of course is Ubuntu-specific knowledge. [23:46:49] andrewbogott: the behavior sounds like https://wiki.archlinux.org/title/systemd-resolved#systemd-resolved_does_not_resolve_hostnames_without_suffix, but I don't have an answer for why systemd-resolved is in the base image to start with