[07:51:47] sal.toolforge.org returning a 404 is expected? [07:56:37] yeah, It was mentioned yesterday that it got impacted by the wikitech move [11:25:18] in puppet, I deploy using file {} a whole directory. Do you know if PCC is supposed to show the individual diff for each file within that directory? [11:25:49] my theory is that is wont show the diff. I have a PCC diff here that shows nothing, and I guess that is because each individual file is not an actual resource in puppet [11:26:06] if you are curious, patch is https://gerrit.wikimedia.org/r/c/operations/puppet/+/1075859 [12:41:34] arturo: correct [12:41:49] cdanis: thanks for confirming [13:55:38] hi all - friendly reminder that in about an hour (15:00 UTC) we'll be doing the last part of the switchover, where we repool eqiad for services (already repooled for traffic). I'll follow up here and in -operations before I start work. thanks! [13:58:50] ACK [14:46:56] if anyone was provisioning `irc1004.yaml` , I got your netbox diff during a makevm cookbook...just an FYI [15:00:48] <_joe_> moritzm / elukey maybe ^^ [15:01:57] inflatador: yep yep you can go ahead thanks! [15:05:34] starting to repool services momentarily (FYI - cwhite, urandom) [15:06:08] ACK np [15:35:22] services have all been repooled as of 15:27, with mediawiki active/active services repooled around 15:14. no issues encountered thus far. [15:35:35] nice [15:35:43] love to see it [15:41:13] nice job team! [15:42:21] thanks, folks :) [15:42:36] cheers scott! [15:46:32] 🎉 [15:53:06] Many thanks, Scott! [15:56:44] alright, with that, I alas need to run off for ~ 15-20m. if anything goes sideways, please ping me on IRC and I can be back in 5m :) [16:15:25] * swfrench-wmf is back [16:31:35] Hi team, the cumin2002 host seems to be unavailable for me. [16:31:52] denisse: I'm logged in on it [16:31:57] and can create a new session too [16:32:47] volans: That's strange, I was running a cookbook and it froze. From another terminal I'm unable to SSH into it. [16:33:25] Looking at verbose output I can see I'm stuck in "Connecting to bast1003.wikimedia.org". [16:33:59] try to use a different bastion maybe? [16:34:08] you might have connection issues with one dc [16:34:23] Yes, I'm trying to connect with bast2003. [16:34:31] and your cookbook is in a tmux so it's safe :) [16:36:50] No luck with bast2003 either, I'm also stuck in "Connecting to bast2003.wikimedia.org". I wonder if my IP address is blocked or something. 🤔 [16:38:06] I flushed my SSH keys from my agent, and added them again. It works now, but I wonder why the issue happened... [18:45:49] 2 [18:45:54] oops