[14:28:14] <wm-bb>	 <sohom_datta> `toolforge jobs load jobs.yaml` seems to be crashing for me :(
[14:29:03] <wm-bb>	 <sohom_datta> https://phabricator.wikimedia.org/P64977
[14:30:38] <wm-bb>	 <sohom_datta> https://github.com/sohomdatta1/qpqtool/blob/main/jobs.yaml is the offending yaml file
[14:32:17] <taavi>	 @sohom_datta: can you file a task please?
[14:36:48] <wm-bb>	 <sohom_datta> Filed T367520
[14:38:35] <andrewbogott>	 @sohom_datta, I'm not positive but I think you need a - on your first line? like '- name: "period-db-import"'
[14:38:50] <andrewbogott>	 I mean, ideally you would get an error message about that
[14:40:19] <taavi>	 I rolled back today's jobs-api deployments and now it seems fine
[14:41:02] <wm-bb>	 <sohom_datta> It's working now (I've added the dash as well just in case)
[14:42:56] <wm-bb>	 <sohom_datta> It was the `-` that was the problem
[14:45:31] <taavi>	 ah, no, I too was testing with an invalid file
[14:49:15] <taavi>	 ok, I closed that one as invalid due to the noise I added and filed T367569 for the real issue
[14:49:16] <stashbot>	 T367569: toolforge jobs load crashes if given YAML is an object (instead of an array) - https://phabricator.wikimedia.org/T367569
[14:49:23] <taavi>	 and reverted my hasty rollback
[16:04:24] <wm-bb>	 <sohom_datta> Toolforge is not being great today, Idk why but the value of TOOL_TOOLSDB_PASSWORD seems to be different from the one in my replica.conf
[16:05:31] <wm-bb>	 <sohom_datta> This is on the qpqtool tool
[16:06:11] <wm-bb>	 <sohom_datta> I assume I can just set it to the correct one and things should be fine ?
[16:40:45] <mutante>	 !log k8splay deleted buster instance dzahn after creating replacement instance dzahn-bookworm and copying over data (T367535)
[16:40:48] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:K8splay/SAL
[16:40:50] <stashbot>	 T367535: Cloud VPS "k8splay" project Buster deprecation - https://phabricator.wikimedia.org/T367535
[16:50:43] <mutante>	 bd808: re: redirects. should I just create a bookworm instance?
[16:51:04] <mutante>	 then put that puppet role on it.. and see what happens
[16:51:13] <mutante>	 is it just nginx and the nginx config?
[16:54:10] <bd808>	 mutante: yeah, that should pretty much be all it takes. Setup a instance with the role, do some localhost curl tests to see that it is emitting the expected redirects, and then switch the proxy config in Horizon to point at the new instance
[16:54:48] <mutante>	 ok, I can at least start 
[16:55:06] <mutante>	 !log codesearch createing instance codesearch9 with bookworm to replace codesearch8 (T367479)
[16:56:09] <wm-bb>	 <lucaswerkmeister> I’d try that and also file a phabricator task, I think (re @sohom_datta: I assume I can just set it to the correct one and things should be fine ?)
[16:59:06] <mutante>	 !log codesearch createing instance codesearch9 with bookworm to replace codesearch8 (T367479)
[16:59:09] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Codesearch/SAL
[16:59:09] <stashbot>	 T367479: Replace or remove Debian Buster VMs in 'codesearch' cloud-vps project - https://phabricator.wikimedia.org/T367479
[17:00:00] <mutante>	 !log redirects creating instance redirects-nginx03 (g4.cores1.ram2.disk20) with bookworm to replace buster (T367550)
[17:00:02] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Redirects/SAL
[17:00:02] <stashbot>	 T367550: Cloud VPS "redirects" project Buster deprecation - https://phabricator.wikimedia.org/T367550
[17:07:39] <mutante>	 ah, all the actual redirect mappings are generated from project Hiera.. nice
[17:08:09] <mutante>	 moves the role from instance to project puppet
[17:31:31] <mutante>	 forgot about an errand, be back later to do this and watch puppet
[17:46:45] <wm-bot>	 !log lucaswerkmeister@tools-bastion-13 tools.wd-image-positions deployed 606e0ca438 (T367188)
[17:46:49] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wd-image-positions/SAL
[17:51:26] <wm-bot>	 !log lucaswerkmeister@tools-bastion-13 tools.wd-image-positions deployed 62f5575dbd (fix logout, noticed while testing T367188)
[17:51:28] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wd-image-positions/SAL
[19:08:30] <andrewbogott>	 !log shutting down and deleting clouddb-services project for T359810
[19:08:31] <stashbot>	 andrewbogott: Unknown project "shutting"
[19:08:31] <stashbot>	 T359810: Are clouddb-wikireplicas-query-1 and the cloudb-services project still useful? - https://phabricator.wikimedia.org/T359810
[19:08:37] <andrewbogott>	 !log clouddb-services shutting down and deleting clouddb-services project for T359810
[19:08:40] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Clouddb-services/SAL
[21:01:32] <mutante>	 bd808: the puppet part was unusually cool, everything worked immediately, no warnings, nothing. also every single one of the redirectsworks with curl on localhost. just when I switch a webproxy to the new backend..not working yet. like some networking or firewalling 
[21:01:58] <mutante>	 phab.wmcloud.org and phorge.wmcloud.org are switched for testing. the rest are unchanged.
[21:03:19] <mutante>	 maybe security group
[21:08:51] <mutante>	 also works from one machine to the IP of the other
[21:40:11] <bd808>	 mutante: sounds like a missing security group on the new instance to me. I’m glad my puppet code was nice to use. :)
[21:41:25] <mutante>	 on the very first run, with no errors and no warnings, after jumping one distro.. that isn't common 
[21:42:11] <mutante>	 yea, I looked at default group and the other group called "web" but web doesn't hardcode that IP
[21:43:29] <mutante>	 the default group might have the subnet that changed for the new instance.. but is not about http
[21:44:32] <mutante>	 maybe I should reboot the machine for good measure because it never has since it was created. heh