[16:18:23] <taavi>	 !log tools.anchor-corrector bump quotas per T350484#9316930
[16:18:27] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.anchor-corrector/SAL
[16:18:28] <stashbot>	 T350484: Request increased quota for anchor-corrector Toolforge tool - https://phabricator.wikimedia.org/T350484
[16:42:58] <andrewbogott>	 !log admin created foundationmemory project for T350760
[16:43:03] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL
[16:43:03] <stashbot>	 T350760: Request creation of foundationmemory VPS project - https://phabricator.wikimedia.org/T350760
[17:21:20] <Krinkle>	 stashbot seems down?
[17:29:15] <andrewbogott>	 stashbot, how's it going?
[17:29:31] <andrewbogott>	 !log testing with a test
[17:31:41] * bd808 looks
[17:31:45] <andrewbogott>	 I'm restarting
[17:32:02] <bd808>	 cool. that usually fixes it 
[17:32:20] <andrewbogott>	 !log you there?
[17:32:21] <stashbot>	 andrewbogott: Unknown project "you"
[17:33:21] <bd808>	 I need to rip the twitter code out of stashbot to quiet the logs :/
[17:34:56] <wm-bb>	 <lucaswerkmeister> oh right
[17:35:21] <wm-bb>	 <lucaswerkmeister> apparently it hasn’t worked in in almost 5 months anyway :/
[17:39:02] <bd808>	 T343157: "The bot has not successfully posted to Twitter since 2023-06-13." so yeah...
[17:39:04] <stashbot>	 T343157: Remove Twitter support - https://phabricator.wikimedia.org/T343157
[19:05:09] <wm-bb>	 <lucaswerkmeister> bd808: would you be interested in Gerrit changes or would you want to do it yourself?
[19:05:21] <wm-bb>	 <lucaswerkmeister> (no idea if I could actually do it, but I have a few days of vacation left… ^^)
[19:07:22] <bd808>	 lucaswerkmeister: I would take a patch, but I can probably do it myself later today too. It's not difficult.
[19:07:36] <bd808>	 oh actually I think I can disable it with config first...
[19:09:47] <wm-bb>	 <lucaswerkmeister> ok :)
[19:16:07] <wm-bot>	 !log tools.stashbot <bd808> Disabled Twitter posting via config (T343157)
[20:33:55] <wm-bot>	 !log tools.stewardbots <anticomposite> ./stewardbots/StewardBot/manage.sh restart # RC not working
[21:22:41] <dduvall>	 taavi: o/
[21:22:55] <taavi>	 o/ which project and instance?
[21:23:55] <dduvall>	 zuul-1001.devtools
[21:25:41] <andrewbogott>	 seems like it can't contact its puppetmaster
[21:25:46] <andrewbogott>	 although another test VM can
[21:26:21] <dduvall>	 i just changed the puppetmaster hiera config to point to the one in that project, but it wasn't working before that either
[21:26:39] <andrewbogott>	 it was working enough to change the puppetmaster in puppet.conf :)
[21:26:40] <taavi>	 it hadn't ran puppet at all on this boot for some reason, I manually triggered it
[21:26:46] <andrewbogott>	 oh ok
[21:26:55] * andrewbogott backs out to avoid toe-stepping
[21:26:59] <taavi>	 Nov 08 21:23:24 zuul-1001 cloud-init[379]: 2023-11-08 21:23:24,825 - schema.py[WARNING]: Invalid cloud-config provided: Please run 'sudo cloud-init schema --system' to see the schema errors.
[21:27:33] <dduvall>	 huh
[21:27:36] <andrewbogott>	 that's interesting... admin-monitoring VMs are starting up Ok
[21:27:42] <taavi>	 the nova metadata service had some blip when that VM was being booted?
[21:27:43] <dduvall>	 i don't remember giving it any cloud-init script
[21:27:48] <taavi>	 anyhow, dduvall: can you log in now?
[21:28:06] <andrewbogott>	 dduvall: I'm also interested in if you can reproduce that with another new VM
[21:28:08] <dduvall>	 taavi: i can! thanks a ton
[21:28:12] <taavi>	 cloud-init is used to provision the base VM and to start the first puppet run
[21:28:26] <andrewbogott>	 dduvall: only if you feel like it though
[21:28:42] <dduvall>	 i'd have to try it in another project. the instance quota is all used up there
[21:28:46] <dduvall>	 but i'm willing, sure
[21:32:04] <dduvall>	 taavi: is that the "Customization Script" in the horizon UI? i didn't select anything there AFAICT
[21:32:55] <dduvall>	 just created `repro-cloud-init-blues-1001.deployment-prep` to see if it happens there
[21:33:33] <taavi>	 most of the cloud-init config is coming from a file we puppetize on the cloudcontrol hosts. what you see horizon might be a part of it, but most of it is coming from elsewhere
[21:33:43] <dduvall>	 ah, ok
[21:37:55] <dduvall>	 andrewbogott: no luck repro'ing it. `repro-cloud-init-blues-1001.deployment-prep` came up just fine and was accessible
[21:38:14] <andrewbogott>	 great, thanks for testing.
[21:38:20] <andrewbogott>	 I'm going to pretend this never happened for now :)
[21:38:28] <dduvall>	 me too :)
[21:38:31] <dduvall>	 thanks again, taavi 
[21:42:50] <wm-bb>	 <harej> How often does puppet run?
[22:01:25] <JJMC89>	 harej: in clound VPS, it should run every 30 min from what I've seen
[22:01:39] <JJMC89>	 *cloud
[22:02:08] <bd808>	 that sounds right. it's a systemd timer with clock splay that Puppet itself sets up somewhere.