[08:24:26] <taavi>	 !log tools depool tools-sgeexec-[0901-0909] (7 nodes total) T277653
[08:24:28] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[08:24:29] <stashbot>	 T277653: Toolforge: add Debian Buster to the grid and eliminate Debian Stretch - https://phabricator.wikimedia.org/T277653
[08:28:53] <Wurgl>	 Does anybody use an android smartphone with wikipedia-app installed? When I open :de:WP:C it opens a version from Sept. 11th 2020.
[08:29:37] <Wurgl>	 Something wrong with some cache? But a cache which is 704 days old?
[09:34:47] <wm-bb>	 <jhsoby> Wurgl: Seeing the same here in my Android app
[09:35:34] <Wurgl>	 Okay. So this is not my local problem and I can write some phab report
[09:35:37] <Wurgl>	 thx
[09:35:42] <wm-bb>	 <jhsoby> np
[13:42:34] <taavi>	 !log toolsbeta run grid-configurator to remove stale config for some removed nodes 
[13:42:36] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL
[13:45:07] <chippy>	 Hello, I tried to resize an instance (maps-warper4 in maps project) to a smaller flavor but in the process it didn't work and is now in Error status
[13:45:49] <chippy>	 the message was "Unexpected vif_type=binding_failed" 
[13:48:35] <chippy>	 at some point I'd like to resize the instance to a larger flavor, but currently the project has no quota less, so was seeing what to do later on when the older debian instance gets shut down and the quota get's freed up.
[14:02:54] <Lucas_WMDE>	 (the bridgebot is double-sending again, cc bd808)
[14:04:21] <wm-bot>	 !log tools.bridgebot <bd808> Double IRC messages to other bridges
[14:04:23] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.bridgebot/SAL
[14:07:25] <Lucas_WMDE>	 thx
[17:10:57] <chippy>	 Hi, can anyone help with a broken Instance - maps-warper4 in maps project. I am unable to start/stop it and it is in the Error status.
[17:11:11] <chippy>	 i cannot ssh into it
[17:18:58] <chippy>	 ahh is it Memorial Day today?
[17:28:43] <andrewbogott>	 chippy: I can look, at least briefly
[17:30:49] <andrewbogott>	 chippy: looks like you only just now created that VM, is that right?
[17:31:37] <andrewbogott>	 If so, I'd say just delete it and re-create.  There's an intermittent issue with instance creation right now and it'll be a few days before anyone can work on it seriously
[17:32:46] <andrewbogott>	 looks to me like it's been in error state since it was created on 2022-05-30T13:41:43Z
[17:34:05] <chippy>	 andrewbogott, i created it a few days ago and have been working on it since to replace maps-warper3
[17:34:13] <chippy>	 the error is new from earlier today
[17:34:29] <andrewbogott>	 I can try to reboot it
[17:34:30] <chippy>	 says it's 2 weeks old
[17:34:35] <chippy>	 yes please
[17:40:35] <chippy>	 i tried to re-size the instance, to be familiar with how that works. The only option was to give it a smaller flavor. Once this instance replaces the old debian one (maps-warper3) then, because the project has used it up  it's quota, I will need to re-size the instance.
[17:41:12] <chippy>	 so once the old instance is deleted, there should be quota available to make it the same spec as before
[17:42:34] <chippy>	 anyhow, it was during the resize operation that the error occured.
[17:43:02] <andrewbogott>	 !log admin restarting neutron-rpc and neutron-api services on cloudcontrol1xxx
[17:43:06] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL
[17:44:37] <andrewbogott>	 oh yeah, the resize probably killed it, that's a somewhat delicate operation
[17:44:50] <andrewbogott>	 I mean, it /should/ work but if anything is amiss...
[17:44:56] <andrewbogott>	 anyway, still seeing if I can revive it
[17:45:19] <chippy>	 Understandable that its a tricky operation tbh. thanks for revival efforts
[18:00:26] <andrewbogott>	 chippy: I did an extremely unconventional thing (forcing a value in the neutron database) but it seems to be back up now.  Can you tell if it still needs the resize or if the resize took affect?
[18:00:41] <andrewbogott>	 s/affect/effect/
[18:02:27] <chippy>	 andrewbogott, many thanks.
[18:02:45] <chippy>	 Resize didn't work but that's okay. I think the instance should be okay with the current/initial size, unless it gets a bunch of traffic at once. 
[18:03:04] <andrewbogott>	 ok, that's easy :)
[18:03:13] <andrewbogott>	 Sorry about the downtime! Hope it behaves better now.
[18:03:31] <chippy>	 I'm sorry for breaking it
[18:04:56] <chippy>	 tomorrow, I aim to ask here for help with the proxy warper.wmflabs.org which need special assistance (to point to the new instance)
[18:05:02] <chippy>	 what time is best to reach out?
[18:09:00] <Zppix>	 !log lta-tracker Add RhinosF1 as a temp project admin
[18:09:02] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Lta-tracker/SAL
[18:42:08] <wm-bot>	 !log tools.wikiloves <jeanfred> Deploy e40c6be (T304327)
[18:42:11] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wikiloves/SAL
[18:42:34] <wm-bot>	 !log tools.wikiloves <jeanfred> Trigger special udpdate of earth2021 and earth2022 for T304327
[18:42:39] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wikiloves/SAL
[19:45:31] <andrewbogott>	 chippy: sorry for the slow response, I'm out tomorrow (and also today).  In theory moving the proxy should be simple, just edit it in horizon.
[20:57:31] <chippy>	 andrewbogott, okay, will try on Wednesday. I was told that the maps project because of traffic had it's proxies split out into something special. I can see the warper.wmflabs.org as a Record Set / Zones in the DNS section but not in the proxy list. I cannot alter the zone record meaningfully. 
[20:58:06] <chippy>	 I imagine I could delete the Zone / Record Set and make a new proxy, but it might break things, and might not be what you want esp. as it was created for a reason
[21:00:59] <andrewbogott>	 You're right maps is weird, I'll have to look at it.
[21:01:04] <chippy>	 thanks