[06:08:56] !log wikisp Restart mariadb and change datadir from /srv/sqldata to /var/lib/mysql [06:08:58] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikisp/SAL [06:11:34] !log wikisp Updating dependences on Mars [06:11:36] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikisp/SAL [06:14:26] Maybe a stupid question but, it's safe to update php7.4 to php8.2? [08:01:43] !log wikisp Mars: update php7.4 to php8.1 [08:01:45] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikisp/SAL [14:50:09] !log admin removing ~100 dangling backup snapshots from eqiad1-compute ceph pool [14:50:13] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [14:51:44] actually, is more like 600 [15:28:07] did removing this snapshots free any Ceph space? [15:30:11] ~8TiB [15:31:04] nice [15:31:50] it might be enought [15:31:51] *enough [15:33:08] I still can cleanup a bit more from cloudbackup1003 I think [19:56:39] !log tools.lexeme-forms deployed ad16425ee2 (l10n updates: nl, uk, zh-hans) [19:56:42] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.lexeme-forms/SAL [20:10:12] I'm having trouble SSHing onto instances in a new Cloud VPS project. It looks like the instance is set up, and ssh is connecting to the bastion bug getting hung up there :( [20:10:41] The host is ui.catalyst-qte.eqiad1.wikimedia.cloud [20:11:25] kindrobot: try using ssh -v, and paste the output somewhere where we can have a look? [20:11:59] (such as https://paste.toolforge.org/, and preferably not directly in here, it doesn’t work well with the bridge :)) [20:12:35] yes, or to phabricator pastes, or practically anything except directly onto irc :P [20:12:57] https://phabricator.wikimedia.org/P52891 [20:14:05] taavi ^ [20:14:08] thanks [20:15:19] that shows you're connecting via the bastion, there goes the most common theory [20:15:39] * taavi looks at the instance itself [20:16:16] ok, looks like the issue is with the security groups [20:16:39] the 'default' security group is applied to the instance, .. but it's missing the required rules to make SSH work for whatever reason [20:17:36] hmh [20:19:13] Ah, yes. It's missing a whole bunch. I didn't change the default security group, so is there a chance it didn't get set up properly when the project was created? [20:19:38] Should I add them for now? Manually? [20:19:49] that's my current theory, I'm looking at the logs if there's anything there [20:24:26] kindrobot: try now? [20:25:30] No joy yet [20:25:54] * taavi facepalms [20:26:22] I'm not seeing the rules added to the default [20:26:27] I checked an another recent project to see if it's a new issue, and then forgot to switch back before manually creating rules for your project [20:26:32] derp [20:27:03] Was it an issue with the other new project, or did I just get lucky ;) [20:27:12] seems to be broken on all new projects :/ [20:27:19] I'll file a task after I've fixed yours [20:27:25] Thank you <3 [20:32:58] It's working <3 [20:33:05] ok, now the groups are in the correct project [20:38:03] also, network bugs seem to have an buy-one-get-another-free offer today. filed T348581 and T348582 [20:38:03] T348582: Neutron policy does not allow the admin role to modify security groups - https://phabricator.wikimedia.org/T348582 [20:38:04] T348581: Neutron default security group provisioning is broken - https://phabricator.wikimedia.org/T348581 [20:38:37] (the latter one made fixing stuff much more annoying since I had to manually add myself as a member to the project and then re-login to Horizon to edit security groups_ [23:15:34] How would I go about figuring out why a trove instance 1) can't be created or 2) be restarted. I think it has to do with the applied configuration group, but I don't know what it wrong.