[07:52:44] <dcaro>	 Morning!
[07:53:04] <volans>	 morning
[07:54:24] <dcaro>	 oh, I want to try https://gitlab.wikimedia.org/volans/wmf-claude/-/tree/lima-vm-unpriv this week xd, might send a bunch of questions your way fyi.
[07:55:07] <volans>	 sure! dhinus is the actual maintainer, I'm just the founder :-P
[07:55:59] <volans>	 he was about to send a PR to wmf-claude to get it in upstream, but then kosta tried a totally different approach, so not sure what we'll end up with.
[08:18:12] <volans>	 FYI I'm about to upgrade in place to trixie cloudcumin2001, noone is using it an noone has used it recently
[08:20:29] <dcaro>	 ack
[08:50:40] <dcaro>	 The electrician turned off the electricity in my flat, I'll be "offline" for a few minutes hopefully 
[09:00:35] <volans>	 {done}, cumin works again on cloudcumin2001
[10:04:17] <dcaro>	 dhinus: in case you missed it https://gitlab.wikimedia.org/repos/cloud/toolforge/logs-cli/-/merge_requests/11 (the prior and next had reviews but that one did not xd)
[10:05:45] <dhinus>	 dcaro: yes I did miss it :D
[12:06:47] <godog>	 greetings
[12:46:49] <dcaro>	 hmm, toolsbeta harbor is getting out of quota (for CI build images)
[12:47:38] <dcaro>	 xd, last cleanup was in march
[12:47:41] <dcaro>	 https://usercontent.irccloud-cdn.com/file/x32fwsgd/image.png
[12:49:22] <andrewbogott>	 Can I get a +1 for this trove request?  https://phabricator.wikimedia.org/T439975  (I'm thinking this needs a project request ticket but I'll just make that myself)
[12:49:27] <andrewbogott>	 also -- good morning!
[12:49:37] <dcaro>	 Raymond_Ndibe: have you been playing lately with toolsbeta harbor? this tag retention rule looks weird (maybe a test?) `For the repositories matching **, tags matching *-??????????????-????????`
[12:50:40] <dcaro>	 120GB DB is quite big
[12:51:44] <dcaro>	 andrewbogott: +1d
[12:54:03] <andrewbogott>	 ty
[12:58:53] <andrewbogott>	 hmmm do we have any kind of naming convention for the trove-only projects?  We should :/
[13:01:55] <dcaro>	 agree, I think there's not, usually people add 'database' or 'db', but it's up to them
[13:12:04] <godog>	 FYI I just merged backup-specific alerts, in case you see some pop up
[13:12:13] <godog>	 T428893 that is
[13:12:13] <stashbot>	 T428893: Add monitoring for backy2 openstack backups - https://phabricator.wikimedia.org/T428893
[13:14:49] <dcaro>	 🎉
[13:48:53] <andrewbogott>	 dcaro:  https://gerrit.wikimedia.org/r/c/cloud/wmcs-cookbooks/+/1351309
[13:54:51] <godog>	 doh of course I deployed the backup alerts to 'cloud' prometheus but that's not right, they are in 'ops' prometheus
[13:54:56] <godog>	 fixing
[13:57:13] <dcaro>	 andrewbogott: +1d
[14:45:28] <andrewbogott>	 dcaro: also https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1445
[14:46:31] <dcaro>	 lgtm
[14:47:21] <andrewbogott>	 thx
[14:47:48] <andrewbogott>	 hm... I think I have no idea how to deploy this
[14:48:57] * andrewbogott reads the docs, which exist
[14:51:53] <andrewbogott>	 dcaro: does this look right?
[14:51:54] <andrewbogott>	 sudo cookbook wmcs.toolforge.component.deploy --cluster-name tools --task-id T439237 --component maintain-kubeusers
[14:51:54] <stashbot>	 T439237: Request increased quota for spacemedia Toolforge tool - https://phabricator.wikimedia.org/T439237
[14:53:30] <dcaro>	 andrewbogott: nope, probably should have said xd, you were not supposed to merge it before deploying
[14:53:45] <dcaro>	 anyhow, now you can add `--git-branch main` so it deploys the main branch
[14:54:03] <andrewbogott>	 ok... 
[14:54:24] <dcaro>	 the idea is that if it got merged, it's deployable, so we deploy before merging to make sure
[14:55:00] <dcaro>	 (no big issue, just that the cookbook expects you to use that flow by default)
[14:55:49] <andrewbogott>	 I've updated the docs, does this look correct now?  https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin#Quota_management
[15:02:58] <andrewbogott>	 dcaro: ^ ?  (if there are different docs someplace else then we should maybe replace that with a link)
[15:03:40] <dcaro>	 LGTM 👍 thanks!
[15:05:51] <andrewbogott>	 of course now the deployment is failing :/
[15:06:22] <dcaro>	 what's the error? (I'm in a meet, so might not reply promptly)
[15:06:31] <andrewbogott>	 ✗ config generate with some existing jobs works, is loadable and creates the same job (slow)
[15:06:48] <andrewbogott>	 https://www.irccloud.com/pastebin/4cKnQHxy/
[15:06:52] <andrewbogott>	 which I can't extract any info from
[17:20:27] <andrewbogott>	 dcaro: it succeeded on a second attempt
[17:25:54] <dcaro>	 yep, sounds like a timeout of sorts :/, you'd have to run manually to see extra logs
[17:35:07] <dcaro>	 can I get a review for https://gitlab.wikimedia.org/repos/cloud/toolforge/logs-api/-/merge_requests/50 ? (simple, but it's breaking a major usecase in prod)
[17:35:27] <dcaro>	 toolforge prod this is, not wikipedia/sre prod
[17:43:42] <dcaro>	 nm, I'll deploy, feel free to review after though
[17:44:31] <andrewbogott>	 looks ok to me, although if the original goal was to get rid of the 500 default, maybe we should make the default a very small number?
[17:44:38] * andrewbogott unclear why the 500 default was removed in the first place
[17:44:49] <andrewbogott>	 arguably 500 is already a very small number :)
[17:48:46] <dcaro>	 it was a refactor
[17:49:30] <dcaro>	 I think the intention was to get all the logs (that's how the other endpoint behaves when passed 0)
[17:49:43] <dcaro>	 but I think 500 is a good enough default for 'following'
[17:50:07] <dcaro>	 (as in, it will fetch 500 logs, then start following new ones)
[17:54:49] <andrewbogott>	 ok!
[18:23:45] * dcaro off
[18:23:51] <dcaro>	 cya tomorrow!