[08:00:06] Cteam: welcome to today 🦄! Don’t forget to post your update in thread. [08:00:06] Feel free to include: [08:00:06] 1. 🕫 Anything you'd like to share about your work [08:00:06] 2. ☏ Anything you'd like to get help with [08:00:06] 3. ⚠ Anything you're currently blocked on [08:00:06] (this message is from a toolforge job under the admin project) [09:55:30] Done: [09:55:30] * [novafullstack] fixed issues with dns entries not being created (just restarted *all* designate daemons) https://phabricator.wikimedia.org/T367235 [09:55:30] * [envvars-api] deployed a fix to show a "you are out of quota" kind of error when you are out of quota and try to create an envvar [09:55:30] Doing: [09:55:30] * [toolforge,auth] back to the drawing board to get the flows sorted out, updated the task with yesterday's meeting info (https://phabricator.wikimedia.org/T363983) [09:55:30] * [toolforge] add alert for k8s api server about to expire certs https://phabricator.wikimedia.org/T366579, it's a bit more complicated than expected, still on it [09:55:30] * [ceph] Update the health dashboard to include router link errors/discards from both sides of the links (started yesterday and when I was almost done, clicking 'save' failed and got okta'd and lost all of it :,( ) [09:55:31] Blockers: [09:55:31] * [api-gateway] sent a patch with auth-enabled api-gateway, not in a rush, but ready for reviews (https://gitlab.wikimedia.org/repos/cloud/toolforge/api-gateway/-/merge_requests/23) - arturo let me know when you are around and we can go over the patch if you want [10:16:57] doing: [10:16:57] * learning about the nova scheduler to figure out how 2 fix the "flavor properties seem to be cached for existing VMs" issue [10:16:57] * wrapping up IR for Ceph incident [10:27:56] ^thanks for that [11:31:50] doing: kyverno-related outage follow up [16:35:48] doing: chasing down owners of abandoned projects and Buster VMs [17:04:36] Done: [17:04:38] * T348407 quarry->toolsdb, verified it works, updated the parent task T151158 and created new subtasks [17:04:40] * watched the recording of the P&T meeting