[08:00:03] Cteam: welcome to today 🦄! Don’t forget to post your update in thread. [08:00:03] Feel free to include: [08:00:03] 1. 🕫 Anything you'd like to share about your work [08:00:03] 2. ☏ Anything you'd like to get help with [08:00:03] 3. ⚠ Anything you're currently blocked on [08:00:03] (this message is from a toolforge job under the admin project) [08:20:11] Done: [08:20:11] * [ceph,alerts,dashboard] Added all the links to the dashboards, for alerts we might have to use librenms own alerting system [08:20:11] Doing: [08:20:11] * Catching up [08:20:11] * [ceph] We'll try to add more latency metrics using ebpf to some of the osds to see if the issues reflect there (https://phabricator.wikimedia.org/T348643) [08:20:11] * [toolforge,api-gateway] Looking into if it's possible to use https://nginx.org/en/docs/http/ngx_http_auth_request_module.html and let nginx do the proxying (we need to be able to set some headers from the result of the auth) [08:20:12] * [toolforge,auth] back to the drawing board to get the flows sorted out, updated the task with yesterday's meeting info (https://phabricator.wikimedia.org/T363983) [08:20:12] * [toolforge] add alert for k8s api server about to expire certs https://phabricator.wikimedia.org/T366579, it's a bit more complicated than expected, still on it [08:20:12] Blockers: [08:20:13] * None [13:01:31] today: [13:01:31] * follow up with kyverno. A few adjustements, including to scale/downscale some components [13:01:31] * prepare to change policies to Enforce [13:25:49] Done: [13:25:51] * T367464 finally found the reason for quarry queries not completing [13:25:53] Doing: [13:25:55] * T367393 enable Superset queries to ToolsDB [13:25:57] * T367415 enable Quarry to query its own database [14:19:47] Done: Migrated all non-broken trove VMs [14:19:47] Doing: Probably reimaging cloudvirts, possibly getting a start on wikitech-static move to GCP