[08:29:32] elukey: IIRC poddisruptionbudget v1beta1 and v1 a equivalent, it has only been bumped to v1. So I suggest you leave it at v1beta1 for now until we're fully 1.23 [09:32:26] jayme: ack yes I was thinking the same, there is a minor difference but nothing important [10:44:02] 10serviceops, 10MW-on-K8s, 10SRE, 10Traffic, and 2 others: Serve production traffic via Kubernetes - https://phabricator.wikimedia.org/T290536 (10Joe) For the record, we decided to start with option 3 and we're starting with rollout phase 1, specifically we'll move test2.wikipedia.org to kubernetes first. [11:04:12] 10serviceops, 10Kubernetes, 10Patch-For-Review: Possible improvements to kube_env - https://phabricator.wikimedia.org/T324091 (10Clement_Goubert) 05Open→03In progress I've implemented most of your suggestions in the linked CR, let me know what you think. Other than that, if we start wanting to do more co... [11:04:24] 10serviceops, 10Kubernetes, 10Patch-For-Review: Possible improvements to kube_env - https://phabricator.wikimedia.org/T324091 (10Clement_Goubert) a:03Clement_Goubert [11:33:22] 10serviceops, 10Kubernetes: Possible improvements to kube_env - https://phabricator.wikimedia.org/T324091 (10Clement_Goubert) 05In progress→03Resolved Deployed in production. Since it's profiles, either `source /etc/profile.d/kube-conf.sh && source /etc/profile.d/kube-env.sh` or restart your ssh session to... [11:52:44] btullis: jayme: I wiki-ed the helmfile destroy/apply dance https://wikitech.wikimedia.org/wiki/Kubernetes/Deployments#Release_breaking_changes corrections and additions welcome [11:53:44] claime: Nice, thanks. [11:56:53] Especially if we can list an example of cases where that is required rather than a simple apply, that's something I don't know yet :) [11:57:05] -an [12:00:14] 10serviceops, 10Traffic-Icebox, 10Platform Team Initiatives (API Gateway): Handle edge cache invalidation for the api gateway - https://phabricator.wikimedia.org/T324200 (10Joe) [12:02:42] 10serviceops, 10Analytics-Clusters, 10Analytics-Radar, 10SRE: Consider Julie for managing Kafka settings, perhaps even integrating with Event Stream Config - https://phabricator.wikimedia.org/T276088 (10LSobanski) [12:07:01] 10serviceops, 10CampaignEvents, 10Wikimedia-Site-requests, 10Campaign-Registration, and 2 others: Run the timezone update script periodically in prod and in beta - https://phabricator.wikimedia.org/T320403 (10Clement_Goubert) p:05Triage→03Medium [12:53:20] 10serviceops, 10Kubernetes: Possible improvements to kube_env - https://phabricator.wikimedia.org/T324091 (10fgiunchedi) Thank you @Clement_Goubert for your help on this -- exactly what I was looking for! [13:24:22] 10serviceops, 10Analytics-Clusters, 10Analytics-Radar, 10Data-Engineering, and 2 others: Consider Julie for managing Kafka settings, perhaps even integrating with Event Stream Config - https://phabricator.wikimedia.org/T276088 (10Ottomata) [14:13:34] 10serviceops, 10Analytics-Clusters, 10Analytics-Radar, 10Data-Engineering-Planning, and 2 others: Consider Julie for managing Kafka settings, perhaps even integrating with Event Stream Config - https://phabricator.wikimedia.org/T276088 (10EChetty) [14:32:21] 10serviceops, 10Kubernetes: Possible improvements to kube_env - https://phabricator.wikimedia.org/T324091 (10taavi) 05Resolved→03Open `kube_env` has tab completion configured, but that doesn't seem to work with `kube-env`. [14:33:13] 10serviceops, 10Kubernetes: Possible improvements to kube_env - https://phabricator.wikimedia.org/T324091 (10Clement_Goubert) >>! In T324091#8435514, @taavi wrote: > `kube_env` has tab completion configured, but that doesn't seem to work with `kube-env`. Checking, thanks. [14:55:11] 10serviceops, 10Kubernetes, 10Patch-For-Review: Possible improvements to kube_env - https://phabricator.wikimedia.org/T324091 (10Clement_Goubert) 05Open→03Resolved @taavi Fixed :) [15:11:11] 10serviceops, 10API Platform (Sprint 02), 10Platform Team Workboards (Platform Engineering Reliability): New Service Request uniqueDevices Endpoint: AQS 2.0 - https://phabricator.wikimedia.org/T320967 (10JArguello-WMF) [15:45:42] 10serviceops, 10Traffic, 10Platform Team Initiatives (API Gateway): Handle edge cache invalidation for the api gateway - https://phabricator.wikimedia.org/T324200 (10Vgutierrez) Re-tagging the task, I'm assuming it got into traffic-icebox by mistake :) [17:08:18] 10serviceops, 10Patch-For-Review: wikikube LIST secrets latency - https://phabricator.wikimedia.org/T323706 (10JMeybohm) Deployed helm-state-metrics 0.2.0 to both staging clusters [17:16:20] 10serviceops, 10Release Pipeline (Blubber), 10Release-Engineering-Team (Priority Backlog 📥): Buildkit erroring with "cannot reuse body, request must be retried" upon multi-platform push - https://phabricator.wikimedia.org/T322453 (10dduvall) [17:47:41] 10serviceops, 10SRE, 10Traffic, 10Platform Team Initiatives (API Gateway): Handle edge cache invalidation for the api gateway - https://phabricator.wikimedia.org/T324200 (10daniel) Note that we only need active purging if/when we emit cache control headers that tell the edge case to cache long-term. One k... [18:04:10] small fix for a UBN issue on the api gateway if anyone has a sec https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/863013/ [19:43:56] 10serviceops, 10DC-Ops, 10SRE, 10ops-eqiad, 10Patch-For-Review: Decommission mw13[07-48] - https://phabricator.wikimedia.org/T306162 (10ops-monitoring-bot) Icinga downtime and Alertmanager silence (ID=6c20a9fc-5041-4ab7-bed4-f80a2643f954) set by rzl@cumin2002 for 1 day, 0:00:00 on 42 host(s) and their se... [20:48:46] 10serviceops, 10Content-Transform-Team-WIP, 10Maps: Re-import full planet data into eqiad and codfw - https://phabricator.wikimedia.org/T314472 (10jijiki) @Jgiannelos and I have successfully completed re-imported full planet data on eqiad. Next up, we are working on ways we can warm up eqiad's tile cache; we... [22:54:50] 10serviceops, 10DC-Ops, 10SRE, 10ops-eqiad, 10Patch-For-Review: Decommission mw13[07-48] - https://phabricator.wikimedia.org/T306162 (10ops-monitoring-bot) cookbooks.sre.hosts.decommission executed by rzl@cumin1001 for hosts: `mw[1307-1326].eqiad.wmnet` - mw1307.eqiad.wmnet (**WARN**) - Downtimed host... [23:35:58] 10serviceops, 10DC-Ops, 10SRE, 10ops-eqiad, 10Patch-For-Review: Decommission mw13[07-48] - https://phabricator.wikimedia.org/T306162 (10ops-monitoring-bot) cookbooks.sre.hosts.decommission executed by rzl@cumin1001 for hosts: `mw[1327-1346].eqiad.wmnet` - mw1327.eqiad.wmnet (**WARN**) - Downtimed host...