[09:42:51] 10Traffic, 10Observability-Alerting, 10serviceops: Timeouts when talking to phabricator API - https://phabricator.wikimedia.org/T341039 (10fgiunchedi) [09:47:25] 10Traffic, 10ops-codfw: ManagementSSHDown - https://phabricator.wikimedia.org/T340960 (10fgiunchedi) Opened a dedicated task for the issue: {T341039} [09:54:14] 10Traffic: Refactor haproxy module to support tls/http settings as different types - https://phabricator.wikimedia.org/T341040 (10Fabfur) [10:01:09] 10Traffic: Refactor haproxy module to support tls/http settings as different types - https://phabricator.wikimedia.org/T341040 (10Fabfur) [10:05:35] 10Traffic, 10RESTBase, 10RESTBase-API, 10SRE: REST API is not invalidating caches after template and/or module changes - https://phabricator.wikimedia.org/T335770 (10akosiaris) @Brycehughes it does, albeit I think you shouldn't be able to reproduce now. This was possibly the result of cache purging taking... [11:57:37] 10Traffic, 10SRE, 10envoy, 10serviceops, 10Patch-For-Review: Upgrade Envoy to supported version - https://phabricator.wikimedia.org/T300324 (10JMeybohm) [11:57:48] 10Traffic, 10SRE, 10envoy, 10serviceops, 10Patch-For-Review: Upgrade Envoy to supported version - https://phabricator.wikimedia.org/T300324 (10JMeybohm) [13:22:00] 10Traffic, 10Observability-Alerting, 10SRE, 10serviceops: Timeouts when talking to phabricator API - https://phabricator.wikimedia.org/T341039 (10fgiunchedi) [13:28:39] 10Traffic, 10Observability-Alerting, 10SRE, 10serviceops: Timeouts when talking to phabricator API - https://phabricator.wikimedia.org/T341039 (10fgiunchedi) [14:07:31] 10Traffic, 10SRE, 10envoy, 10serviceops, 10Patch-For-Review: Upgrade Envoy to supported version - https://phabricator.wikimedia.org/T300324 (10JMeybohm) All nodes and most k8s deployments have been updated to run 1.23.10, only exceptions are api-gateway and rest-gateway which still run 1.18 as well as da... [14:09:16] 10Traffic, 10SRE, 10envoy, 10serviceops, 10Patch-For-Review: Upgrade Envoy to supported version - https://phabricator.wikimedia.org/T300324 (10JMeybohm) [15:10:47] hi all wondering if you could help with a dnsdisc issue im seeing (cc sukhe)? [15:12:03] jbond: Traffic meeting happening right now but hth! [15:12:05] what's up? [15:12:21] sukhe: it can wait, can yuo ping me when you are out? [15:12:48] sure [15:12:55] thanks [15:15:40] 10Traffic, 10MW-on-K8s, 10SRE, 10serviceops, and 2 others: Direct 0.5% of all traffic to mw-on-k8s - https://phabricator.wikimedia.org/T341078 (10Clement_Goubert) [15:15:58] 10Traffic, 10MW-on-K8s, 10SRE, 10serviceops, and 2 others: Serve production traffic via Kubernetes - https://phabricator.wikimedia.org/T290536 (10Clement_Goubert) [15:16:10] 10Traffic, 10MW-on-K8s, 10SRE, 10serviceops, and 2 others: Direct 0.5% of all traffic to mw-on-k8s - https://phabricator.wikimedia.org/T341078 (10Clement_Goubert) 05Open→03In progress p:05Triage→03High [15:17:17] 10Traffic, 10MW-on-K8s, 10SRE, 10serviceops, and 2 others: Serve production traffic via Kubernetes - https://phabricator.wikimedia.org/T290536 (10Clement_Goubert) [16:25:35] jbond: resolved? [16:26:00] sukhe: yes thanks [16:26:15] nice [16:26:47] for https://gerrit.wikimedia.org/r/c/operations/puppet/+/933473, PCC output https://puppet-compiler.wmflabs.org/output/933473/42230/ [16:27:11] not in a rush to merge this so whenever you have time (tomorrow even), I would appreciate a second pair of eyes on the output [16:27:14] and then we can merge this [16:27:51] sukhe: looks good to me, ship it [16:28:12] ok thanks, hopefully this ship actually sails :P [16:28:26] :) wishing you a good tail wind ;) [17:09:17] 10Traffic, 10RESTBase, 10RESTBase-API, 10SRE: REST API is not invalidating caches after template and/or module changes - https://phabricator.wikimedia.org/T335770 (10Brycehughes) @akosiaris Yep all clear now from Georgia (the country). However, this lasted much more than "several minutes". What do you thin...