[00:09:11] 10Traffic, 10Privacy Engineering, 10SRE, 10Patch-For-Review: Remove obsolete "Permissions-Policy: interest-cohort" header - https://phabricator.wikimedia.org/T312823 (10BCornwall) a:03BCornwall [08:11:45] 10Traffic, 10SRE, 10serviceops: Upgrade envoyproxy to 1.16.2 - https://phabricator.wikimedia.org/T271407 (10Vgutierrez) AFAIK this was only impacting envoy as the TLS terminator of the CDN and we went with HAProxy so this can be closed [15:23:22] 10netops, 10Infrastructure-Foundations, 10ops-codfw: codfw: Bad power supply on cr1-codfw(PEM 0) - https://phabricator.wikimedia.org/T329943 (10Papaul) [15:32:47] 10netops, 10Infrastructure-Foundations, 10SRE, 10ops-codfw: codfw: Bad power supply on cr1-codfw(PEM 0) - https://phabricator.wikimedia.org/T329943 (10Papaul) Case Number 2023-0217-642078 Case Type Tech Priority P2 - High Status Dispatch [16:11:05] 10Traffic, 10SRE: Prometheus Varnish exporter alert: add runbook and link to dashboard - https://phabricator.wikimedia.org/T289974 (10BCornwall) 05In progress→03Resolved Fixed in https://gerrit.wikimedia.org/r/c/operations/alerts/+/889887 [16:25:54] 10Traffic, 10SRE, 10Patch-For-Review: Test envoyproxy as a WMF's CDN TLS terminator with real traffic - https://phabricator.wikimedia.org/T271421 (10BCornwall) [16:26:08] 10Traffic, 10SRE, 10serviceops: Upgrade envoyproxy to 1.16.2 - https://phabricator.wikimedia.org/T271407 (10BCornwall) 05Open→03Resolved a:03BCornwall Great, thanks! [16:39:02] 10Traffic, 10Observability-Alerting, 10SRE: Move (or delete?) trafficserver restart count alert from icinga to alerts.git - https://phabricator.wikimedia.org/T327791 (10BCornwall) 05Open→03Resolved Fixed in https://gerrit.wikimedia.org/r/c/operations/puppet/+/889881 [16:57:36] 10Traffic, 10Privacy Engineering, 10SRE, 10Patch-For-Review: Remove obsolete "Permissions-Policy: interest-cohort" header - https://phabricator.wikimedia.org/T312823 (10BCornwall) 05Open→03In progress [18:05:34] 10netops, 10Infrastructure-Foundations, 10SRE, 10ops-codfw: codfw: Bad power supply on cr1-codfw(PEM 0) - https://phabricator.wikimedia.org/T329943 (10Papaul) p:05Triage→03High [18:06:40] 10netops, 10Infrastructure-Foundations, 10SRE, 10ops-codfw: codfw: Bad power supply on cr1-codfw(PEM 0) - https://phabricator.wikimedia.org/T329943 (10Papaul) ***** RMA DETAILS ***** RMA Number: R200447858 Defective Line Item Number: 100 Defective Serial Number: Defective Product ID: MX480-PWR2520-AC-S Def... [18:33:17] 10Wikimedia-Apache-configuration, 10DNS, 10SRE, 10Traffic-Icebox, 10MW-1.40-notes (1.40.0-wmf.24; 2023-02-20): Remove aliases `minnan` and `zh-cfr` for the Min Nan Wikipedia - https://phabricator.wikimedia.org/T230382 (10BCornwall) 05In progress→03Resolved Thanks to rzl for deploying this. The domain... [18:41:27] 10Traffic, 10SRE: Remove old and unused libvarnishapi - https://phabricator.wikimedia.org/T300247 (10BCornwall) 05In progress→03Resolved a:03BCornwall `varnish` replaces `libvarnishapi1` so it's been obsoleted in a packaging sense. Since we've re-imaged all the cp servers `libvarnishapi1` has been removed. [18:49:37] 10Traffic, 10DNS, 10SRE, 10Software-Licensing: Add LICENSE to operations/dns scripts - https://phabricator.wikimedia.org/T291323 (10BCornwall) p:05Medium→03Low [19:10:04] 10Traffic, 10DNS, 10SRE, 10Software-Licensing: Add LICENSE to operations/dns scripts - https://phabricator.wikimedia.org/T291323 (10BCornwall) a:03BCornwall Looks like @RKemper has also contributed since this was opened. I'm going to assume this is the best place to collect agreement to the license chang... [19:10:10] 10Traffic, 10DNS, 10SRE, 10Software-Licensing: Add LICENSE to operations/dns scripts - https://phabricator.wikimedia.org/T291323 (10BCornwall) 05Open→03In progress [19:14:56] 10Traffic, 10SRE, 10Data Pipelines (Sprint 08): Document Impact of Jan 8&9 Traffic Data Loss - https://phabricator.wikimedia.org/T326658 (10JArguello-WMF) 05Open→03Resolved [19:27:51] 10Traffic, 10SRE, 10serviceops: Feedback for new service IP flowchart - https://phabricator.wikimedia.org/T279296 (10BCornwall) 05Open→03Resolved a:03BCornwall Seeing as that flowchart is in use at https://wikitech.wikimedia.org/wiki/Wikimedia_network_guidelines and there's not been any activity for a... [19:29:31] 10Traffic, 10SRE: Package and deploy ATS 9.1.4 - https://phabricator.wikimedia.org/T325563 (10BCornwall) 05Open→03Resolved a:03BCornwall Confirmed via cumin that all hosts are running 9.1.4. Closing. [19:30:55] 10Traffic, 10SRE: Package and deploy ATS 9.1.4 - https://phabricator.wikimedia.org/T325563 (10BCornwall) [19:32:52] 10Traffic, 10SRE, 10Patch-For-Review: haproxy: work on systemd unit hardening (cp hosts) - https://phabricator.wikimedia.org/T323944 (10BCornwall) Looks like this still isn't rolled out based on my check on a random cp node. Still intend to roll this out? [21:10:07] 10Traffic, 10netops, 10DBA, 10Data-Persistence, and 9 others: codfw row B switches upgrade - https://phabricator.wikimedia.org/T327991 (10colewhite) [21:54:21] 10Traffic, 10Infrastructure-Foundations, 10SRE: Feature request: sre.hardware.upgrade-firmware should allow option to defer NIC firmware installation to next reboot - https://phabricator.wikimedia.org/T323717 (10BCornwall) 05Open→03Resolved a:03BCornwall Looks like this has been fixed, so I'll close. [22:55:30] 10HTTPS, 10Traffic, 10Diff-blog, 10SRE, 10Technical Blog: Send HSTS header on all Wordpress VIP-hosted domains - https://phabricator.wikimedia.org/T270034 (10Sbenchagra) @BCornwall the max-age has been increased to 106384710. Could you confirm all looks good?