[08:45:37] 10netops, 10Infrastructure-Foundations, 10SRE, 10Patch-For-Review: Add network-layer protections to avoid inadvertently lowering IRB MTU - https://phabricator.wikimedia.org/T329799 (10Peachey88) [09:47:39] 10netops, 10Infrastructure-Foundations, 10SRE, 10Patch-For-Review: Add network-layer protections to avoid inadvertently lowering IRB MTU - https://phabricator.wikimedia.org/T329799 (10cmooney) [09:48:10] 10netops, 10Infrastructure-Foundations, 10SRE, 10Patch-For-Review: Add network-layer protections to avoid inadvertently lowering IRB MTU - https://phabricator.wikimedia.org/T329799 (10cmooney) [09:49:40] vgutierrez: in case you didn't see them (as the patches have been parked for a while) https://gerrit.wikimedia.org/r/c/operations/dns/+/808198 and https://gerrit.wikimedia.org/r/c/operations/puppet/+/808199 are again ready for review [09:50:06] thanks for the ping [10:00:33] thanks for the quick review! [10:02:07] vgutierrez: in which order do they need to be merged/deployed? [10:05:15] [meeting] [10:14:06] I'd say DNS first [10:18:11] cool, thx! [14:36:22] 10Traffic, 10Analytics-Radar, 10Data-Engineering-Icebox, 10SRE: Requests to (hard) redirect pages return their target's contents but are counted as pageviews to the redirect page - https://phabricator.wikimedia.org/T125015 (10LSobanski) [15:17:31] 10Traffic: haproxy is able to load the same GeoIP & IP-to-ASN data as Varnish does - https://phabricator.wikimedia.org/T329849 (10CDanis) [15:17:38] 10Traffic: haproxy is able to load the same GeoIP & IP-to-ASN data as Varnish does - https://phabricator.wikimedia.org/T329849 (10CDanis) [16:31:21] 10netops, 10Infrastructure-Foundations, 10SRE, 10ops-codfw: codfw: Relocate servers to make space for new switches in rowA and rowB - https://phabricator.wikimedia.org/T326564 (10Papaul) [16:56:55] 10Traffic, 10netops, 10Data-Persistence-Backup, 10SRE, and 2 others: Depool codfw swift cluster - https://phabricator.wikimedia.org/T267338 (10jcrespo) [17:31:49] 10HTTPS, 10Diff-blog, 10SRE, 10Technical Blog: Send HSTS header on all Wordpress VIP-hosted domains - https://phabricator.wikimedia.org/T270034 (10Sbenchagra) @Dzahn techblog.wikimedia.org should be good. Could you confirm? I am looking into: diff.wikimedia.org wikimediaendowment.org one.wikimedia.org [17:48:09] 10Traffic, 10Infrastructure-Foundations, 10SRE, 10SRE-tools, 10IPv6: Some Traffic clusters apparently do not support IPv6 - https://phabricator.wikimedia.org/T271144 (10BCornwall) [18:52:00] 10HTTPS, 10Diff-blog, 10SRE, 10Technical Blog: Send HSTS header on all Wordpress VIP-hosted domains - https://phabricator.wikimedia.org/T270034 (10Dzahn) Hi @Sbenchagra, thank you very much for that. Yes, I can confirm generally all 4 have HSTS headers now/meanwhile. That is great! There is some detail t... [18:55:14] 10HTTPS, 10Diff-blog, 10SRE, 10Technical Blog: Send HSTS header on all Wordpress VIP-hosted domains - https://phabricator.wikimedia.org/T270034 (10Dzahn) @BCornwall I realize "Traffic-Icebox" has been removed but consulting input from traffic would still be valuable for this one. Should we close it as reso... [18:59:25] 10HTTPS, 10Diff-blog, 10SRE, 10Technical Blog: Send HSTS header on all Wordpress VIP-hosted domains - https://phabricator.wikimedia.org/T270034 (10CKoerner_WMF) [19:03:10] 10HTTPS, 10Diff-blog, 10SRE, 10Technical Blog: Send HSTS header on all Wordpress VIP-hosted domains - https://phabricator.wikimedia.org/T270034 (10Sbenchagra) @Dzahn techblog should now have "includeSubdomains;preload" [19:09:44] 10HTTPS, 10Diff-blog, 10SRE, 10Technical Blog: Send HSTS header on all Wordpress VIP-hosted domains - https://phabricator.wikimedia.org/T270034 (10Sbenchagra) [19:12:50] 10HTTPS, 10Diff-blog, 10SRE, 10Technical Blog: Send HSTS header on all Wordpress VIP-hosted domains - https://phabricator.wikimedia.org/T270034 (10BCornwall) [19:14:38] 10HTTPS, 10Diff-blog, 10SRE, 10Technical Blog: Send HSTS header on all Wordpress VIP-hosted domains - https://phabricator.wikimedia.org/T270034 (10BCornwall) @Dzahn Happy to give consultation where needed but since we don't manage any of the sites I figured we needn't be added. Truthfully, I'm not sure who... [19:15:09] 10Traffic, 10SRE, 10SRE-swift-storage: Revisit CDN<-->Swift communication - https://phabricator.wikimedia.org/T317616 (10Aklapper) [19:16:58] 10HTTPS, 10Diff-blog, 10SRE, 10Technical Blog: Send HSTS header on all Wordpress VIP-hosted domains - https://phabricator.wikimedia.org/T270034 (10Sbenchagra) [19:17:19] 10HTTPS, 10Diff-blog, 10SRE, 10Technical Blog: Send HSTS header on all Wordpress VIP-hosted domains - https://phabricator.wikimedia.org/T270034 (10Sbenchagra) soundlogo.wikimedia.org is now done! [19:38:11] 10HTTPS, 10Traffic, 10Diff-blog, 10SRE, 10Technical Blog: Send HSTS header on all Wordpress VIP-hosted domains - https://phabricator.wikimedia.org/T270034 (10BCornwall) [19:43:48] 10HTTPS, 10Traffic, 10Diff-blog, 10SRE, 10Technical Blog: Send HSTS header on all Wordpress VIP-hosted domains - https://phabricator.wikimedia.org/T270034 (10BCornwall) Hi, @Sbenchagra, thanks for doing this. I'm still unable to confirm that soundlogo has `;includeSubdomains;preload` in the header. Maybe... [19:50:04] 10HTTPS, 10Traffic, 10Diff-blog, 10SRE, 10Technical Blog: Send HSTS header on all Wordpress VIP-hosted domains - https://phabricator.wikimedia.org/T270034 (10Sbenchagra) Hi @BCornwall, Not a forgotten "save" click. There might be a delay, since it was done about 30 minutes ago. [20:06:18] 10HTTPS, 10Traffic, 10Diff-blog, 10SRE, 10Technical Blog: Send HSTS header on all Wordpress VIP-hosted domains - https://phabricator.wikimedia.org/T270034 (10BCornwall) Yep, seems to have been a delay. It's active now! Thanks for doing all that, @Sbenchagra Hate to be a pest, but would you also be willi... [20:10:01] 10HTTPS, 10Traffic, 10Diff-blog, 10SRE, 10Technical Blog: Send HSTS header on all Wordpress VIP-hosted domains - https://phabricator.wikimedia.org/T270034 (10Sbenchagra) Great! No worries @BCornwall! Let me get back to you on that. Thank you [20:10:31] 10HTTPS, 10Traffic, 10Diff-blog, 10SRE, 10Technical Blog: Send HSTS header on all Wordpress VIP-hosted domains - https://phabricator.wikimedia.org/T270034 (10BCornwall) a:03Sbenchagra [20:19:32] 10Traffic, 10SRE: HTTP/2 requests fail with too-long URLs - https://phabricator.wikimedia.org/T209590 (10BCornwall) 05Open→03Resolved a:03BCornwall I can confirm that the stack nowadays happily works the same with H1 and H2. A 414 is also returned using the hyper module as Valentin did above. [20:28:23] 10Traffic, 10SRE, 10SRE Observability: Investigate cp5006 crash - https://phabricator.wikimedia.org/T292506 (10BCornwall) 05Open→03Invalid @Vgutierrez, @BBlack, @ssingh This task seems quite too old to really act upon since the context is likely lost (not to mention that ats-tls isn't used any more). I d... [22:18:52] 10Traffic, 10Observability-Alerting, 10SRE: Move (or delete?) trafficserver restart count alert from icinga to alerts.git - https://phabricator.wikimedia.org/T327791 (10BCornwall) a:03BCornwall Forgive me if I'm off base but hasn't this already been done with T300723? We merged in https://gerrit.wikimedia.... [22:31:41] 10Traffic, 10Observability-Alerting, 10SRE: Move (or delete?) trafficserver restart count alert from icinga to alerts.git - https://phabricator.wikimedia.org/T327791 (10BCornwall) I see, I had forgotten to remove it from puppet. I've created https://gerrit.wikimedia.org/r/889881 to address that. [22:37:49] 10Traffic, 10SRE: Package and deploy ATS 9.1.4 - https://phabricator.wikimedia.org/T325563 (10BCornwall) Since 9.1.4 seems to be active on all cp hosts, should this and T325726 be closed? [22:43:34] 10Traffic, 10SRE, 10PM: Clean up Traffic tag/workboard - https://phabricator.wikimedia.org/T289787 (10BCornwall) I think that part of the effort of getting our board(s) under control is redefining a ticket's anatomy. I believe this ticket is too broad and more appropriately belongs in as OKRs or general plan... [22:44:43] 10Traffic, 10SRE, 10PM: Clean up Traffic tag/workboard - https://phabricator.wikimedia.org/T289787 (10BCornwall) To clarify, I can understand the usage of encompassing, "epic"-style tickets; I view those as separate functions than orphaned mega-tickets. [22:46:57] 10Traffic, 10SRE: Let's Encrypt issuance chains update - https://phabricator.wikimedia.org/T283164 (10BCornwall) Seeing as the event has passed and all subtasks are closed, is this ready to be closed? [22:49:48] 10Traffic, 10SRE, 10serviceops: Upgrade envoyproxy to 1.16.2 - https://phabricator.wikimedia.org/T271407 (10BCornwall) Envoy seems to be on 1.18.2 now. Can this be closed, or was there any other deployment need this ticket addresses? [22:50:52] 10Traffic, 10SRE: Prometheus Varnish exporter alert: add runbook and link to dashboard - https://phabricator.wikimedia.org/T289974 (10BCornwall) a:03BCornwall [23:03:09] 10Traffic, 10SRE: Prometheus Varnish exporter alert: add runbook and link to dashboard - https://phabricator.wikimedia.org/T289974 (10BCornwall) 05Open→03In progress [23:13:23] 10Traffic, 10SRE: Package and deploy varnish 6.0.11 - https://phabricator.wikimedia.org/T326634 (10BCornwall) 05Open→03Resolved a:03BCornwall Looks like this has been rolled out to all cp nodes. Great job! [23:13:32] 10Traffic, 10SRE: Package and deploy varnish 6.0.11 - https://phabricator.wikimedia.org/T326634 (10BCornwall)