[03:11:07] 10Traffic, 10MediaWiki-Uploading, 10SRE, 10serviceops: Unexpected upload speed to commons - https://phabricator.wikimedia.org/T288481 (10Krinkle) [07:51:45] 10Traffic, 10SRE, 10SRE Observability (FY2021/2022-Q1): Use Grizzly for Varnish SLO Grafana dashboard - https://phabricator.wikimedia.org/T289036 (10ema) >>! In T289036#7321876, @herron wrote: > Also, I updated the wikitech docs with this information as well as a hint to run 'grr preview' in these cases, whi... [09:18:32] 10Traffic, 10netops, 10Infrastructure-Foundations, 10SRE: Unable to load en.wikipedia.org from 84.19.61.192/26 - https://phabricator.wikimedia.org/T279503 (10A189605) Thanks. I'd say you can close this one down, thanks for you and your teams support. [09:21:03] 10Traffic, 10netops, 10Infrastructure-Foundations, 10SRE: Unable to load en.wikipedia.org from 84.19.61.192/26 - https://phabricator.wikimedia.org/T279503 (10ayounsi) 05Open→03Resolved a:03cmooney Great news! Out of curiosity, is it possible to know the root cause? Thanks [09:23:59] 10Traffic, 10netops, 10Infrastructure-Foundations, 10SRE: Unable to load en.wikipedia.org from 84.19.61.192/26 - https://phabricator.wikimedia.org/T279503 (10A189605) We're still not aware of the root cause, but it certainly isn't yourselves given some recent testing we've conducted. [09:43:56] (VarnishTrafficDrop) firing: 65% GET drop in text@ during the past 30 minutes - https://grafana.wikimedia.org/d/000000180/varnish-http-requests?viewPanel=6 - https://alerts.wikimedia.org [09:48:56] (VarnishTrafficDrop) resolved: 67% GET drop in text@ during the past 30 minutes - https://grafana.wikimedia.org/d/000000180/varnish-http-requests?viewPanel=6 - https://alerts.wikimedia.org [10:25:37] 10Traffic, 10SRE, 10Patch-For-Review: Test haproxy as a WMF's CDN TLS terminator with real traffic - https://phabricator.wikimedia.org/T290005 (10Vgutierrez) [17:38:39] 10Traffic, 10SRE, 10Patch-For-Review: Test haproxy as a WMF's CDN TLS terminator with real traffic - https://phabricator.wikimedia.org/T290005 (10Vgutierrez) [23:40:37] 10Traffic, 10SRE, 10PM: Clean up Traffic tag/workboard - https://phabricator.wikimedia.org/T289787 (10BBlack) #traffic-icebox now exists as a new tag with a process-informative description (click it and read!). I've bulk (+silent) moved all open #traffic tickets which had no activity for >= 6 months over to...