[06:50:56] (EdgeTrafficDrop) firing: 69% request drop in text@codfw during the past 30 minutes - https://wikitech.wikimedia.org/wiki/Monitoring/EdgeTrafficDrop - https://grafana.wikimedia.org/d/000000479/frontend-traffic?viewPanel=12&orgId=1&from=now-24h&to=now&var-site=codfw&var-cache_type=text - https://alerts.wikimedia.org [06:55:56] (EdgeTrafficDrop) resolved: 69% request drop in text@codfw during the past 30 minutes - https://wikitech.wikimedia.org/wiki/Monitoring/EdgeTrafficDrop - https://grafana.wikimedia.org/d/000000479/frontend-traffic?viewPanel=12&orgId=1&from=now-24h&to=now&var-site=codfw&var-cache_type=text - https://alerts.wikimedia.org [06:56:56] (EdgeTrafficDrop) firing: 69% request drop in text@codfw during the past 30 minutes - https://wikitech.wikimedia.org/wiki/Monitoring/EdgeTrafficDrop - https://grafana.wikimedia.org/d/000000479/frontend-traffic?viewPanel=12&orgId=1&from=now-24h&to=now&var-site=codfw&var-cache_type=text - https://alerts.wikimedia.org [07:01:56] (EdgeTrafficDrop) resolved: 69% request drop in text@codfw during the past 30 minutes - https://wikitech.wikimedia.org/wiki/Monitoring/EdgeTrafficDrop - https://grafana.wikimedia.org/d/000000479/frontend-traffic?viewPanel=12&orgId=1&from=now-24h&to=now&var-site=codfw&var-cache_type=text - https://alerts.wikimedia.org [09:37:39] 10Traffic, 10SRE, 10Patch-For-Review: Test envoyproxy as a WMF's CDN TLS terminator with real traffic - https://phabricator.wikimedia.org/T271421 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by vgutierrez@cumin1001 for host cp2040.codfw.wmnet with OS buster [10:16:35] 10Traffic, 10SRE, 10Patch-For-Review: Test envoyproxy as a WMF's CDN TLS terminator with real traffic - https://phabricator.wikimedia.org/T271421 (10Vgutierrez) [10:17:31] 10Traffic, 10SRE, 10Patch-For-Review: Test envoyproxy as a WMF's CDN TLS terminator with real traffic - https://phabricator.wikimedia.org/T271421 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by vgutierrez@cumin1001 for host cp2040.codfw.wmnet with OS buster completed: - cp2040 (**PASS*... [10:47:09] hi all just wanted to ping in here in case it was missed. WMCS plan to delete the traffic cloud project on the 1st feb cc vgutierrez sukhe [10:47:25] https://wikitech.wikimedia.org/wiki/News/Cloud_VPS_2021_Purge [10:49:38] yeah.. let me mark that project as active [10:49:43] and increase your beer counter jbond [10:49:54] thx <3 [10:50:15] :) np [11:04:02] 10Traffic, 10SRE, 10Patch-For-Review: Test envoyproxy as a WMF's CDN TLS terminator with real traffic - https://phabricator.wikimedia.org/T271421 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by vgutierrez@cumin1001 for host cp1088.eqiad.wmnet with OS buster [11:10:56] (VarnishPrometheusExporterDown) firing: Varnish Exporter on instance cp1088:9331 is unreachable - https://alerts.wikimedia.org [11:20:56] (VarnishPrometheusExporterDown) resolved: Varnish Exporter on instance cp1088:9331 is unreachable - https://alerts.wikimedia.org [11:35:56] (VarnishPrometheusExporterDown) firing: Varnish Exporter on instance cp1088:9331 is unreachable - https://alerts.wikimedia.org [11:40:56] (VarnishPrometheusExporterDown) resolved: Varnish Exporter on instance cp1088:9331 is unreachable - https://alerts.wikimedia.org [11:49:59] 10Traffic, 10SRE, 10Patch-For-Review: Test envoyproxy as a WMF's CDN TLS terminator with real traffic - https://phabricator.wikimedia.org/T271421 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by vgutierrez@cumin1001 for host cp1088.eqiad.wmnet with OS buster completed: - cp1088 (**WARN*... [11:51:44] 10Traffic, 10SRE, 10Patch-For-Review: Test envoyproxy as a WMF's CDN TLS terminator with real traffic - https://phabricator.wikimedia.org/T271421 (10Vgutierrez) [13:55:49] 10Traffic, 10Data-Engineering, 10Metrics-Platform, 10SRE, 10Patch-For-Review: VarnishKafka to propagate user agent client hints headers to webrequest - https://phabricator.wikimedia.org/T299401 (10EChetty) [17:17:18] 10HTTPS, 10Privacy Engineering, 10SRE, 10Traffic-Icebox, 10Security: Investigate our mitigation strategy for HTTPS response length attacks - https://phabricator.wikimedia.org/T92298 (10JFishback_WMF) [17:51:23] 10netops, 10DC-Ops, 10Infrastructure-Foundations, 10SRE, 10ops-eqiad: Q2:(Need By: TBD) Rows E/F network racking task - https://phabricator.wikimedia.org/T292095 (10cmooney) @Jclark-ctr I think we can assign the CR ports like this: ` cr1-eqiad et-1/0/2 ----> lsw1-e1-eqiad et-0/0/48 cr2-eqiad et-1/0/2...