[06:56:56] <jinxer-wm>	 (EdgeTrafficDrop) firing: 69% request drop in text@codfw during the past 30 minutes - https://wikitech.wikimedia.org/wiki/Monitoring/EdgeTrafficDrop - https://grafana.wikimedia.org/d/000000479/frontend-traffic?viewPanel=12&orgId=1&from=now-24h&to=now&var-site=codfw&var-cache_type=text - https://alerts.wikimedia.org
[07:01:56] <jinxer-wm>	 (EdgeTrafficDrop) resolved: 69% request drop in text@codfw during the past 30 minutes - https://wikitech.wikimedia.org/wiki/Monitoring/EdgeTrafficDrop - https://grafana.wikimedia.org/d/000000479/frontend-traffic?viewPanel=12&orgId=1&from=now-24h&to=now&var-site=codfw&var-cache_type=text - https://alerts.wikimedia.org
[10:28:34] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10SRE, 10Traffic-Icebox, 10Patch-For-Review: Collect netflow data for internal traffic - https://phabricator.wikimedia.org/T263277 (10ayounsi) I went the "set a different sampling pipeline for internal flows" way with the above POC for the reasons mentioned in T263...
[12:19:10] <wikibugs>	 10HTTPS, 10DBA, 10Privacy Engineering, 10SRE, and 3 others: dbtree loads third party resources (from google.com/jsapi) - https://phabricator.wikimedia.org/T96499 (10Marostegui) 05Stalled→03Declined We are going to deprecate tendril in favour of orchestrator, we've already opened it for people under NDA...
[19:14:46] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10SRE, 10Patch-For-Review, 10Sustainability (Incident Followup): Use next-hop-self for iBGP sessions - https://phabricator.wikimedia.org/T295672 (10cmooney) Earlier in the week I attempted to remove the "metric-out minimum-igp" from the iBGP session between cr1-eqi...
[23:16:56] <jinxer-wm>	 (EdgeTrafficDrop) firing: 63% request drop in text@esams during the past 30 minutes - https://wikitech.wikimedia.org/wiki/Monitoring/EdgeTrafficDrop - https://grafana.wikimedia.org/d/000000479/frontend-traffic?viewPanel=12&orgId=1&from=now-24h&to=now&var-site=esams&var-cache_type=text - https://alerts.wikimedia.org
[23:21:57] <jinxer-wm>	 (EdgeTrafficDrop) resolved: 61% request drop in text@esams during the past 30 minutes - https://wikitech.wikimedia.org/wiki/Monitoring/EdgeTrafficDrop - https://grafana.wikimedia.org/d/000000479/frontend-traffic?viewPanel=12&orgId=1&from=now-24h&to=now&var-site=esams&var-cache_type=text - https://alerts.wikimedia.org