[15:57:33] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10SRE: Upgrade Fastnetmon to 1.2.1 - https://phabricator.wikimedia.org/T271228 (10MoritzMuehlenhoff) @ayounsi I've built a backport of fastnetmon 1.2.1 for bullseye-wikimedia. It's not yet uploaded to apt.wikimedia.org, let's sync up for some smoke testing when you're...
[21:27:57] <jinxer-wm>	 (HAProxyEdgeTrafficDrop) firing: 60% request drop in text@esams during the past 30 minutes - https://wikitech.wikimedia.org/wiki/Monitoring/EdgeTrafficDrop - https://grafana.wikimedia.org/d/000000479/frontend-traffic?viewPanel=12&orgId=1&from=now-24h&to=now&var-site=esams&var-cache_type=text - https://alerts.wikimedia.org/?q=alertname%3DHAProxyEdgeTrafficDrop
[21:32:57] <jinxer-wm>	 (HAProxyEdgeTrafficDrop) firing: (3) 49% request drop in text@eqiad during the past 30 minutes - https://wikitech.wikimedia.org/wiki/Monitoring/EdgeTrafficDrop  - https://alerts.wikimedia.org/?q=alertname%3DHAProxyEdgeTrafficDrop
[21:37:57] <jinxer-wm>	 (HAProxyEdgeTrafficDrop) resolved: (3) 49% request drop in text@eqiad during the past 30 minutes - https://wikitech.wikimedia.org/wiki/Monitoring/EdgeTrafficDrop  - https://alerts.wikimedia.org/?q=alertname%3DHAProxyEdgeTrafficDrop
[21:55:57] <jinxer-wm>	 (HAProxyEdgeTrafficDrop) firing: 31% request drop in text@eqsin during the past 30 minutes - https://wikitech.wikimedia.org/wiki/Monitoring/EdgeTrafficDrop - https://grafana.wikimedia.org/d/000000479/frontend-traffic?viewPanel=12&orgId=1&from=now-24h&to=now&var-site=eqsin&var-cache_type=text - https://alerts.wikimedia.org/?q=alertname%3DHAProxyEdgeTrafficDrop
[22:00:57] <jinxer-wm>	 (HAProxyEdgeTrafficDrop) resolved: 39% request drop in text@eqsin during the past 30 minutes - https://wikitech.wikimedia.org/wiki/Monitoring/EdgeTrafficDrop - https://grafana.wikimedia.org/d/000000479/frontend-traffic?viewPanel=12&orgId=1&from=now-24h&to=now&var-site=eqsin&var-cache_type=text - https://alerts.wikimedia.org/?q=alertname%3DHAProxyEdgeTrafficDrop
[23:37:57] <jinxer-wm>	 (HAProxyEdgeTrafficDrop) firing: (3) 63% request drop in text@drmrs during the past 30 minutes - https://wikitech.wikimedia.org/wiki/Monitoring/EdgeTrafficDrop  - https://alerts.wikimedia.org/?q=alertname%3DHAProxyEdgeTrafficDrop
[23:42:57] <jinxer-wm>	 (HAProxyEdgeTrafficDrop) resolved: (4) 64% request drop in text@drmrs during the past 30 minutes - https://wikitech.wikimedia.org/wiki/Monitoring/EdgeTrafficDrop  - https://alerts.wikimedia.org/?q=alertname%3DHAProxyEdgeTrafficDrop
[23:56:11] <wikibugs>	 10Traffic, 10SRE, 10Wikimedia-Incident: All wikis down: error 503 (resolved, follow-up pending) - https://phabricator.wikimedia.org/T308940 (10Dzahn) 05Open→03Resolved a:03Dzahn https://wikitech.wikimedia.org/wiki/Incidents/2022-05-21_-_varnish_cache_busting