[02:39:57] (VarnishTrafficDrop) firing: 68% GET drop in text@ during the past 30 minutes - https://grafana.wikimedia.org/d/000000180/varnish-http-requests?viewPanel=6 - https://alerts.wikimedia.org [02:44:57] (VarnishTrafficDrop) resolved: 69% GET drop in text@ during the past 30 minutes - https://grafana.wikimedia.org/d/000000180/varnish-http-requests?viewPanel=6 - https://alerts.wikimedia.org [09:27:57] (VarnishTrafficDrop) firing: 64% GET drop in text@ulsfo during the past 30 minutes - https://grafana.wikimedia.org/d/000000180/varnish-http-requests?viewPanel=6 - https://alerts.wikimedia.org [09:32:57] (VarnishTrafficDrop) resolved: 69% GET drop in text@ulsfo during the past 30 minutes - https://grafana.wikimedia.org/d/000000180/varnish-http-requests?viewPanel=6 - https://alerts.wikimedia.org [09:36:44] bblack: the alerting patch worked, we've got DC names back into the alerts ^ \o/ [09:39:17] 10Traffic, 10SRE, 10SRE Observability: VarnishTrafficDrop IRC alert does not include DC name anymore - https://phabricator.wikimedia.org/T291149 (10ema) 05Open→03Resolved a:03ema >>! In T291149#7361022, @gerritbot wrote: > %%%[operations/alerts@master] VarnishTrafficDrop: fix site label in summary%%% >... [13:48:07] 10Traffic, 10netops, 10Infrastructure-Foundations, 10SRE: TATA SKY Broadband (AS134674) issues with connecting to upload.wikimedia.org - https://phabricator.wikimedia.org/T275234 (10Perryprog) This issue has been reported on Znuny on Ticket#2021091710000804, so it's safe to assume the problem is still pres... [14:42:28] 10Traffic, 10netops, 10Infrastructure-Foundations, 10SRE: TATA SKY Broadband (AS134674) issues with connecting to upload.wikimedia.org - https://phabricator.wikimedia.org/T275234 (10cmooney) Recent email exchange on this as they contacted us again: > From: Cathal Mooney [mailto:cmooney@wikimedia.org] > Se... [15:28:57] (VarnishTrafficDrop) firing: 58% GET drop in text@codfw during the past 30 minutes - https://grafana.wikimedia.org/d/000000180/varnish-http-requests?viewPanel=6 - https://alerts.wikimedia.org [15:33:57] (VarnishTrafficDrop) resolved: 65% GET drop in text@codfw during the past 30 minutes - https://grafana.wikimedia.org/d/000000180/varnish-http-requests?viewPanel=6 - https://alerts.wikimedia.org [21:46:57] (VarnishTrafficDrop) firing: 61% GET drop in text@eqsin during the past 30 minutes - https://grafana.wikimedia.org/d/000000180/varnish-http-requests?viewPanel=6 - https://alerts.wikimedia.org [21:51:57] (VarnishTrafficDrop) resolved: 68% GET drop in text@eqsin during the past 30 minutes - https://grafana.wikimedia.org/d/000000180/varnish-http-requests?viewPanel=6 - https://alerts.wikimedia.org [22:05:23] hi team :) next week I want to add 3 new LVSs for shellbox: https://gerrit.wikimedia.org/r/q/topic:shellbox2-lvs [22:05:44] I'll ping on Monday EU morning to figure out a good time to do this