[06:46:56] (EdgeTrafficDrop) firing: 68% request drop in text@codfw during the past 30 minutes - https://wikitech.wikimedia.org/wiki/Monitoring/EdgeTrafficDrop - https://grafana.wikimedia.org/d/000000479/frontend-traffic?viewPanel=12&orgId=1&from=now-24h&to=now&var-site=codfw&var-cache_type=text - https://alerts.wikimedia.org [06:51:56] (EdgeTrafficDrop) resolved: 69% request drop in text@codfw during the past 30 minutes - https://wikitech.wikimedia.org/wiki/Monitoring/EdgeTrafficDrop - https://grafana.wikimedia.org/d/000000479/frontend-traffic?viewPanel=12&orgId=1&from=now-24h&to=now&var-site=codfw&var-cache_type=text - https://alerts.wikimedia.org [06:58:26] (EdgeTrafficDrop) firing: 68% request drop in text@codfw during the past 30 minutes - https://wikitech.wikimedia.org/wiki/Monitoring/EdgeTrafficDrop - https://grafana.wikimedia.org/d/000000479/frontend-traffic?viewPanel=12&orgId=1&from=now-24h&to=now&var-site=codfw&var-cache_type=text - https://alerts.wikimedia.org [07:03:26] (EdgeTrafficDrop) resolved: 69% request drop in text@codfw during the past 30 minutes - https://wikitech.wikimedia.org/wiki/Monitoring/EdgeTrafficDrop - https://grafana.wikimedia.org/d/000000479/frontend-traffic?viewPanel=12&orgId=1&from=now-24h&to=now&var-site=codfw&var-cache_type=text - https://alerts.wikimedia.org [15:18:38] 10Acme-chief, 10User-dcaro, 10cloud-services-team (Kanban): acme-chief sometimes doesn't refresh certificates because it ignores SIGHUP - https://phabricator.wikimedia.org/T273956 (10Majavah) [15:19:25] 10Acme-chief, 10Toolforge, 10cloud-services-team (Kanban): problem with let's encrypt cert for star.tools.wmflabs.org - https://phabricator.wikimedia.org/T298353 (10Majavah) 05Open→03Resolved a:03Majavah [16:53:01] 10Traffic, 10Data-Engineering, 10Metrics-Platform, 10SRE: VarnishKafka to propagate user agent client hints headers to webrequest - https://phabricator.wikimedia.org/T299401 (10jbond) p:05Triage→03Medium [16:57:48] 10Traffic, 10DNS, 10SRE, 10WMSE (IT): Need Assistance adding DNS records to claim domain - https://phabricator.wikimedia.org/T300076 (10MRamirez_WMF) Hi @jbond , these required settings remain the same. Thanks for the assist! TXT name ‎@‎ (or skip if not supported by provider) TXT value Copy recordMS=08C5... [16:57:54] 10Traffic, 10SRE, 10SRE Observability (FY2021/2022-Q3), 10User-fgiunchedi: Migrate Traffic Prometheus alerts from Icinga to Alertmanager - https://phabricator.wikimedia.org/T300723 (10jbond) p:05Triage→03Medium [16:58:25] 10netops, 10Data-Engineering, 10Infrastructure-Foundations, 10Product-Analytics, and 2 others: Maybe restrict domains accessible by webproxy - https://phabricator.wikimedia.org/T300977 (10jbond) p:05Triage→03Low [17:00:11] 10netops, 10Discovery, 10Infrastructure-Foundations, 10SRE: Speed up network connections for Elastic hosts - https://phabricator.wikimedia.org/T301577 (10jbond) p:05Triage→03Medium [18:09:46] 10netops, 10Infrastructure-Foundations: Validate new (anycast) IPv6 /48 announcement being accepted by transits - https://phabricator.wikimedia.org/T301900 (10cmooney) p:05Triage→03Low [19:25:23] 10Traffic, 10DNS, 10SRE, 10Patch-For-Review, 10WMSE (IT): Need Assistance adding DNS records to claim domain - https://phabricator.wikimedia.org/T300076 (10jbond) I have created the change and will make sure it gets deployed tomorrow. > P.S I'll probably need to request addition of another set of DNS re... [19:25:34] 10Traffic, 10DNS, 10SRE, 10Patch-For-Review, 10WMSE (IT): Need Assistance adding DNS records to claim domain - https://phabricator.wikimedia.org/T300076 (10jbond) p:05Triage→03Medium [19:53:54] 10Traffic, 10netops, 10Infrastructure-Foundations, 10SRE, 10Patch-For-Review: Enable IPv6 for Wikidough - https://phabricator.wikimedia.org/T301165 (10cmooney) Up and running from esams :) ` cathal@nbgw:~$ dig -b 2001:470:1f09:32c::103 +nsid +https www.toutless.com @wikimedia-dns.org. ; <<>> DiG 9.17.2...