[00:35:45] 10Traffic, 10Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice: Set expiry time for GeoIP cookies - https://phabricator.wikimedia.org/T122097 (10nshahquinn-wmf) I believe the GeoIP cookie is set by Varnish, which is maintained by #traffic. [09:04:57] 10Traffic, 10serviceops: Set per-request timeout on ATS-BE - https://phabricator.wikimedia.org/T315533 (10Vgutierrez) [09:05:23] 10Traffic, 10serviceops: Set per-request timeout on ATS-BE - https://phabricator.wikimedia.org/T315533 (10Vgutierrez) p:05Triage→03Medium [09:29:35] 10Traffic: Create program to interact with Atlas RIPE API - https://phabricator.wikimedia.org/T315536 (10MMandere) [10:20:56] (HAProxyEdgeTrafficDrop) firing: 54% request drop in text@codfw during the past 30 minutes - https://wikitech.wikimedia.org/wiki/Monitoring/EdgeTrafficDrop - https://grafana.wikimedia.org/d/000000479/frontend-traffic?viewPanel=12&orgId=1&from=now-24h&to=now&var-site=codfw&var-cache_type=text - https://alerts.wikimedia.org/?q=alertname%3DHAProxyEdgeTrafficDrop [10:25:56] (HAProxyEdgeTrafficDrop) resolved: 60% request drop in text@codfw during the past 30 minutes - https://wikitech.wikimedia.org/wiki/Monitoring/EdgeTrafficDrop - https://grafana.wikimedia.org/d/000000479/frontend-traffic?viewPanel=12&orgId=1&from=now-24h&to=now&var-site=codfw&var-cache_type=text - https://alerts.wikimedia.org/?q=alertname%3DHAProxyEdgeTrafficDrop [10:26:56] (HAProxyEdgeTrafficDrop) firing: 57% request drop in text@codfw during the past 30 minutes - https://wikitech.wikimedia.org/wiki/Monitoring/EdgeTrafficDrop - https://grafana.wikimedia.org/d/000000479/frontend-traffic?viewPanel=12&orgId=1&from=now-24h&to=now&var-site=codfw&var-cache_type=text - https://alerts.wikimedia.org/?q=alertname%3DHAProxyEdgeTrafficDrop [10:31:56] (HAProxyEdgeTrafficDrop) resolved: 67% request drop in text@codfw during the past 30 minutes - https://wikitech.wikimedia.org/wiki/Monitoring/EdgeTrafficDrop - https://grafana.wikimedia.org/d/000000479/frontend-traffic?viewPanel=12&orgId=1&from=now-24h&to=now&var-site=codfw&var-cache_type=text - https://alerts.wikimedia.org/?q=alertname%3DHAProxyEdgeTrafficDrop [10:33:56] (HAProxyEdgeTrafficDrop) firing: 57% request drop in text@codfw during the past 30 minutes - https://wikitech.wikimedia.org/wiki/Monitoring/EdgeTrafficDrop - https://grafana.wikimedia.org/d/000000479/frontend-traffic?viewPanel=12&orgId=1&from=now-24h&to=now&var-site=codfw&var-cache_type=text - https://alerts.wikimedia.org/?q=alertname%3DHAProxyEdgeTrafficDrop [10:38:56] (HAProxyEdgeTrafficDrop) resolved: 69% request drop in text@codfw during the past 30 minutes - https://wikitech.wikimedia.org/wiki/Monitoring/EdgeTrafficDrop - https://grafana.wikimedia.org/d/000000479/frontend-traffic?viewPanel=12&orgId=1&from=now-24h&to=now&var-site=codfw&var-cache_type=text - https://alerts.wikimedia.org/?q=alertname%3DHAProxyEdgeTrafficDrop [10:45:56] (HAProxyEdgeTrafficDrop) firing: 65% request drop in text@codfw during the past 30 minutes - https://wikitech.wikimedia.org/wiki/Monitoring/EdgeTrafficDrop - https://grafana.wikimedia.org/d/000000479/frontend-traffic?viewPanel=12&orgId=1&from=now-24h&to=now&var-site=codfw&var-cache_type=text - https://alerts.wikimedia.org/?q=alertname%3DHAProxyEdgeTrafficDrop [10:50:56] (HAProxyEdgeTrafficDrop) resolved: 65% request drop in text@codfw during the past 30 minutes - https://wikitech.wikimedia.org/wiki/Monitoring/EdgeTrafficDrop - https://grafana.wikimedia.org/d/000000479/frontend-traffic?viewPanel=12&orgId=1&from=now-24h&to=now&var-site=codfw&var-cache_type=text - https://alerts.wikimedia.org/?q=alertname%3DHAProxyEdgeTrafficDrop [12:19:58] 10netops, 10Infrastructure-Foundations, 10SRE, 10ops-eqiad: Link from lsw1-e1-eqiad to lsw1-f2-eqiad down - https://phabricator.wikimedia.org/T315052 (10cmooney) @Cmjohnson @Jclark-ctr can we get someone to visit the DC urgently to look at this? I'm concerned we have a link down between core devices for o... [13:35:47] Hello! I would like to acknowledge you guys that *.wmflabs.org has been blocked in China by the GFW since Aug 16 [13:35:48] * tranve_wiki[m] uploaded an image: (70KiB) < https://libera.ems.host/_matrix/media/r0/download/matrix.org/sypXneoPJNMsHDvVFjrFglYT/image.png > [13:35:59] from gfwatch.org [13:36:36] Also please see T297551 [13:37:04] tranve_wiki[m]: thanks for sharing, we are aware on both counts [13:38:01] thank you! [17:20:36] 10Traffic, 10Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice: Set expiry time for GeoIP cookies - https://phabricator.wikimedia.org/T122097 (10nshahquinn-wmf) [17:22:12] 10Traffic, 10Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice: Set expiry time for GeoIP cookies - https://phabricator.wikimedia.org/T122097 (10nshahquinn-wmf) Important comments from T175691: >>! In T175691#3601975, @AndyRussG wrote: > It's not a negligible number of users whose GeoIP location doe...