[08:05:19] 06Traffic, 10Observability-Alerting, 13Patch-For-Review: urldownloader service should be a paging alert - https://phabricator.wikimedia.org/T435648#12306584 (10MatthewVernon) Possibly-silly question. The Wikitech page says "It is not a highly available service and as such no assumptions should be made in cod... [08:24:06] 06Traffic, 06MediaWiki-API-Platform-Team, 06MediaWiki-Core-Platform-Team, 10MediaWiki-File-management, and 5 others: Allow MediaWiki to make authenticated requests to other instances of MediaWiki - https://phabricator.wikimedia.org/T424768#12306626 (10Tgr) We should prioritize this (some time after we fix... [09:06:59] 06Traffic, 10Observability-Alerting, 13Patch-For-Review: urldownloader service should be a paging alert - https://phabricator.wikimedia.org/T435648#12306782 (10brouberol) @MatthewVernon FWIW, urldownloader was made highly available as part of https://phabricator.wikimedia.org/T429175 [09:34:04] 06Traffic, 06MediaWiki-API-Platform-Team, 10ServiceOps-SharedInfra, 06ServiceOps (Next quarter): map the /api/ prefix to /w/rest.php - https://phabricator.wikimedia.org/T364400#12306878 (10MLechvien-WMF) [09:44:25] \o Any objections to me merging https://gerrit.wikimedia.org/r/c/operations/dns/+/1337904 and running the usual authdns-update? [10:26:23] 06Traffic, 06MediaWiki-API-Platform-Team, 10ServiceOps-SharedInfra, 06ServiceOps (Next quarter): map the /api/ prefix to /w/rest.php - https://phabricator.wikimedia.org/T364400#12307074 (10Clement_Goubert) [10:35:12] 06Traffic, 06MediaWiki-API-Platform-Team, 10ServiceOps-SharedInfra, 06ServiceOps (Next quarter): map the /api/ prefix to /w/rest.php - https://phabricator.wikimedia.org/T364400#12307095 (10Clement_Goubert) 05Stalled→03Invalid I'm gonna be bold and close this task. Invalid is the closest status I ca... [10:42:20] klausman: That seems completely reasonable [11:22:48] ack, thx! [11:36:31] 10netops, 06Traffic, 06Infrastructure-Foundations, 06SRE, 07Wikimedia-Incident: Wikimedia's eqsin datacenter (Asia Pacific) had network connectivity issues - https://phabricator.wikimedia.org/T284986#12307394 (10MLechvien-WMF) [11:37:41] 06Traffic, 06MediaWiki-Core-Platform-Team, 10MediaWiki-libs-BagOStuff, 10Page Content Service, and 2 others: cache_text cluster consistently backlogged on purge requests - https://phabricator.wikimedia.org/T249325#12307418 (10MLechvien-WMF) [11:37:50] 06Traffic, 06MediaWiki-Core-Platform-Team, 10MediaWiki-ResourceLoader, 06SRE, 07Wikimedia-Incident: load.php?modules=startup miss rate tripled on 2020-02-05 - https://phabricator.wikimedia.org/T247020#12307430 (10MLechvien-WMF) [11:38:10] 10netops, 06Infrastructure-Foundations, 06SRE, 07Wikimedia-Incident: Improve resiliency of the eqsin transport link - https://phabricator.wikimedia.org/T236878#12307437 (10MLechvien-WMF) [11:38:31] 10netops, 06Infrastructure-Foundations, 06SRE, 07Wikimedia-Incident: asw2-d2-eqiad crash - https://phabricator.wikimedia.org/T233645#12307447 (10MLechvien-WMF) [11:39:18] 10netops, 06Infrastructure-Foundations, 06SRE, 07Wikimedia-Incident: asw2-a-eqiad FPC5 gets disconnected every 10 minutes - https://phabricator.wikimedia.org/T201145#12307473 (10MLechvien-WMF) [12:05:47] 06Traffic, 06SRE: Wikipedia and Wiktionary pages returning "upstream connect error or disconnect/reset before headers" - https://phabricator.wikimedia.org/T436004#12307605 (10MLechvien-WMF) [12:57:24] merged and updated [13:11:46] 06Traffic, 06MediaWiki-API-Platform-Team, 10MediaWiki-File-management, 10MediaWiki-General, and 5 others: Allow MediaWiki to make authenticated requests to other instances of MediaWiki - https://phabricator.wikimedia.org/T424768#12307933 (10JTweed-WMF) [13:59:39] 06Traffic, 10Liberica, 10Prod-Kubernetes, 06ServiceOps, and 2 others: Migrate Wikikube k8s apiserver and services to IPIP - https://phabricator.wikimedia.org/T420436#12308110 (10Jelto) [14:15:34] 06Traffic, 10Observability-Alerting, 13Patch-For-Review: urldownloader service should be a paging alert - https://phabricator.wikimedia.org/T435648#12308189 (10Blake) Before this is made paging, it might be helpful to provide additional context in https://wikitech.wikimedia.org/wiki/Url-downloader#Troublesho... [14:19:28] 06Traffic, 06MediaWiki-API-Platform-Team, 10MediaWiki-File-management, 10MediaWiki-General, and 5 others: Allow MediaWiki to make authenticated requests to other instances of MediaWiki - https://phabricator.wikimedia.org/T424768#12308225 (10Tgr) We plan to pick this up in a quarter or two. If someone wants... [16:14:57] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-eqiad, 06SRE: eqiad row A/B switch refresh prep - https://phabricator.wikimedia.org/T418012#12309022 (10RobH) IRC Sync update: * All switches except the lsw and ssw in A8 are racked, connected to SCS * I've added columns for the switch fiber cable... [16:34:51] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-eqiad, 06SRE: eqiad row A/B switch refresh prep - https://phabricator.wikimedia.org/T418012#12309116 (10cmooney) >>! In T418012#12309022, @RobH wrote: > * All switches except the lsw and ssw in A8 are racked, connected to SCS Can we confirm what po... [16:39:28] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-eqiad, 06SRE: eqiad row A/B switch refresh prep - https://phabricator.wikimedia.org/T418012#12309140 (10RobH) >>! In T418012#12309116, @cmooney wrote: >>>! In T418012#12309022, @RobH wrote: >> * All switches except the lsw and ssw in A8 are racked,...