[08:20:03] 10netops, 06Infrastructure-Foundations, 10Prod-Kubernetes, 06ServiceOps: changeprop-jobqueue in codfw rows E & F has no IPv6 connectivity to kafka-main - https://phabricator.wikimedia.org/T433881#12181867 (10cmooney) Apologies for this one folks, it seems this was due to a misconfiguration on our end that'... [08:26:07] 06Traffic, 06Data-Persistence, 06MediaWiki-Media-Platform-Team, 13Patch-For-Review: Move thumbnail caching from upload cluster to text - https://phabricator.wikimedia.org/T427465#12181880 (10SLyngshede-WMF) Puppet failed when applying the first patch on magru: ` Aug 04 08:17:40 cp7009 varnishd[1901]: CLI... [08:35:39] 10netops, 06Infrastructure-Foundations, 06SRE: Consider removing BFD on datacentre IBGP peerings - https://phabricator.wikimedia.org/T433675#12181927 (10ayounsi) Sounds good to me! [09:19:22] 10netops, 06Infrastructure-Foundations, 10Prod-Kubernetes, 06ServiceOps, 13Patch-For-Review: changeprop-jobqueue in codfw rows E & F has no IPv6 connectivity to kafka-main - https://phabricator.wikimedia.org/T433881#12182149 (10jijiki) 05Open→03Resolved p:05Triage→03High a:03jijiki I revert... [09:35:40] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-eqiad, 06SRE: eqiad row A/B switch refresh prep - https://phabricator.wikimedia.org/T418012#12182272 (10cmooney) [12:57:32] hello traffic friends - I'm starting preparation for the main-etcd switchover. as part of that, you'll see a couple of operations/dns patches going by (FYI only). [12:58:35] Have fun :-) [13:22:16] hello traffic! I'd like to merge that[0] backend.yaml/text.yaml change to start testing gitlab's connectivity behind CDN. The custom values are detailed on the commit message. I was told to coordinate with you so here I am :) let me know if there is a preferred time for that kind of merge. [0] https://gerrit.wikimedia.org/r/c/operations/puppet/+/1290731 [14:33:32] 10netops, 06Infrastructure-Foundations, 10Prod-Kubernetes, 06ServiceOps: changeprop-jobqueue in codfw rows E & F has no IPv6 connectivity to kafka-main - https://phabricator.wikimedia.org/T433881#12183670 (10RLazarus) Thanks both! [14:49:18] 06Traffic, 06DC-Ops, 10ops-eqsin, 06SRE: cp5022 is unreachable - https://phabricator.wikimedia.org/T414411#12183764 (10RobH) pre-onsite checks: host mgmt is online and i can connect to it, but I'm having issues on its idrac interface. I want to push new ilom firmware to it before handoff, but having issue... [15:26:46] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-codfw, 06SRE: codfw: pod AB switches upgrade (2026) - https://phabricator.wikimedia.org/T426197#12183919 (10cmooney) [15:47:52] arnaudb: Sorry for the delay - sukhbir is out today, so I'd say maybe tomorrow would be better [15:48:30] If you're itching to get it done, I can be around though [15:48:50] it's just that s might have more context for everything than I do :) [15:49:38] its OK to wait for tomorrow, if Sukhbir is too busy I'll come and bother you if you have some time to spare [15:49:58] happy to. Thanks! [15:50:35] 06Traffic, 06DC-Ops, 10ops-drmrs: hw troubleshooting: Memory failure for cp6008.drmrs.wmnet - https://phabricator.wikimedia.org/T431651#12184021 (10RobH) Updates: Valerie shipped (3) replacement dimms, (1) for immediate use and (2) spares. https://www.dhl.com/us-en/home/tracking.html?tracking-id=15%204938%... [15:50:41] 06Traffic, 06DC-Ops, 10ops-drmrs: hw troubleshooting: Memory failure for cp6008.drmrs.wmnet - https://phabricator.wikimedia.org/T431651#12184022 (10RobH) a:03RobH [16:46:00] 06Traffic, 07Browser-Support-Apple-Safari, 07Browser-Support-Firefox, 07Browser-Support-Google-Chrome, 07User-notice: Discovery: Deprecation of TLS 1.2 - https://phabricator.wikimedia.org/T367821#12184332 (10Volker_E) [17:12:17] 06Traffic, 07Browser-Support-Apple-Safari, 07Browser-Support-Firefox, 07Browser-Support-Google-Chrome, 07User-notice: Discovery: Deprecation of TLS 1.2 - https://phabricator.wikimedia.org/T367821#12184471 (10BCornwall) @EMill-WMF & @Rsilvola: There's been some interest about the current state of things a... [18:19:47] 06Traffic, 10Beta-Cluster-Infrastructure: Puppet agent failure detected on instance deployment-cache-upload08 in project deployment-prep - https://phabricator.wikimedia.org/T434005#12184780 (10BCornwall) 05Open→03Resolved a:03BCornwall Caused by https://gerrit.wikimedia.org/r/c/operations/puppet/+/13... [18:28:11] 06Traffic: GeoDNS: consider sending CN to eqsin - https://phabricator.wikimedia.org/T378744#12184818 (10BCornwall) Friendly poke: I have https://gerrit.wikimedia.org/r/c/operations/dns/+/1306727 tee'd up. [18:31:21] 06Traffic, 13Patch-For-Review: varnish wikimedia_trust ACL isn't used anymore - https://phabricator.wikimedia.org/T399688#12184834 (10BCornwall) @joe Thanks for the feedback. I've since reworked it - would you mind a second look? Thanks. [19:36:58] 06Traffic, 07Browser-Support-Apple-Safari, 07Browser-Support-Firefox, 07Browser-Support-Google-Chrome, 07User-notice: Discovery: Deprecation of TLS 1.2 - https://phabricator.wikimedia.org/T367821#12185086 (10gh87) As I see, the task is still not yet "stalled" at this time. I can't believe you all that yo... [20:09:48] 06Traffic, 07Browser-Support-Apple-Safari, 07Browser-Support-Firefox, 07Browser-Support-Google-Chrome, 07User-notice: Discovery: Deprecation of TLS 1.2 - https://phabricator.wikimedia.org/T367821#12185127 (10TheDJ) >>! In T367821#12185086, @gh87 wrote: > As I see, the task is still not yet "stalled" at t... [21:14:19] 06Traffic, 10Maps, 06SRE: Possibility to allow Wikimedia Maps usage on all Wikibase Cloud instances - https://phabricator.wikimedia.org/T429191#12185282 (10MSantos) @Anton.Kokh, @ssingh or @Lydia_Pintscher do we have a sense of how much traffic that is going to bring to the maps servers? Usually I approve ad... [21:34:44] 06Traffic, 07Browser-Support-Apple-Safari, 07Browser-Support-Firefox, 07Browser-Support-Google-Chrome, 07User-notice: Discovery: Deprecation of TLS 1.2 - https://phabricator.wikimedia.org/T367821#12185358 (10BCornwall) Please also note that the deprecation takes a long time, too.