[06:37:42] 10Traffic, 10netops, 10DBA, 10Data-Engineering, and 9 others: codfw row C switches upgrade - https://phabricator.wikimedia.org/T334049 (10Marostegui) @Ladsgroup During this operation, replication codfw -> eqiad is still active, so as there are codfw masters involved (even if codfw will be depooled), rememb... [12:57:37] 10Traffic, 10netops, 10DBA, 10Data-Engineering, and 9 others: codfw row C switches upgrade - https://phabricator.wikimedia.org/T334049 (10fgiunchedi) [13:44:03] 10Traffic, 10PyBal: Clean up debs/pybal branches - https://phabricator.wikimedia.org/T335455 (10Legoktm) [13:57:14] 10netops, 10Infrastructure-Foundations, 10SRE, 10observability, 10Patch-For-Review: Prometheus: ingest SONiC metrics - https://phabricator.wikimedia.org/T335027 (10fgiunchedi) >>! In T335027#8795348, @ayounsi wrote: > Thanks for the quick reply! This now works: > ` > prometheus1006:~$ curl lsw1-e8-eqiad.... [13:59:56] 10Traffic, 10PyBal: Clean up debs/pybal branches - https://phabricator.wikimedia.org/T335455 (10MoritzMuehlenhoff) When the patches get properly restarted, please make sure to also import the attached patch which was applied for the bullseye build (back then I didn't realise the repo was still in use since mas... [18:15:14] 10Traffic, 10PyBal: Clean up debs/pybal branches - https://phabricator.wikimedia.org/T335455 (10BCornwall) 05Open→03In progress a:03BCornwall I've removed the `ipv6`, `precise`, `mp-bgp` and branches as their commits have since been merged to - at least - 1.15-stretch (via `git log 1.15-stretch..... [18:55:30] 10Traffic, 10PyBal: Clean up debs/pybal branches - https://phabricator.wikimedia.org/T335455 (10MoritzMuehlenhoff) Or simply rename 1.15-stretch to "main" and designate it as the default branch in the git-buildpackage, all older branches can stick around in case anyone wants to dig up older patches. [19:03:01] 10Traffic: Write a cookbook to handle upgrades of ATS - https://phabricator.wikimedia.org/T335531 (10BCornwall) [19:03:47] 10Traffic: Write a cookbook to handle upgrades of ATS - https://phabricator.wikimedia.org/T335531 (10BCornwall) @KOfori Are we to automate the Packaging section in the [[ https://wikitech.wikimedia.org/wiki/Apache_Traffic_Server#Packaging | Wikitech documentation ]]? If so, do we want to maintain an abstraction... [19:04:12] 10Traffic: Write a cookbook to handle upgrades of ATS - https://phabricator.wikimedia.org/T335531 (10BCornwall) p:05Triage→03Medium [19:09:30] 10Traffic: Write a cookbook to handle restarts of Wikimedia DNS - https://phabricator.wikimedia.org/T335533 (10BCornwall) [19:11:05] 10Traffic: Write a cookbook to handle restarts of Wikimedia DNS - https://phabricator.wikimedia.org/T335533 (10BCornwall) p:05Triage→03Medium @ssingh Can you please let me know what restarting Wikimedia DNS entails? I'd like to also get information on why e.g. "systemctl restart svc1 svc2 svc3" is not a feas... [19:17:17] 10Traffic, 10Infrastructure-Foundations, 10SRE-tools: Cookbook to depool a site in AuthDNS - https://phabricator.wikimedia.org/T334048 (10BCornwall) @ayounsi Thanks for the report! I have a naive question: Would it be possible/more correct to interface confctl/etcd rather than a cookbook? That (by my observa... [19:41:40] 10Traffic, 10Infrastructure-Foundations, 10SRE-tools: Cookbook to depool a site in AuthDNS - https://phabricator.wikimedia.org/T334048 (10BBlack) I like this direction (etcd). It's not super-trivial, but we've complained a lot even internally about the lack of etcd support for depooling whole sites at the p... [23:08:10] 10netops, 10Infrastructure-Foundations, 10SRE, 10observability: Prometheus: ingest SONiC metrics - https://phabricator.wikimedia.org/T335027 (10colewhite)