[04:16:46] 06Traffic, 10DNS, 06SRE, 10WikiLearn, 13Patch-For-Review: DNS records for WikiLearn - https://phabricator.wikimedia.org/T365435#9819594 (10Asaf) Yes, agreed. Our vendor made a mistake, and I pasted it verbatim. 😅 [08:53:16] 06Traffic, 06Data-Engineering, 10Observability-Logging: HAProxy should not log information we don't actually need - https://phabricator.wikimedia.org/T365566 (10Fabfur) 03NEW [10:23:30] 10netops, 06Infrastructure-Foundations, 06SRE: Move device AS numbers out of Homer YAML and source from Netbox - https://phabricator.wikimedia.org/T365572 (10cmooney) 03NEW p:05Triage→03Low [11:10:50] 10netops, 06Infrastructure-Foundations, 06SRE: Move device AS numbers out of Homer YAML and source from Netbox - https://phabricator.wikimedia.org/T365572#9820507 (10cmooney) The other thing that strikes me here is how to manage the individual-device ASNs, for instance 4265003001 on asw1-bw27-esams. And als... [12:50:48] 06Traffic, 13Patch-For-Review: Use IPIP encapsulation on lvs<-->upload cluster - https://phabricator.wikimedia.org/T357257#9820905 (10Vgutierrez) [13:13:58] 10netops, 06Infrastructure-Foundations, 13Patch-For-Review: Juniper: use export-format state-data json compact - https://phabricator.wikimedia.org/T362523#9820973 (10ayounsi) Tested on a MX204 running Junos 21.2 and 22.4R3.25, the returned JSON is invalid... See for example a basic `show interfaces xe-0/1/2... [13:47:20] 10netops, 06cloud-services-team, 06Infrastructure-Foundations, 06SRE, 13Patch-For-Review: Move WMCS servers to 1 single NIC - https://phabricator.wikimedia.org/T319184#9821206 (10aborrero) 05Open→03Stalled blocking until {T364984} is fixed, so we don't risk having another cloudvirt offline. [13:54:23] 06Traffic, 13Patch-For-Review: Use IPIP encapsulation on lvs<-->upload cluster - https://phabricator.wikimedia.org/T357257#9821308 (10Vgutierrez) [15:18:19] 06Traffic, 13Patch-For-Review: Use IPIP encapsulation on lvs<-->upload cluster - https://phabricator.wikimedia.org/T357257#9822008 (10Vgutierrez) [15:36:14] 06Traffic, 10DNS, 06SRE, 10WikiLearn, 13Patch-For-Review: DNS records for WikiLearn - https://phabricator.wikimedia.org/T365435#9822077 (10Dzahn) Hi @Asaf I have this change in code review now, adding 3 new values. https://gerrit.wikimedia.org/r/c/operations/dns/+/1034565/4/templates/learn.wiki I am... [15:37:56] 06Traffic, 10DNS, 06SRE, 10WikiLearn, 13Patch-For-Review: DNS records for WikiLearn - https://phabricator.wikimedia.org/T365435#9822081 (10Dzahn) 05Open→03In progress p:05Triage→03High [16:15:02] 06Traffic, 10DNS, 06SRE, 10WikiLearn, 13Patch-For-Review: DNS records for WikiLearn - https://phabricator.wikimedia.org/T365435#9822258 (10ssingh) If you want to send from the wikimedia.org domain, then yes, that's what you will need. There is no concern with the records themselves. [16:19:13] 06Traffic, 10DNS, 06SRE, 10WikiLearn, 13Patch-For-Review: DNS records for WikiLearn - https://phabricator.wikimedia.org/T365435#9822279 (10Asaf) In that case, I apologize for the confusion I created. We do want to send from comdevteam@wikimedia.org, so please amend the patch to the original CNAMEs reques... [16:20:23] 06Traffic, 10DNS, 06SRE, 10WikiLearn, 13Patch-For-Review: DNS records for WikiLearn - https://phabricator.wikimedia.org/T365435#9822302 (10Dzahn) 05In progress→03Open [16:24:27] 06Traffic, 06MW-Interfaces-Team, 06serviceops: map the /api/ prefix to /w/rest.php - https://phabricator.wikimedia.org/T364400#9822316 (10daniel) [16:29:46] 06Traffic, 06Content-Transform-Team, 06MW-Interfaces-Team, 10RESTBase Sunsetting: Remove long term caching and active purging for Parsoid endpoints in RESTbase - https://phabricator.wikimedia.org/T365630 (10daniel) 03NEW [16:30:02] 06Traffic, 06Content-Transform-Team, 06MW-Interfaces-Team, 10RESTBase Sunsetting: Remove long term caching and active purging for Parsoid endpoints in RESTbase - https://phabricator.wikimedia.org/T365630#9822376 (10daniel) [16:48:51] 06Traffic, 06Content-Transform-Team, 06MW-Interfaces-Team, 10RESTBase Sunsetting: Remove long term caching and active purging for Parsoid endpoints in RESTbase - https://phabricator.wikimedia.org/T365630#9822532 (10BBlack) I'm a little leery of dropping the TTL really-short. I get the argument for the nor... [17:07:33] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-codfw, and 2 others: Problem re-imaging hosts on row-wide vlan on EVPN switches - https://phabricator.wikimedia.org/T365204#9822720 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by cmooney@cumin1002 for host sretest2002.wikim... [17:08:48] 06Traffic, 06Content-Transform-Team, 06MW-Interfaces-Team, 10RESTBase Sunsetting: Remove long term caching and active purging for Parsoid endpoints in RESTbase - https://phabricator.wikimedia.org/T365630#9822721 (10daniel) >>! In T365630#9822532, @BBlack wrote: > Re-routing sounds better? Or perhaps even-... [17:24:10] 06Traffic, 10DNS, 06SRE, 10WikiLearn: DNS records for WikiLearn - https://phabricator.wikimedia.org/T365435#9822762 (10ssingh) Hi @jhathaway: I wanted to get your input about this. The request here is to add a DKIM record for wikimedia.org so that learn.wiki can allow sending email from comdevteam@wikimedi... [17:33:50] 06Traffic, 06Content-Transform-Team, 06MW-Interfaces-Team, 10RESTBase Sunsetting: Remove long term caching and active purging for Parsoid endpoints in RESTBase - https://phabricator.wikimedia.org/T365630#9822799 (10Krinkle) [17:34:00] 06Traffic, 06Content-Transform-Team, 06MW-Interfaces-Team, 10RESTBase Sunsetting: Remove long term caching and active purging for Parsoid endpoints in RESTBase - https://phabricator.wikimedia.org/T365630#9822797 (10Krinkle) [17:56:04] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-codfw, and 2 others: Problem re-imaging hosts on row-wide vlan on EVPN switches - https://phabricator.wikimedia.org/T365204#9822855 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by cmooney@cumin1002 for host sretest2002.wikimedia... [18:05:32] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-codfw, and 2 others: Problem re-imaging hosts on row-wide vlan on EVPN switches - https://phabricator.wikimedia.org/T365204#9822898 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by cmooney@cumin1002 for host sretest2002.wikim... [18:06:15] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-codfw, and 2 others: Problem re-imaging hosts on row-wide vlan on EVPN switches - https://phabricator.wikimedia.org/T365204#9822895 (10cmooney) Ok seems like we have a solution. I added the "forward-only" statement to the EVPN switches in codfw row A... [18:33:33] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-codfw, and 2 others: Problem re-imaging hosts on row-wide vlan on EVPN switches - https://phabricator.wikimedia.org/T365204#9822951 (10ops-monitoring-bot) cookbooks.sre.hosts.decommission executed by cmooney@cumin1002 for hosts: `sretest2002.wikimedia... [18:43:24] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-codfw, and 2 others: Problem re-imaging hosts on row-wide vlan on EVPN switches - https://phabricator.wikimedia.org/T365204#9822970 (10cmooney) @Papaul @Jhancock.wm I'm done with sretest2002 now and ran the decom cookbook so feel free to put it back i... [19:12:12] 06Traffic, 10DNS, 06SRE, 10WikiLearn: DNS records for WikiLearn - https://phabricator.wikimedia.org/T365435#9823026 (10jhathaway) @ssingh thanks for raising your concerns. I agree that our concerns are similar to those in T231387 and @mark's [recommendations](https://phabricator.wikimedia.org/T231387#54681... [21:32:16] 10Acme-chief: acme-chief: add support for serving individual files over the puppet file system api - https://phabricator.wikimedia.org/T364589#9823525 (10jhathaway) @Vgutierrez patch works great in prod, thanks!