[07:39:26] 10Traffic, 10Maps, 10SRE: Allow Wikimedia Maps usage on Wikimedia Commons Android app - https://phabricator.wikimedia.org/T349280 (10Marostegui) [08:24:39] 10Acme-chief, 10Toolforge, 10cloud-services-team, 10Patch-For-Review: toolforge acme-chief: Failed to generate additional resources using 'eval_generate': Could not intern_multiple from application/json: 416: unexpected token at '{"checksum":{"type":"md5","val' - https://phabricator.wikimedia.org/T349384 (1... [09:00:11] 10netops, 10Infrastructure-Foundations, 10SRE: Put Dell SONiC switches in production - https://phabricator.wikimedia.org/T335028 (10ayounsi) [09:00:19] 10netops, 10Infrastructure-Foundations, 10SRE: Bring Juniper switches in eqiad racks E5-7 and F5-7 online and ready for servers - https://phabricator.wikimedia.org/T334230 (10ayounsi) [09:00:27] 10netops, 10DC-Ops, 10Infrastructure-Foundations, 10SRE, 10ops-eqiad: Cabling for Eqiad racks E5-8 and F5-8 - https://phabricator.wikimedia.org/T334231 (10ayounsi) 05Open→03Resolved Thanks! I'll open a different task if needed to debug any remaining issue, but looks like it's a Dell bug so far. [09:34:07] 10netops, 10Infrastructure-Foundations, 10SRE: Bring Juniper switches in eqiad racks E5-7 and F5-7 online and ready for servers - https://phabricator.wikimedia.org/T334230 (10cmooney) 05Open→03Resolved All devices online and ready for servers in their racks. [09:38:31] 10netops, 10Infrastructure-Foundations, 10SRE: CRs ECMP traffic to LVS VIPs despite higher MED on backup route - https://phabricator.wikimedia.org/T348446 (10cmooney) 05Open→03Resolved a:03cmooney Closing this task. As mentioned internal traffic may not always go to the activve LVS until we complete T... [10:05:04] 10Traffic, 10SRE, 10Patch-For-Review: Investigate IPVS IPIP encapsulation support - https://phabricator.wikimedia.org/T348837 (10cmooney) > Regarding UDP based protocols, DNS over UDP is usually capped at 512 bytes. While that was true at one stage, most DNS implementations now support [[ https://datatracke... [10:20:35] 10netops, 10Cloud-VPS, 10Infrastructure-Foundations, 10SRE, and 2 others: Change cloud-instance-transport vlan subnets from /30 to /29 - https://phabricator.wikimedia.org/T348140 (10cmooney) Commands for change later on: ` wmcs-openstack port unset ca4cb8c7-bfb8-440b-8e41-74bb8e834717 --fixed-ip subnet=clo... [10:31:26] 10netops, 10Infrastructure-Foundations, 10SRE: Automate L3 Switch to Core Router BGP peerings (and remove OSPF on drmrs switches) - https://phabricator.wikimedia.org/T349125 (10cmooney) 05Open→03Resolved [10:31:34] 10netops, 10Infrastructure-Foundations, 10SRE, 10Patch-For-Review: Consolidate Automation Templates for DC Switches - https://phabricator.wikimedia.org/T312635 (10cmooney) [10:32:11] 10netops, 10Infrastructure-Foundations, 10SRE: Add non-EVPN L3 Switch routing policy definitions to Homer - https://phabricator.wikimedia.org/T344601 (10cmooney) 05Open→03Resolved [10:32:19] 10netops, 10Infrastructure-Foundations, 10SRE, 10Patch-For-Review: Consolidate Automation Templates for DC Switches - https://phabricator.wikimedia.org/T312635 (10cmooney) [10:37:04] 10netops, 10Infrastructure-Foundations, 10SRE, 10Patch-For-Review: Consolidate Automation Templates for DC Switches - https://phabricator.wikimedia.org/T312635 (10cmooney) [10:47:22] 10netops, 10Infrastructure-Foundations, 10SRE: Announce internal/core routes from CRs to L3 switches - https://phabricator.wikimedia.org/T344547 (10cmooney) >>! In T344547#9108360, @ayounsi wrote: > ` > set policy-options policy-statement Switch_out term from_bgp from as-path core_and_local_LVS > set policy-... [11:31:49] XioNoX, topranks https://gitlab.wikimedia.org/vgutierrez/ipip-multiqueue-optimizer/ --> I've included prometheus metrics as well so it should be usable in production with the proper puppetization [11:34:32] vgutierrez: really cool stuff I was looking at it earlier [13:33:06] 10netops, 10Cloud-VPS, 10Infrastructure-Foundations, 10SRE, 10User-dcaro: Change cloud-instance-transport vlan subnets from /30 to /29 - https://phabricator.wikimedia.org/T348140 (10dcaro) [13:33:58] 10netops, 10Cloud-VPS, 10Infrastructure-Foundations, 10SRE, 10User-dcaro: Change cloud-instance-transport vlan subnets from /30 to /29 - https://phabricator.wikimedia.org/T348140 (10dcaro) 05In progress→03Resolved This went as expected, and all the changes have been applied :) Thanks a lot @cmooney ! [17:31:31] 10Traffic, 10SRE, 10Patch-For-Review: Upgrade Traffic hosts to bookworm - https://phabricator.wikimedia.org/T342154 (10BCornwall) [17:44:57] 10Traffic, 10SRE, 10Patch-For-Review: Upgrade Traffic hosts to bookworm - https://phabricator.wikimedia.org/T342154 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by brett@cumin2002 for host dns6002.wikimedia.org with OS bookworm [18:45:25] 10Traffic, 10SRE, 10Patch-For-Review: Upgrade Traffic hosts to bookworm - https://phabricator.wikimedia.org/T342154 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by brett@cumin2002 for host dns6002.wikimedia.org with OS bookworm completed: - dns6002 (**PASS**) - Downtimed on Icinga/Al... [18:48:27] 10netops, 10Data-Engineering, 10Infrastructure-Foundations, 10SRE, and 2 others: [Maintenance] Netflow/pmacct: use forwardingStatus - https://phabricator.wikimedia.org/T331707 (10Ahoelzl) [18:52:26] 10netops, 10Data-Engineering, 10Infrastructure-Foundations, 10SRE, and 2 others: [Maintenance] Netflow/pmacct: use forwardingStatus - https://phabricator.wikimedia.org/T331707 (10Ahoelzl) 05Open→03Resolved [18:52:41] 10netops, 10Data-Engineering, 10Infrastructure-Foundations, 10SRE, and 2 others: [Maintenance] Netflow/pmacct: use forwardingStatus - https://phabricator.wikimedia.org/T331707 (10Ahoelzl) [19:26:05] 10Traffic, 10SRE, 10Patch-For-Review: Upgrade Traffic hosts to bookworm - https://phabricator.wikimedia.org/T342154 (10BCornwall) [19:33:28] 10netops, 10Infrastructure-Foundations, 10SRE: Put Dell SONiC switches in production - https://phabricator.wikimedia.org/T335028 (10VRiley-WMF) [20:18:47] 10Traffic, 10SRE, 10Patch-For-Review: Upgrade Traffic hosts to bookworm - https://phabricator.wikimedia.org/T342154 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by brett@cumin2002 for host dns4004.wikimedia.org with OS bookworm [21:37:52] 10Traffic, 10SRE, 10Patch-For-Review: Upgrade Traffic hosts to bookworm - https://phabricator.wikimedia.org/T342154 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by brett@cumin2002 for host dns4004.wikimedia.org with OS bookworm completed: - dns4004 (**PASS**) - Downtimed on Icinga/Al... [22:06:29] 10Traffic, 10SRE, 10Patch-For-Review: Upgrade Traffic hosts to bookworm - https://phabricator.wikimedia.org/T342154 (10BCornwall)