[08:28:39] 10Traffic, 10Data-Persistence, 10SRE-tools, 10Spicerack, 10serviceops: Switch conftool to use the version 3 etcd datastore - https://phabricator.wikimedia.org/T350565 (10Joe) [09:40:07] 10Traffic: Add custom HAProxy backend only for healthchecks - https://phabricator.wikimedia.org/T348851 (10Fabfur) [11:00:23] 10Traffic, 10SRE, 10GitLab (Project Migration): Migrate Traffic repositories from Gerrit to Gitlab - https://phabricator.wikimedia.org/T347623 (10LSobanski) Would operations/software/knead-wikidough and operations/software/liberica fit here as well? [11:00:27] 10Traffic: Provide a TCP MSS clamping mechanism for real servers - https://phabricator.wikimedia.org/T350462 (10Vgutierrez) @cmooney @ayounsi I'm aware that perfoming TCP MSS clamping at the network layer isn't a great solution from your point of view but could you elaborate on the downsides of doing it at that... [12:21:15] 10Traffic, 10Patch-For-Review: Add custom HAProxy backend only for healthchecks - https://phabricator.wikimedia.org/T348851 (10Fabfur) [12:28:02] 10netops, 10Infrastructure-Foundations, 10SRE: Support Anycast GW on EVPN switches without unique IP - https://phabricator.wikimedia.org/T350579 (10cmooney) p:05Triage→03Medium [13:13:58] 10netops, 10Infrastructure-Foundations, 10SRE, 10Patch-For-Review: Use default BGP multihop TTL between devices - https://phabricator.wikimedia.org/T350488 (10ayounsi) For cross-sites router to router we use the TTL value to eventually take down the session if the BGP session takes a too long path, it's cl... [13:55:45] 10netops, 10Infrastructure-Foundations, 10SRE, 10ops-eqiad: Move asw2-c8-eqiad to spares - https://phabricator.wikimedia.org/T349798 (10ayounsi) a:03Jclark-ctr [14:00:18] 10netops, 10Infrastructure-Foundations, 10SRE, 10ops-eqiad: Move asw2-c8-eqiad to spares - https://phabricator.wikimedia.org/T349798 (10ayounsi) Switch has been removed from config and powered off. All yours to do the remaining steps. I think https://netbox.wikimedia.org/dcim/cables/5708/ are 40G optics,... [14:04:01] 10netops, 10Infrastructure-Foundations, 10SRE: Packet Drops on Eqiad ASW -> CR uplinks - https://phabricator.wikimedia.org/T291627 (10ayounsi) [14:04:07] 10netops, 10Infrastructure-Foundations, 10SRE-Sprint-Week-Sustainability-March2023, 10ops-eqiad: eqiad: upgrade row C and D uplinks from 4x10G to 1x40G - https://phabricator.wikimedia.org/T313463 (10ayounsi) 05Resolved→03Open When working on something else I noticed that those were still in Netbox: htt... [14:05:55] 10netops, 10Infrastructure-Foundations, 10SRE, 10ops-eqiad: Move asw2-c8-eqiad to spares - https://phabricator.wikimedia.org/T349798 (10ayounsi) [14:26:03] 10netops, 10Infrastructure-Foundations, 10SRE, 10Patch-For-Review: Use default BGP multihop TTL between devices - https://phabricator.wikimedia.org/T350488 (10cmooney) 05Open→03Resolved >>! In T350488#9308379, @ayounsi wrote: > For cross-sites router to router we use the TTL value to eventually take do... [14:27:56] 10netops, 10Infrastructure-Foundations, 10SRE, 10Patch-For-Review: Use default BGP multihop TTL between devices - https://phabricator.wikimedia.org/T350488 (10cmooney) 05Resolved→03Open Eh not sure how I accidentally set this to resolved! [14:46:29] 10Traffic, 10Patch-For-Review: Add custom HAProxy backend only for healthchecks - https://phabricator.wikimedia.org/T348851 (10Fabfur) [15:16:59] 10Traffic, 10MW-on-K8s, 10SRE, 10serviceops, and 2 others: Move 25% of mediawiki external requests to mw on k8s - https://phabricator.wikimedia.org/T348122 (10Joe) >>! In T348122#9240973, @matmarex wrote: > The Kubernetes work so far has caused problems with cross-wiki Echo notifications (see T223413, T342... [17:03:39] 10netops, 10DC-Ops, 10Infrastructure-Foundations, 10SRE, 10ops-eqiad: eqiad: Connect IC-374549 - https://phabricator.wikimedia.org/T350504 (10Jclark-ctr) a:03VRiley-WMF [18:14:56] 10netops, 10DC-Ops, 10Infrastructure-Foundations, 10SRE, 10ops-eqiad: eqiad: Connect IC-374549 - https://phabricator.wikimedia.org/T350504 (10VRiley-WMF) Ran the cable and plugged it into requested ports. [18:36:39] 10Traffic, 10MW-on-K8s, 10SRE, 10serviceops, and 2 others: Move 25% of mediawiki external requests to mw on k8s - https://phabricator.wikimedia.org/T348122 (10matmarex) It seems it was the same cause, as both issues look fixed to me. Thanks! [19:23:39] 10netops, 10DC-Ops, 10Infrastructure-Foundations, 10SRE, 10ops-eqiad: eqiad: Connect IC-374549 - https://phabricator.wikimedia.org/T350504 (10cmooney) Thanks @VRiley-WMF. Right now we can't see the status as the port needs to be enabled for 100G. But that involves resetting PIC 0/1 completely which wil... [19:49:20] 10Traffic, 10DC-Ops, 10SRE, 10ops-esams: Q1:rack/setup/install new esams/knams hosts - https://phabricator.wikimedia.org/T344174 (10RobH) 05Open→03Resolved a:03RobH