[12:50:27] <wikibugs>	 10SRE-tools, 10Infrastructure-Foundations, 10SRE, 10serviceops, 10Documentation: Documentation updates in decom workflow - https://phabricator.wikimedia.org/T287388 (10Aklapper)
[13:04:17] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10Observability-Metrics, 10SRE, 10Patch-For-Review: add traceroute measurements to RIPE Atlas prometheus data - https://phabricator.wikimedia.org/T251156 (10ayounsi) 05Open→03Resolved a:05CDanis→03ayounsi This is done, opened T299640 for further improvements.
[14:09:18] <wikibugs>	 10Puppet, 10Infrastructure-Foundations, 10SRE, 10Patch-For-Review, 10User-jbond: Hieradata yaml style checking - https://phabricator.wikimedia.org/T236954 (10Volans) Although not optimal, in the worse case scenario in which we will be unable to find/modify a tool to preserve empty lines, we could also co...
[15:19:12] <klausman>	 topranks: so I saw this on my ISP's RSS feed just now: https://as13030.net/status/?ticket=4255027
[15:19:32] <klausman>	 And wouldn't you know it, going via bast1003 to a prod host now is fast for both v4 and v6!
[15:20:01] <topranks>	 Ah ok
[15:20:37] <cdanis>	 FWIW I personally disagree with the SSH config as we ship it and I think it's generally better to always go to one's closest bastion hosts, wherever your traffic is destined for within production
[15:20:39] <klausman>	 Thoigh neither MIA nor LAX showed up in my traces
[15:20:39] <cdanis>	 :)
[15:20:54] <topranks>	 Mia1 was in the traceroute, I’m assuming it’s minneapolis.
[15:21:00] <klausman>	 I will stay with the AMS bastion
[15:21:03] <topranks>	 So yeah that makes a lot of sense
[15:21:15] <topranks>	 They're in the trace back.
[15:21:36] <klausman>	 Ah, right
[15:22:27] <topranks>	 https://phabricator.wikimedia.org/P18815
[15:23:40] <klausman>	 I had only looked at the forward traces, so I missed that
[15:23:42] <topranks>	 Hop 4 there must be LA, as in Los Angelas, given LAX is mentioned by Init7.
[15:24:01] <topranks>	 So I guess it confirms the reverse path was the issue.
[15:24:22] <klausman>	 Welp, problem solved, twice over :)
[15:24:42] <topranks>	 Yep you can see it's changed now
[15:24:42] <topranks>	 https://phabricator.wikimedia.org/P18948
[15:24:53] <topranks>	 Either way I'll chase up with them on the direct peering in eqiad.
[15:25:35] <topranks>	 Problem link or not, going from Virginia to Los Angeles to get to Europe is bad navigation!
[15:26:34] <klausman>	 Aye!
[15:27:04] <klausman>	 Though geographic proximity is not always a good stand-in for network proximity ;)
[15:27:14] <klausman>	 yes, I am aware of c0 
[15:27:35] <topranks>	 Then you're ahead of me.... c0???
[15:27:56] <klausman>	 Speed of light in a vacuum
[15:28:55] <topranks>	 "Speed of light in a fiber that just needs to stop off at a few places on the way home" is unfortunately different yep!
[23:05:03] <wikibugs>	 10netops, 10DC-Ops, 10Infrastructure-Foundations, 10SRE, 10ops-eqiad: Q2:(Need By: TBD) Rows E/F network racking task - https://phabricator.wikimedia.org/T292095 (10Jclark-ctr)