[10:08:56] 10netops, 10DC-Ops, 10Infrastructure-Foundations, 10SRE, and 2 others: Q1:(Need By: TBD) rack/setup/install cloudswift100[12] - https://phabricator.wikimedia.org/T289882 (10ayounsi) @cmooney, I agree with your take on the security aspect. We're not in a typical service provider (ISP)/customer relationship... [10:35:48] 10netops, 10DC-Ops, 10Infrastructure-Foundations, 10SRE, and 2 others: Q1:(Need By: TBD) rack/setup/install cloudswift100[12] - https://phabricator.wikimedia.org/T289882 (10aborrero) From a certain point of view what we're doing here is validating [[https://wikitech.wikimedia.org/wiki/Cross-Realm_traffic_g... [11:47:47] 10netops, 10Infrastructure-Foundations, 10SRE: Use next-hop-self for iBGP sessions - https://phabricator.wikimedia.org/T295672 (10cmooney) a:03cmooney So of course there is a complication. Currently we have a single BGP session between adjacent CR routers, peered over the loopback IPv4 addresses either si... [12:22:21] 10SRE-tools, 10netops, 10Infrastructure-Foundations: Netbox - PuppetDB audit 2021-11 - https://phabricator.wikimedia.org/T295762 (10Volans) 05Open→03In progress p:05Triage→03Medium [12:31:01] 10SRE-tools, 10netops, 10Analytics, 10Infrastructure-Foundations: an-worker hosts: Netbox - PuppetDB interfaces discrepancies - https://phabricator.wikimedia.org/T295763 (10Volans) [12:31:13] 10SRE-tools, 10netops, 10Analytics, 10Infrastructure-Foundations: an-worker hosts: Netbox - PuppetDB interfaces discrepancies - https://phabricator.wikimedia.org/T295763 (10Volans) p:05Triage→03Medium [13:18:26] 10netops, 10Infrastructure-Foundations, 10SRE: Rebuild ping* hosts with 10G disks - https://phabricator.wikimedia.org/T295767 (10MoritzMuehlenhoff) [14:05:18] 10SRE-tools, 10netops, 10Cloud-Services, 10Infrastructure-Foundations, 10SRE: WMCS VIPs: Netbox netmask inconsistencies - https://phabricator.wikimedia.org/T295774 (10Volans) [14:15:01] 10SRE-tools, 10netops, 10Cloud-Services, 10Infrastructure-Foundations, 10SRE: cloudnet VLAN Netbox discrepancies - https://phabricator.wikimedia.org/T295776 (10Volans) [14:18:08] 10netops, 10Infrastructure-Foundations, 10SRE, 10Patch-For-Review: Rebuild Routinator (rpki) VMs with larger disk - https://phabricator.wikimedia.org/T292503 (10ops-monitoring-bot) cookbooks.sre.hosts.decommission executed by cmooney@cumin2002 for hosts: `rpki2001.codfw.wmnet` - rpki2001.codfw.wmnet (**PAS... [14:19:04] 10SRE-tools, 10netops, 10Analytics, 10Infrastructure-Foundations, 10SRE: an-worker hosts: Netbox - PuppetDB interfaces discrepancies - https://phabricator.wikimedia.org/T295763 (10elukey) The only recent thing that I recall is T276239, but not for all workers mentioned. I checked quickly the dry-run for... [15:04:35] 10netops, 10Infrastructure-Foundations, 10SRE: Use next-hop-self for iBGP sessions - https://phabricator.wikimedia.org/T295672 (10ayounsi) For the sake of completeness, another option could be to add the fffff: IP to the loopback address, but that would be more of a workaround than a long term solution.... [15:13:42] 10SRE-tools, 10netops, 10Infrastructure-Foundations, 10SRE: Netbox - PuppetDB audit 2021-11 - https://phabricator.wikimedia.org/T295762 (10Volans) [15:13:48] 10SRE-tools, 10netops, 10Cloud-Services, 10Infrastructure-Foundations, 10SRE: cloudnet VLAN Netbox discrepancies - https://phabricator.wikimedia.org/T295776 (10Volans) 05Open→03Resolved a:03Volans After verifying that the changes were all expected and the VLAN bits were actually an artifact of how... [15:15:49] 10netops, 10Infrastructure-Foundations, 10SRE, 10fundraising-tech-ops: Upgrade pfw to Junos 20+ - https://phabricator.wikimedia.org/T295691 (10Jgreen) @ayounsi I think it would be fine to do the codfw pfw's this year. Please ping me on IRC when you have some time to discuss. [15:21:51] 10SRE-tools, 10netops, 10Analytics, 10Infrastructure-Foundations, 10SRE: an-worker hosts: Netbox - PuppetDB interfaces discrepancies - https://phabricator.wikimedia.org/T295763 (10BTullis) Yes I thought this was a bit odd. I saw there was a bit of re-imaging here: T231067#6891049 but that was before my t... [15:22:08] 10SRE-tools, 10netops, 10Analytics, 10Data-Engineering, and 3 others: an-worker hosts: Netbox - PuppetDB interfaces discrepancies - https://phabricator.wikimedia.org/T295763 (10BTullis) a:03BTullis [16:07:06] 10SRE-tools, 10netops, 10Analytics, 10Data-Engineering, and 3 others: an-worker hosts: Netbox - PuppetDB interfaces discrepancies - https://phabricator.wikimedia.org/T295763 (10BTullis) ==an-worker1104== ====Current interfaces snapshot: {F34750374,width=600} ====Current interfaces: * eno1 - SFTP+ - connect... [17:08:50] topranks, volans, looks like nautobot is getting capirca integration - https://youtu.be/3yq5v2fbTcI?t=883 [17:09:40] :) [17:38:16] 10netops, 10Infrastructure-Foundations, 10SRE, 10ops-codfw: Can't commit on asw-b-codfw - https://phabricator.wikimedia.org/T295118 (10Papaul) I downgrade Junos on QFX5100 at https://netbox.wikimedia.org/dcim/rack-elevations/ and did a request system zeroize on it . This is the one we will be using to repl... [17:39:04] XioNoX: that is awesome. Doesn’t make the task of choosing in any way easier!! [17:40:31] yeah, not sure it's compatible with Juniper, also we don't have much FW needs, so text files are easier to review, audit, etc... [17:41:09] to me the main takeaway is that there are more and more people using Capirca (not just google). So it's easier to trust the tool [17:44:43] 10SRE-tools, 10netops, 10Infrastructure-Foundations, 10SRE, 10serviceops: Support services VIPs with not marked as VIP in Netbox - https://phabricator.wikimedia.org/T295793 (10Volans) [17:46:20] 10netops, 10Infrastructure-Foundations, 10SRE, 10ops-codfw: Can't commit on asw-b-codfw - https://phabricator.wikimedia.org/T295118 (10ayounsi) That works for me, thanks, can you send a calendar invite? Note that the link in your comment doesn't point to any specific device. [18:00:24] 10SRE-tools, 10netops, 10Analytics, 10Data-Engineering, and 3 others: an-worker hosts: Netbox - PuppetDB interfaces discrepancies - https://phabricator.wikimedia.org/T295763 (10BTullis) So at first glance, this looks like the Netbox script will do the right thing. It will delete and recreate the the cable,... [18:08:04] 10SRE-tools, 10netops, 10Analytics, 10Data-Engineering, and 3 others: an-worker hosts: Netbox - PuppetDB interfaces discrepancies - https://phabricator.wikimedia.org/T295763 (10Volans) @BTullis fwiw +1 from my end, thanks for having a look. [19:15:01] 10netops, 10Infrastructure-Foundations, 10SRE, 10Traffic: drmrs: primary software task - https://phabricator.wikimedia.org/T282788 (10Majavah) [20:52:23] 10netops, 10DC-Ops, 10Infrastructure-Foundations, 10ops-ulsfo: ulsfo: (2) mx80s to become temp cr[34]-drmrs - https://phabricator.wikimedia.org/T295819 (10RobH)