[08:33:10] <wikibugs>	 10Traffic, 10MW-on-K8s, 10SRE, 10serviceops: Add configuration file support to mw-on-k8s.lua ATS script - https://phabricator.wikimedia.org/T336037 (10Joe) 05In progress→03Resolved
[08:33:22] <wikibugs>	 10Traffic, 10MW-on-K8s, 10SRE, 10serviceops, and 2 others: Serve production traffic via Kubernetes - https://phabricator.wikimedia.org/T290536 (10Joe)
[08:34:25] <wikibugs>	 10Traffic, 10MW-on-K8s, 10SRE, 10serviceops: Add traffic sampling support to mw-on-k8s.lua ATS script - https://phabricator.wikimedia.org/T336038 (10Joe) 05In progress→03Resolved
[08:34:36] <wikibugs>	 10Traffic, 10MW-on-K8s, 10SRE, 10serviceops, and 2 others: Serve production traffic via Kubernetes - https://phabricator.wikimedia.org/T290536 (10Joe)
[09:23:53] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10SRE: Netbox network report failing - timeout error getting connected_endpoint prefix - https://phabricator.wikimedia.org/T321704 (10cmooney) This appears to be happening more often now, and is starting to cause considerable noise in the dc-ops  irc channel.  @volans...
[09:33:59] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10SRE: Netbox network report failing - timeout errors - https://phabricator.wikimedia.org/T321704 (10cmooney)
[12:15:06] <wikibugs>	 10Traffic, 10MW-on-K8s, 10SRE, 10serviceops, and 2 others: Serve production traffic via Kubernetes - https://phabricator.wikimedia.org/T290536 (10Joe)
[12:22:21] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10SRE, 10SRE-tools: Setup zero touch provisioning (ZTP) for network devices - https://phabricator.wikimedia.org/T336485 (10cmooney) @volans I had a word with @ayounsi on this and we both feel if we can make it work via HTTP to the apt server that's probably best.  I...
[12:39:33] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10SRE, 10Patch-For-Review: Update network SSH keys to ssh-ed25519 - https://phabricator.wikimedia.org/T336769 (10ayounsi)
[13:35:01] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10SRE, 10Patch-For-Review: Migrate row E/F network aggregation to dedicated Spine switches - https://phabricator.wikimedia.org/T322937 (10ops-monitoring-bot) Icinga downtime and Alertmanager silence (ID=c4ef01af-e7d5-458f-ae46-17500f124165) set by cmooney@cumin1001 f...
[14:51:01] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10SRE: Netbox network report failing - timeout errors - https://phabricator.wikimedia.org/T321704 (10ayounsi) Upgrade to 3.2.9 didn't help, but we were expecting it a bit.  At this point I guess that it's related to the steady increase of Netbox usage and we should loo...
[17:11:46] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10SRE: Create mechanism to disable IPv6 RA generation on irb interfaces when required - https://phabricator.wikimedia.org/T337057 (10cmooney) p:05Triage→03Medium
[17:15:41] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10SRE, 10Patch-For-Review: Create mechanism to disable IPv6 RA generation on irb interfaces when required - https://phabricator.wikimedia.org/T337057 (10cmooney)
[17:15:49] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10SRE: Migrate row E/F network aggregation to dedicated Spine switches - https://phabricator.wikimedia.org/T322937 (10cmooney)
[17:42:20] <wikibugs>	 10netops, 10DC-Ops, 10Infrastructure-Foundations, 10SRE, 10ops-eqiad: Q2:(Need By: TBD) Rows E/F network racking task - https://phabricator.wikimedia.org/T292095 (10cmooney) @Jclark-ctr all went well with that today thank you for your help.  For the next phase we need to move the following links:  |No|Ra...
[17:44:04] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10SRE: Migrate row E/F network aggregation to dedicated Spine switches - https://phabricator.wikimedia.org/T322937 (10cmooney) The migration went fine today, very quick move and all came up as expected.  EVPN MAC-move BGP signalling worked flawlessly was nice to see in...
[17:46:35] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10SRE, 10Patch-For-Review: Create mechanism to disable IPv6 RA generation on irb interfaces when required - https://phabricator.wikimedia.org/T337057 (10cmooney) Another option, similar to the above patch, would maybe to make it a global toggle for a device.  So like...
[17:52:34] <wikibugs>	 10netops, 10DC-Ops, 10Infrastructure-Foundations, 10SRE, 10ops-eqiad: Q2:(Need By: TBD) Rows E/F network racking task - https://phabricator.wikimedia.org/T292095 (10ssingh) >>! In T292095#8865511, @cmooney wrote: > @Jclark-ctr all went well with that today thank you for your help. >  > For the next phase...
[18:28:09] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10SRE, 10SRE-tools: Setup zero touch provisioning (ZTP) for network devices - https://phabricator.wikimedia.org/T336485 (10Volans) Ok that sounds like a plan, let's try first if the FQDN link works and if not we'll fallback to the IP. Based on the test we might add t...
[18:30:59] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10SRE: Netbox network report failing - timeout errors - https://phabricator.wikimedia.org/T321704 (10Volans) I'll try to have a look next week, but for now I downtimed the alert so it doesn't spam too much until the end of the month. https://icinga.wikimedia.org/cgi-bi...