[07:36:20] 10netops, 10Infrastructure-Foundations, 10SRE, 10Patch-For-Review: Add Dell switches support to Homer/Cookbooks - https://phabricator.wikimedia.org/T320638 (10ayounsi) Quick status update regarding Homer. With those 3 patches: * Initial OpenConfig/SONiC support to wmf-netbox - https://gerrit.wikimedia.org/... [09:08:34] 10netops, 10Infrastructure-Foundations, 10SRE, 10Patch-For-Review: Add Dell switches support to Homer/Cookbooks - https://phabricator.wikimedia.org/T320638 (10cmooney) Amazing work! Looks great. >>! In T320638#9082582, @ayounsi wrote: > * The ordering can be problematic (`# TODO needs to happen after the... [12:17:11] 10netops, 10Infrastructure-Foundations, 10SRE: Announce new public IPv6 prefix from Amsterdam for knams migration - https://phabricator.wikimedia.org/T343216 (10cmooney) Range is being accepted by Arelion according to their looking glass: ` Router: adm-b6 / Amsterdam (Iron Mountain, Haarlem) Command: show bg... [12:26:31] 10netops, 10Infrastructure-Foundations, 10SRE: Announce new public IPv6 prefix from Amsterdam for knams migration - https://phabricator.wikimedia.org/T343216 (10cmooney) HE also accepting and path I'm taking from home connection: ` core1.ams7.he.net> show ipv6 bgp routes detail 2a02:ec80:300::/48 Number... [12:30:48] 10netops, 10Infrastructure-Foundations, 10SRE: Announce new public IPv6 prefix from Amsterdam for knams migration - https://phabricator.wikimedia.org/T343216 (10cmooney) Reachable from VPS in the UK although not sure exactly how it's coming in to us: ` root@uk:~# mtr -z -b -w -c 10 2a02:ec80:300:ffff::187 St... [12:34:37] 10netops, 10Infrastructure-Foundations, 10SRE: Announce new public IPv6 prefix from Amsterdam for knams migration - https://phabricator.wikimedia.org/T343216 (10cmooney) Also accepted by Liberty Global. They also see a transit route via Tele2 (AS1257) so getting picked up there, as well as from Deutsche Tel... [12:34:56] 10netops, 10Infrastructure-Foundations, 10SRE, 10Patch-For-Review: New IP and Vlan allocations for esams knams move - https://phabricator.wikimedia.org/T343214 (10cmooney) [12:35:23] 10netops, 10Infrastructure-Foundations, 10SRE: Announce new public IPv6 prefix from Amsterdam for knams migration - https://phabricator.wikimedia.org/T343216 (10cmooney) 05Open→03Resolved [12:57:39] 10netops, 10Infrastructure-Foundations, 10SRE, 10Patch-For-Review, 10User-jbond: Investigate the potential benefits of BGPalerter - https://phabricator.wikimedia.org/T230600 (10ayounsi) 05Open→03Resolved a:03jbond All done! Assigned to jbond as he did most of the work! [13:57:56] 10Traffic, 10DC-Ops, 10SRE, 10ops-eqiad: Q1:rack/setup/install cp1[098-113] - https://phabricator.wikimedia.org/T342159 (10Jclark-ctr) [14:31:13] 10Traffic, 10DC-Ops, 10SRE, 10ops-eqiad: Q1:rack/setup/install cp1[098-113] - https://phabricator.wikimedia.org/T342159 (10RobH) >>! In T342159#9067390, @ssingh wrote: >>>! In T342159#9025176, @RobH wrote: >> Please note parent task 341588 has the range of cp1[090-105] however, cp1090 is already live/in us... [20:36:58] so the bonus question has to be why the auth DNS dashboard shows such a drop in TCP connections since ns2 IP was changed earlier [20:37:20] (nothing seems wrong so nobody panic - but is an unusual one) [20:38:26] * brett removes his parachute back [21:32:28] topranks: yeah I would say we need to look at this even though everything else looks ok... [21:33:17] what's slightly surprising is it's not just a drop on dns300x, drop is present in eqiad/codfw too [21:33:42] why does dns fall back to tcp? udp failures right? [21:33:48] also used for zone transfers? [21:34:33] not sure why we might see a change tbh [21:35:54] overall number of requests seems normal and no increase in errors [21:54:18] DoT connections too to our auth servers but that should have been a blip [21:54:26] will check properly later