[06:53:19] 10netops, 06Infrastructure-Foundations, 10Observability-Logging: ~5k/logs/sec from netdev - https://phabricator.wikimedia.org/T412143#11498634 (10ayounsi) After a few back and forth, JTAC configured a lab switch to try reproduce and will monitor the issue for a few weeks : > Thanks for the update. I've recen... [07:29:09] 10netops, 06Infrastructure-Foundations: asw1-b12-drmrs stopped reporting metrics - https://phabricator.wikimedia.org/T413181#11498639 (10ayounsi) Opened case 2026-0107-016071 Data seems fully back for gNMI (and looks like it never went away). For SNMP it worked briefly over the new year but it's now gone aga... [10:26:03] 10netops, 06Infrastructure-Foundations, 06SRE: Inaccurate stats reported by cr2-codfw - https://phabricator.wikimedia.org/T400205#11499053 (10cmooney) 05Open→03Resolved a:03cmooney I should have updated here, Juniper advise this is fixed in 23.4R2-S3 and beyond, which was released in Novemeber 2025... [10:27:05] 10netops, 06Infrastructure-Foundations: asw1-b12-drmrs stopped reporting metrics - https://phabricator.wikimedia.org/T413181#11499063 (10cmooney) > Could it be something similar to T400205: Inaccurate stats reported by cr2-codfw ? Yeah it seems quite similar, but given that seemed to be some quirk due to the... [13:08:56] 10netops, 06Infrastructure-Foundations, 13Patch-For-Review: access request - read-only access to pfw's for Avishua Stein (astein) - https://phabricator.wikimedia.org/T413826#11499528 (10ayounsi) 05Open→03Resolved a:03ayounsi I'm pushing the change right now, you should be good to go in 15/30min. [14:40:32] elukey: thanks a lot for the investigation on T412451. Are you done with one of those servers, and could I take it over? [14:40:32] T412451: 4 failed reimages on wdqs1029, 1030, 1031, 1032 - https://phabricator.wikimedia.org/T412451 [14:43:09] gehel: o/ not yet, I am still working on 1032, and I'd need to do another round of reimage to see if the issue re-happens [14:43:13] how urgent are those? [14:45:05] Not too much. We need to prepare them for the Wikidata Platform team to be able to test blazegraph alternatives. They have wdqs1028 working, so they're not blocked. [14:45:42] It would be nice to have a second server ready by next week, but I still think it's more important to get to the bottom of this than to rush. [15:17:25] ok I'll try my best :) [15:39:06] your best sounds good! thanks! [16:20:19] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-ulsfo, and 2 others: ULSFO: New switch configuration - https://phabricator.wikimedia.org/T408892#11500375 (10Papaul) @ssingh Hello and Happy New year I just wanted to check with you once again if it is now safe to resume the loopback IP changes on the... [16:38:43] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-ulsfo, and 2 others: ULSFO: New switch configuration - https://phabricator.wikimedia.org/T408892#11500425 (10ssingh) >>! In T408892#11500375, @Papaul wrote: > @ssingh Hello and Happy New year I just wanted to check with you once again if it is now saf... [19:15:34] 10netops, 10fundraising-tech-ops, 06Infrastructure-Foundations: Remove pfw configuration related to former pybal/LVS service - https://phabricator.wikimedia.org/T414015 (10Jgreen) 03NEW