[03:11:41] FIRING: NetboxAccounting: Netbox - Accounting job failed - https://wikitech.wikimedia.org/wiki/Netbox#Report_Alert - https://netbox.wikimedia.org/core/jobs/ - https://alerts.wikimedia.org/?q=alertname%3DNetboxAccounting [04:11:41] RESOLVED: NetboxAccounting: Netbox - Accounting job failed - https://wikitech.wikimedia.org/wiki/Netbox#Report_Alert - https://netbox.wikimedia.org/core/jobs/ - https://alerts.wikimedia.org/?q=alertname%3DNetboxAccounting [08:54:07] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-codfw, and 2 others: mr1-codfw: add second uplink to lsw1-a2-codfw - https://phabricator.wikimedia.org/T410717#11491000 (10ayounsi) Nice, you can create the /31 prefix in netbox, similar to the other link : https://netbox.wikimedia.org/ipam/prefixes/890/ [09:40:38] 10netops, 06FR-donorrelations, 06Fundraising-Backlog, 10fundraising-tech-ops, and 2 others: Routing issue from payments hosts to 62.146.255.205 - https://phabricator.wikimedia.org/T413382#11491119 (10ayounsi) Looks like that IP is not part of the `payments-processors` address-set so it's not being NATed.... [09:41:17] 10netops, 06FR-donorrelations, 06Fundraising-Backlog, 10fundraising-tech-ops, and 2 others: Routing issue from payments hosts to 62.146.255.205 - https://phabricator.wikimedia.org/T413382#11491120 (10ayounsi) a:03Dwisehaupt [12:34:47] 10Mail, 06Infrastructure-Foundations: leftovers mx1001/2002 references - https://phabricator.wikimedia.org/T413765 (10ayounsi) 03NEW p:05Triage→03Low [15:05:40] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-codfw, 06SRE: msw1-b6-codfw down - https://phabricator.wikimedia.org/T413715#11492257 (10Jhancock.wm) the msw didn't just go down. it died. getting a replacement out of storage. [15:23:12] 10CAS-SSO, 06Infrastructure-Foundations: Provide an official Docker image for CAS-SSO - https://phabricator.wikimedia.org/T412826#11492334 (10LSobanski) p:05Triage→03Low [15:23:50] 10CAS-SSO, 06Infrastructure-Foundations: Provide an official Docker image for CAS-SSO - https://phabricator.wikimedia.org/T412826#11492338 (10SLyngshede-WMF) a:03SLyngshede-WMF We already have the ability to build the docker image, but it makes sense to push it to the registry. Also good to have for other pr... [15:24:59] 10netops, 06Infrastructure-Foundations: asw1-b12-drmrs stopped reporting metrics - https://phabricator.wikimedia.org/T413181#11492343 (10LSobanski) p:05Triage→03Medium a:03ayounsi [15:56:02] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-codfw, 06SRE: msw1-b6-codfw down - https://phabricator.wikimedia.org/T413715#11492530 (10Jhancock.wm) switch has been replaced. all alerts have cleared. netbox updated. closing other tickets. [15:56:09] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-codfw, 06SRE: msw1-b6-codfw down - https://phabricator.wikimedia.org/T413715#11492532 (10Jhancock.wm) 05Open→03Resolved a:03Jhancock.wm [17:24:39] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-codfw, 06SRE: mr1-codfw: add second uplink to lsw1-a2-codfw - https://phabricator.wikimedia.org/T410717#11492952 (10Papaul) 05Open→03Resolved Configuration done ` lsw1-a3-codfw# run show route receive-protocol bgp 10.192.254.15 inet.0: 99... [17:27:43] 10netops, 06Infrastructure-Foundations, 06SRE: InboundInterfaceErrors alerts firing for Nokia switches on v25.10.1 - https://phabricator.wikimedia.org/T412733#11492967 (10Papaul) I sent a Follow up email to Nokia to ask if there were any updates. [18:45:26] 10netops, 06FR-donorrelations, 06Fundraising-Backlog, 10fundraising-tech-ops, and 2 others: Routing issue from payments hosts to 62.146.255.205 - https://phabricator.wikimedia.org/T413382#11493488 (10Dwisehaupt) @ayounsi I have a question for clarification. When we made the move to ipset based rules in ipt... [21:02:00] 10netops, 06Infrastructure-Foundations: access request - read-only access to pfw's for Avishua Stein (astein) - https://phabricator.wikimedia.org/T413826 (10AStein-WMF) 03NEW [23:49:23] 10netbox, 06DC-Ops, 06Infrastructure-Foundations, 10ops-codfw, 06SRE: codfw:cr* router power not balance on all 4 PEM's - https://phabricator.wikimedia.org/T401937#11494266 (10Papaul) 05Open→03Resolved ` Hi Cathal, We have updated the previously shared KB to include details on how the current be... [23:53:46] 10netops, 06Infrastructure-Foundations, 06SRE: InboundInterfaceErrors alerts firing for Nokia switches on v25.10.1 - https://phabricator.wikimedia.org/T412733#11494278 (10Papaul) ` Joao Passo (Nokia) 2:21 PM (3 hours ago) to me, supportservices@nokiacom Hello Papaul, I’ve requested an update to the R&D tea...