[00:47:39] <wikibugs>	 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-eqsin, 06SRE: EQSIN:Switch refresh diagram and wiring - https://phabricator.wikimedia.org/T423724#12169791 (10Papaul)
[01:47:05] <wikibugs>	 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-eqsin, 06SRE: EQSIN:Switch refresh diagram and wiring - https://phabricator.wikimedia.org/T423724#12169848 (10Papaul)
[02:39:36] <wikibugs>	 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-eqsin, 06SRE: EQSIN:New switch setup/configuration - https://phabricator.wikimedia.org/T418439#12169932 (10Papaul)
[03:17:25] <jinxer-wm>	 RESOLVED: SystemdUnitFailed: generate_vrts_aliases.service on mx-in2001:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[03:20:25] <jinxer-wm>	 FIRING: SystemdUnitFailed: generate_vrts_aliases.service on mx-in2001:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[03:34:46] <wikibugs>	 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-eqsin, and 2 others: EQSIN:New switch setup/configuration - https://phabricator.wikimedia.org/T418439#12169969 (10Papaul) I didn't disable one of the BGP session from mr1 to cr since we didn't make the connection from asw1-603-eqsin to cr2-cr3 et-0/0/...
[07:20:25] <jinxer-wm>	 FIRING: SystemdUnitFailed: generate_vrts_aliases.service on mx-in2001:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[08:31:32] <wikibugs>	 10netops, 06Infrastructure-Foundations: Investigate network devices disk encryption with TPM - https://phabricator.wikimedia.org/T433579 (10ayounsi) 03NEW p:05Triage→03Low
[09:15:56] <wikibugs>	 10netops, 06DC-Ops, 06Infrastructure-Foundations, 13Patch-For-Review: Consider removing cable IDs from interface descriptions - https://phabricator.wikimedia.org/T432689#12170482 (10ayounsi) Merged and deploying it. It's going to take a bit of time as it means a different diff per device.
[09:16:13] <wikibugs>	 10netops, 06DC-Ops, 06Infrastructure-Foundations, 13Patch-For-Review: Consider removing cable IDs from interface descriptions - https://phabricator.wikimedia.org/T432689#12170484 (10ayounsi) 05Open→03Resolved a:03ayounsi
[11:20:40] <jinxer-wm>	 FIRING: SystemdUnitFailed: generate_vrts_aliases.service on mx-in2001:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[11:53:04] <wikibugs>	 10netops, 06Infrastructure-Foundations, 06SRE: Core router upgrades 2026 #2 - https://phabricator.wikimedia.org/T431748#12171023 (10ayounsi) 05Open→03Resolved a:03ayounsi All done here.
[11:55:01] <wikibugs>	 10netops, 06Infrastructure-Foundations, 06SRE: cr2-esams rpd failure after enabling bgp 'graceful-shutdown' (June 2026) - https://phabricator.wikimedia.org/T429386#12171034 (10ayounsi) 05Open→03Resolved Closing this task as all the problematic 23.4R2-S7 have been upgraded. The remaining ones will go...
[11:56:33] <wikibugs>	 10netops, 06Infrastructure-Foundations, 06SRE: Core router upgrades 2026 #2 - https://phabricator.wikimedia.org/T431748#12171036 (10ayounsi) 05Resolved→03Open a:05ayounsi→03None
[15:20:25] <jinxer-wm>	 RESOLVED: SystemdUnitFailed: generate_vrts_aliases.service on mx-in2001:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[15:23:25] <jinxer-wm>	 FIRING: SystemdUnitFailed: generate_vrts_aliases.service on mx-in2001:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[15:41:52] <wikibugs>	 10netops, 06Infrastructure-Foundations, 10ops-eqsin, 06SRE, 06Traffic: eqsin downtime scheduling for switch upgrade - https://phabricator.wikimedia.org/T433097#12171900 (10RobH) 05Open→03Resolved Resolution Notes:  * Work was completed without any unexpected downtime * The fibers/optics needed ha...
[16:43:25] <jinxer-wm>	 FIRING: [2x] SystemdUnitFailed: generate_vrts_aliases.service on mx-in2001:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[16:48:25] <jinxer-wm>	 FIRING: [2x] SystemdUnitFailed: generate_vrts_aliases.service on mx-in2001:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[18:07:59] <jinxer-wm>	 FIRING: [2x] SystemdUnitFailed: generate_vrts_aliases.service on mx-in2001:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[19:33:50] <Emperor>	 I've just opened T433635 - I can't reimage ms-be2082 or reprovision it, seemingly because redfish is bust?
[19:33:50] <stashbot>	 T433635: Unable to reimage or reprovision ms-be2082 due to redfish connection errors - https://phabricator.wikimedia.org/T433635
[20:48:29] <wikibugs>	 10SRE-tools, 10Cumin, 06DC-Ops, 06Infrastructure-Foundations, and 2 others: add dcops group to run sre.hosts.downtime cookbook - https://phabricator.wikimedia.org/T433409#12172905 (10Dzahn) The change is about editing the permissions of the `datacenter-ops` shell group.  And that group has the "approval:"-...
[20:51:42] <wikibugs>	 10SRE-tools, 10Cumin, 06DC-Ops, 06Infrastructure-Foundations, and 3 others: add dcops group to run sre.hosts.downtime cookbook - https://phabricator.wikimedia.org/T433409#12172933 (10Dzahn)
[22:08:00] <jinxer-wm>	 FIRING: [2x] SystemdUnitFailed: generate_vrts_aliases.service on mx-in2001:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[23:33:18] <wikibugs>	 10SRE-tools, 10Cumin, 06DC-Ops, 06Infrastructure-Foundations, and 3 others: add dcops group to run sre.hosts.downtime cookbook - https://phabricator.wikimedia.org/T433409#12173180 (10RobH) a:03wiki_willy Willy,  Can you give your approval as DC Ops manager to increase the rights of the dc ops user group...