[01:20:40] FIRING: SystemdUnitFailed: debian-weekly-rebuild.service on build2004:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [05:20:40] FIRING: SystemdUnitFailed: debian-weekly-rebuild.service on build2004:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [06:50:16] 10netops, 10Cloud-VPS, 06Data-Platform-SRE, 10Data-Services, and 3 others: Plan to make clouddumps more resilient and easier to operate - https://phabricator.wikimedia.org/T411248#12084087 (10fgiunchedi) >>! In T411248#12080727, @ayounsi wrote: >>>! In T411248#12076020, @fgiunchedi wrote: >>>>! In T411248#... [09:20:40] FIRING: SystemdUnitFailed: debian-weekly-rebuild.service on build2004:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [09:48:03] 10netops, 06Infrastructure-Foundations, 06Data-Platform-SRE (2026-07-03 - 2026-07-31), 07Kubernetes: Calico IPv4 block exhaustion on dse-k8s cluster, blocking new node provisioning - https://phabricator.wikimedia.org/T429773#12084768 (10Gehel) [09:52:38] 10netops, 10Cloud-Services, 06collaboration-services, 06Data-Persistence, and 6 others: codfw: rack B5 maintenance - https://phabricator.wikimedia.org/T430918#12084857 (10Gehel) [13:16:03] 10netops, 06Infrastructure-Foundations: Upgrade netflow hosts to Trixie - https://phabricator.wikimedia.org/T424478#12085652 (10MoritzMuehlenhoff) [13:20:40] FIRING: SystemdUnitFailed: debian-weekly-rebuild.service on build2004:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [13:20:41] 10netops, 06Infrastructure-Foundations: Upgrade netflow hosts to Trixie - https://phabricator.wikimedia.org/T424478#12085664 (10MoritzMuehlenhoff) I reviewed the Puppet classes and everything should be ready for Trixie. How about we create a separate parallel, test node in magru first and when it's working we... [15:40:24] 10netops, 06Infrastructure-Foundations: Upgrade netflow hosts to Trixie - https://phabricator.wikimedia.org/T424478#12086094 (10cmooney) >>! In T424478#12085664, @MoritzMuehlenhoff wrote: > I reviewed the Puppet classes and everything should be ready for Trixie. How about we create a separate parallel, test no... [17:20:40] FIRING: SystemdUnitFailed: debian-weekly-rebuild.service on build2004:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [17:22:44] 10netops, 06Infrastructure-Foundations, 06SRE: cr2-esams rpd failure after enabling bgp 'graceful-shutdown' (June 2026) - https://phabricator.wikimedia.org/T429386#12086301 (10cmooney) FWIW I upgraded cr3-ulsfo as I had to drain it to reset the PIC and it was on known-bad release 23.4R2-S7. [20:56:46] 10SRE-tools, 06Infrastructure-Foundations, 10Spicerack: wait_for_optimal() should ignore acked alerts - https://phabricator.wikimedia.org/T319277#12086714 (10Tvassilian) 05Open→03In progress [21:20:40] FIRING: SystemdUnitFailed: debian-weekly-rebuild.service on build2004:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed