[01:17:05] FIRING: [4x] PKICertificateExpiry: Intermediate certificate in the trust chain for kafka expires in 29d 11h 49m 25s - https://wikitech.wikimedia.org/wiki/PKI/CA_Operations - TODO - https://alerts.wikimedia.org/?q=alertname%3DPKICertificateExpiry [02:16:27] FIRING: [2x] SystemdUnitFailed: docker-reporter-kubernetes-aux_eqiad-images.service on build2004:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [05:17:05] FIRING: [4x] PKICertificateExpiry: Intermediate certificate in the trust chain for kafka expires in 29d 7h 49m 25s - https://wikitech.wikimedia.org/wiki/PKI/CA_Operations - TODO - https://alerts.wikimedia.org/?q=alertname%3DPKICertificateExpiry [06:16:27] FIRING: [2x] SystemdUnitFailed: docker-reporter-kubernetes-aux_eqiad-images.service on build2004:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [07:45:12] 10netops, 06Infrastructure-Foundations: Restart lsw1-a2-codfw for T437984 - https://phabricator.wikimedia.org/T439134#12367681 (10ayounsi) [07:49:42] 10SRE-tools, 06Infrastructure-Foundations, 13Patch-For-Review: Cookbook for rack depool - https://phabricator.wikimedia.org/T327300#12367696 (10ayounsi) 05Open→03Resolved a:03ayounsi Going to close that one, the pool and depool actions have been tested, now supports also the "manual" policy. Some r... [07:58:31] 10netops, 06Infrastructure-Foundations: Restart lsw1-a2-codfw for T437984 - https://phabricator.wikimedia.org/T439134#12367758 (10ayounsi) 05Open→03Resolved a:03ayounsi Well, CyrusOne took care of the reboot with the power outage. ` lsw1-a2-codfw> show system uptime [...] System booted: 2026-09-23... [08:00:56] 10netops, 06Infrastructure-Foundations: Junos file descriptors exhaustion - https://phabricator.wikimedia.org/T437984#12367825 (10ayounsi) [08:01:02] 10netops, 06Infrastructure-Foundations: Junos file descriptors exhaustion - https://phabricator.wikimedia.org/T437984#12367826 (10ayounsi) 05Open→03Resolved a:03ayounsi All done! [09:01:10] FIRING: [3x] SystemdUnitFailed: docker-reporter-kubernetes-aux_eqiad-images.service on build2004:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [09:17:05] FIRING: [4x] PKICertificateExpiry: Intermediate certificate in the trust chain for kafka expires in 29d 3h 49m 25s - https://wikitech.wikimedia.org/wiki/PKI/CA_Operations - TODO - https://alerts.wikimedia.org/?q=alertname%3DPKICertificateExpiry [12:17:19] 10netbox, 06Infrastructure-Foundations: Netbox: setup netbox-bgp - https://phabricator.wikimedia.org/T439408#12369100 (10ayounsi) p:05Triage→03Medium [12:17:49] 10netbox, 06Infrastructure-Foundations: Netbox: setup netbox-bgp - https://phabricator.wikimedia.org/T439408#12369107 (10ayounsi) [13:01:27] FIRING: [3x] SystemdUnitFailed: docker-reporter-kubernetes-aux_eqiad-images.service on build2004:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [14:42:43] 10netbox, 06Infrastructure-Foundations: https://netbox-exports.wikimedia.org/dns.git is very slow to clone (using dumb HTTP) - https://phabricator.wikimedia.org/T276403#12369751 (10LSobanski) p:05Triage→03Low [14:45:04] 10netops, 06Infrastructure-Foundations: Junos: ae speed not reported corectly through gNMI - https://phabricator.wikimedia.org/T439202#12369774 (10ayounsi) p:05Triage→03Medium [14:47:01] 10netbox, 06Infrastructure-Foundations: Netbox: reports alerting issues - https://phabricator.wikimedia.org/T439130#12369801 (10LSobanski) 05Open→03Stalled p:05Triage→03Low [17:01:27] FIRING: [3x] SystemdUnitFailed: docker-reporter-kubernetes-aux_eqiad-images.service on build2004:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [18:36:11] FIRING: [4x] SystemdUnitFailed: docker-reporter-kubernetes-aux_eqiad-images.service on build2004:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [22:39:01] FIRING: [3x] SystemdUnitFailed: docker-reporter-kubernetes-aux_eqiad-images.service on build2004:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed