[01:28:25] FIRING: SystemdUnitFailed: netbox_ganeti_ulsfo_sync.service on netbox1003:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [05:28:25] FIRING: SystemdUnitFailed: netbox_ganeti_ulsfo_sync.service on netbox1003:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [09:28:25] FIRING: SystemdUnitFailed: netbox_ganeti_ulsfo_sync.service on netbox1003:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [10:27:43] 10netops, 06Infrastructure-Foundations: Multiple unreachable hosts in eqiad - https://phabricator.wikimedia.org/T382772 (10LSobanski) 03NEW [10:29:43] 10netops, 06Infrastructure-Foundations: Multiple unreachable hosts in eqiad - https://phabricator.wikimedia.org/T382772#10423372 (10Marostegui) ` root@db1187:~# uptime 10:29:10 up 26 days, 21:02, 1 user, load average: 0.99, 1.23, 1.23 ` ` root@db1179:~# uptime 10:29:29 up 56 days, 48 min, 1 user, load a... [12:37:56] 10netops, 06Infrastructure-Foundations: Multiple unreachable hosts in eqiad - https://phabricator.wikimedia.org/T382772#10423402 (10Ladsgroup) That looks like a netwok blip (I know I'm stating the obvious :D) [12:42:37] 10netops, 06Infrastructure-Foundations: Multiple unreachable hosts in eqiad - https://phabricator.wikimedia.org/T382772#10423407 (10Marostegui) >>! In T382772#10423402, @Ladsgroup wrote: > That looks like a netwok blip (I know I'm stating the obvious :D) Yes, this was discussed in irc. We created the task ju... [13:28:25] FIRING: SystemdUnitFailed: netbox_ganeti_ulsfo_sync.service on netbox1003:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [17:28:25] FIRING: SystemdUnitFailed: netbox_ganeti_ulsfo_sync.service on netbox1003:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [19:02:01] FIRING: NTPNoSynced: NTP not synced on graphite1005:9100 - https://wikitech.wikimedia.org/wiki/NTP - TODO - https://alerts.wikimedia.org/?q=alertname%3DNTPNoSynced [20:17:01] RESOLVED: NTPNoSynced: NTP not synced on graphite1005:9100 - https://wikitech.wikimedia.org/wiki/NTP - TODO - https://alerts.wikimedia.org/?q=alertname%3DNTPNoSynced [21:28:25] FIRING: SystemdUnitFailed: netbox_ganeti_ulsfo_sync.service on netbox1003:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed