[08:22:25] (SystemdUnitFailed) firing: netbox_report_accounting_run.service on netbox1002:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [08:47:25] (SystemdUnitFailed) resolved: netbox_report_accounting_run.service on netbox1002:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [11:20:13] 10netops, 06Infrastructure-Foundations, 06SRE, 06Traffic, 13Patch-For-Review: ASW single-point of failure for LVS VIPs at POPs - https://phabricator.wikimedia.org/T362772#9739921 (10cmooney) @ayounsi pointed out another option we may have here to address the switch being single-point of failure. Using d... [11:25:46] 10netops, 06Infrastructure-Foundations, 06SRE, 06Traffic, 13Patch-For-Review: ASW single-point of failure for LVS VIPs at POPs - https://phabricator.wikimedia.org/T362772#9739939 (10cmooney) [13:50:39] any objections to adding redis to wmf-auto-restarts? netbox should handle redis briefly being unavailable just fine (after all, we never had an issue from manual restarts so far) [14:28:35] moritzm: we don't use redis on the netbox hosts anymore, we use the central redis instance instead [14:28:44] they're probably leftovers from before the migration [14:29:21] ah, even better [14:31:55] confirmed, profile::netbox::redis_host points to rdb1013, I'll uninstall redis-server on the netbix hosts [14:32:27] 10netops, 06Infrastructure-Foundations, 06SRE: Cloud IPv6 subnets - https://phabricator.wikimedia.org/T187929#9740697 (10aborrero) 05Stalled→03Open reopening -- we might want to take a look at this soon. [14:52:14] moritzm: I have subscribed you to a bug installing OpenJDK 11 under Buster ( https://phabricator.wikimedia.org/T363339 ) [14:52:26] update-alternatives: using /usr/lib/jvm/java-11-openjdk-amd64/lib/jexec to provide /usr/bin/jexec (jexec) in auto mode [14:52:26] update-alternatives: error: error creating symbolic link '/usr/share/binfmts/jar.dpkg-tmp': No such file or directory [14:52:44] I don't plan to investigate it since that is for a CI Docker image I will "simply" upgrade it to Bullseye [14:52:48] or maybe Bookworm [14:53:01] and pretend the error never happened. But maybe something somewhere might hit it as well :) [14:54:37] sounds good. there's no Java 11 for Bookworm, if you need the combination of 8+11 in one image, then using bullseye is the best way forward [14:56:27] Gerrit and Jenkins are using Java 11 so I will base the docker images on Bullseye [14:56:38] and then I guess during the summer migrate them to Bookworm + Java 17 [14:59:11] sounds good [16:06:25] (SystemdUnitFailed) firing: (2) generate_vrts_aliases.service on mx1001:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [17:06:25] (SystemdUnitFailed) resolved: (2) generate_vrts_aliases.service on mx1001:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [18:00:25] (SystemdUnitFailed) firing: wmf_auto_restart_redis-server.service on idm2001:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [18:10:25] (SystemdUnitFailed) firing: (2) wmf_auto_restart_redis-server.service on idm1001:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [22:10:25] (SystemdUnitFailed) firing: (2) wmf_auto_restart_redis-server.service on idm1001:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed