[01:00:25] (SystemdUnitFailed) firing: sync-puppet-volatile.service on puppetserver1003:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [01:15:25] (SystemdUnitFailed) resolved: sync-puppet-volatile.service on puppetserver1003:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [06:19:25] (SystemdUnitFailed) firing: netbox_report_accounting_run.service on netbox1002:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [06:49:25] (SystemdUnitFailed) resolved: netbox_report_accounting_run.service on netbox1002:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [09:39:59] XioNoX, topranks: note to self: when releasing a new wmflib we need to do also a Netbox deploy to include the new version in the venv! [09:41:05] volans: can you add it to the doc? [09:41:09] good call hadn’t occurred to me [09:41:52] I've hot-fixed it upgrading the wmflib dep in the venv on netbox1002 and netbox2002, I've also build the new deps, but I'd rather deploy it next week, as it has some major changes for other deps [09:43:37] I've added https://wikitech.wikimedia.org/w/index.php?title=Python%2FWmflib&diff=2172496&oldid=1915427 , but I never check the docs for upgrading wmflib... so I might forgot next time too [09:44:26] this is why I'd rather now deploy it now: [09:44:27] https://gerrit.wikimedia.org/r/c/operations/software/netbox-deploy/+/1024838/1/frozen-requirements-bullseye.txt [09:45:47] makes sens [09:45:53] thx for working on a saturday :) [09:46:01] various deps changed the major version and setuptools got pushed back of many versions... so I'd rather check that and test it on netbox-dev next week [09:46:08] papaul confirmed the hotfix works [09:46:33] I thought we had already added devices to netbox before the trip, hence I didn't think it could cause issues [09:46:56] no prob, you took care of all the burdens this time, it's a small effort and I was near my laptop :)