[00:46:55] FIRING: SystemdUnitFailed: swift_rclone_sync.service on ms-be1069:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [04:46:55] FIRING: SystemdUnitFailed: swift_rclone_sync.service on ms-be1069:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [07:21:55] RESOLVED: SystemdUnitFailed: swift_rclone_sync.service on ms-be1069:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [09:47:10] Amir1: looks like the parsercache p1 job ran okay from my end, am I right? [09:54:35] I check now [10:44:44] Everything looks fine https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&refresh=1m&var-job=mysql-parsercache&var-server=pc1011&var-port=9104&from=now-24h&to=now [11:07:44] great! Would you be comfortable with us migrating all of the other parsercache jobs? [13:23:42] hnowlan: sure! [13:58:34] Amir1: <3 [15:11:15] Could I get a +1 on https://gerrit.wikimedia.org/r/c/operations/puppet/+/1143118 please? Then I can decommission this failed host [15:21:25] Emperor: {{done}} [15:24:27] thanks :) [15:32:58] hi DP, db1247 (an s4 replica) just crashed, haven't started looking at it yet but I did depool [15:33:03] do you want me to open a ticket? [15:44:52] cdanis: I'm not a DBA, but the SOP ( https://wikitech.wikimedia.org/wiki/MariaDB/Troubleshooting#Depooling_a_replica ) is to open a ticket, so I'd suggest you go ahead, please [15:45:29] Emperor: indeed, Scott has opened T393612 since [15:45:31] T393612: db1247 crash or restart - 15:29 on 2025-05-07 - https://phabricator.wikimedia.org/T393612 [15:46:12] cool, thanks