[05:19:27] <jinxer-wm>	 FIRING: SystemdUnitFailed: check-private-data.service on db1155:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[05:23:46] <marostegui>	 ^ checking
[05:25:49] <marostegui>	 [9049307.944609] MCE: Killing mysqld:4072 due to hardware memory corruption fault at 7fcfec064904
[05:25:50] <marostegui>	 nice
[05:29:27] <jinxer-wm>	 RESOLVED: SystemdUnitFailed: check-private-data.service on db1155:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[05:30:09] <icinga-wm>	 PROBLEM - MariaDB sustained replica lag on s7 on db1155 is CRITICAL: 114.5 ge 10 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1155&var-port=13317
[05:31:09] <icinga-wm>	 RECOVERY - MariaDB sustained replica lag on s7 on db1155 is OK: (C)10 ge (W)5 ge 0 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1155&var-port=13317
[14:41:51] <jynus>	 FYI https://phabricator.wikimedia.org/T394673