[00:56:24] <jinxer-wm>	 FIRING: SystemdUnitFailed: wmf_auto_restart_prometheus-mysqld-exporter@s8.service on db1216:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[04:56:24] <jinxer-wm>	 FIRING: SystemdUnitFailed: wmf_auto_restart_prometheus-mysqld-exporter@s8.service on db1216:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[07:02:53] <jynus>	 handling db1216's alert- I probably didn't do a great job to cleanup old system unit
[07:06:25] <jinxer-wm>	 RESOLVED: SystemdUnitFailed: wmf_auto_restart_prometheus-mysqld-exporter@s8.service on db1216:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[07:35:17] <marostegui>	 I am moving s5 codfw to SBR
[09:02:45] <Emperor>	 is it me, or has the team=data-persistence option vanished from alerts.wikimedia.org ?
[09:03:47] <federico3>	 marostegui: for https://phabricator.wikimedia.org/T393296 the host is in RMA and it might take a while to get a new one: should I keep the task as High priority and assigned to myself?
[09:04:46] <marostegui>	 federico3: Up to you
[09:05:07] <marostegui>	 Probably assign it to VRiley
[09:05:20] <marostegui>	 As we are just waiting for them to figure it out with dell
[09:07:48] <federico3>	 OK, I'll split it into a task for them and a tracking task for us
[09:09:07] <marostegui>	 federico3: I don't think there is a need for that
[09:09:14] <marostegui>	 We can all just follow that task
[09:09:23] <marostegui>	 Otherwise we are going to end up with more confusion
[09:11:18] <federico3>	 ack
[12:43:29] <taavi>	 is there intentionally no mention of T394624 on https://wikitech.wikimedia.org/wiki/Map_of_database_maintenance?
[12:43:30] <stashbot>	 T394624: db1155 HW memory errors - https://phabricator.wikimedia.org/T394624
[12:43:45] <marostegui>	 I logged it yesterday on SAL
[12:43:49] <marostegui>	 But not there
[12:43:59] <marostegui>	 04:51 marostegui: Stop mariadb on db1155, wiki replicas will show lag on: s2, s4, s6 and s7 T394624
[12:44:02] <marostegui>	 That is from yesterday
[14:17:03] <dhinus>	 q: is there any sensitive data in the enwiki db schema, that should not be public? I don't think so, but I wanted to double check with you folks before I push it to a public git repo
[14:17:06] <dhinus>	 context: https://gerrit.wikimedia.org/r/c/operations/puppet/+/1148394/1/modules/profile/files/wmcs/db/wikireplicas/README.md