[01:08:09] <icinga-wm>	 PROBLEM - MariaDB sustained replica lag on m1 on db1217 is CRITICAL: 31.2 ge 2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1217&var-port=13321
[01:10:59] <icinga-wm>	 RECOVERY - MariaDB sustained replica lag on m1 on db1217 is OK: (C)2 ge (W)1 ge 0 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1217&var-port=13321
[07:20:43] <arnaudb>	 increasing db2109 downtime as I don't think it'll be back soon enough
[08:55:59] <jynus>	 I'me cleaning up dbprov1004- lots of snapshots failed while I was on vacation, and as documented, they are not cleaned up for debugging reasons
[08:56:27] <jynus>	 even if backups are retried when they fail
[09:36:43] <jynus>	 arnaudb just recovered x1 database section from backups @ db1225, well done!
[09:49:29] <arnaudb>	 thank you jynus for the guidance 🙏