[03:20:31] <icinga-wm>	 PROBLEM - MariaDB sustained replica lag on es4 on es2022 is CRITICAL: 9 ge 2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=es2022&var-port=9104
[03:24:31] <icinga-wm>	 PROBLEM - MariaDB sustained replica lag on es4 on es2022 is CRITICAL: 3 ge 2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=es2022&var-port=9104
[03:25:31] <icinga-wm>	 RECOVERY - MariaDB sustained replica lag on es4 on es2022 is OK: (C)2 ge (W)1 ge 0.8 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=es2022&var-port=9104
[03:30:31] <icinga-wm>	 PROBLEM - MariaDB sustained replica lag on es4 on es2022 is CRITICAL: 8.4 ge 2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=es2022&var-port=9104
[03:31:31] <icinga-wm>	 RECOVERY - MariaDB sustained replica lag on es4 on es2022 is OK: (C)2 ge (W)1 ge 0.6 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=es2022&var-port=9104
[03:48:31] <icinga-wm>	 PROBLEM - MariaDB sustained replica lag on es4 on es2022 is CRITICAL: 3 ge 2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=es2022&var-port=9104
[03:51:31] <icinga-wm>	 RECOVERY - MariaDB sustained replica lag on es4 on es2022 is OK: (C)2 ge (W)1 ge 0 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=es2022&var-port=9104
[12:00:21] <Amir1>	 >  	codfw 	snapshot 	wrong_size 	2 hours ago 	1.1 TB 	-9.6 % 	The previous backup had a size of 1.2 TB, a change larger than 5.0%. 
[12:00:25] <Amir1>	 (s3)
[18:13:03] <icinga-wm>	 PROBLEM - MariaDB sustained replica lag on s8 on db1214 is CRITICAL: 4.8 ge 2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1214&var-port=9104
[18:18:05] <icinga-wm>	 RECOVERY - MariaDB sustained replica lag on s8 on db1214 is OK: (C)2 ge (W)1 ge 0.2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1214&var-port=9104
[18:29:05] <icinga-wm>	 PROBLEM - MariaDB sustained replica lag on s8 on db1214 is CRITICAL: 3.4 ge 2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1214&var-port=9104
[18:35:03] <icinga-wm>	 RECOVERY - MariaDB sustained replica lag on s8 on db1214 is OK: (C)2 ge (W)1 ge 0.6 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1214&var-port=9104
[23:45:53] <icinga-wm>	 PROBLEM - MariaDB sustained replica lag on s7 on db2218 is CRITICAL: 3 ge 2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db2218&var-port=9104
[23:46:53] <icinga-wm>	 RECOVERY - MariaDB sustained replica lag on s7 on db2218 is OK: (C)2 ge (W)1 ge 0 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db2218&var-port=9104