[09:48:30] PROBLEM - MariaDB sustained replica lag on s8 on db1214 is CRITICAL: 2.8 ge 2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1214&var-port=9104 [09:51:30] RECOVERY - MariaDB sustained replica lag on s8 on db1214 is OK: (C)2 ge (W)1 ge 0.8 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1214&var-port=9104 [11:28:30] PROBLEM - MariaDB sustained replica lag on s8 on db1214 is CRITICAL: 3.8 ge 2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1214&var-port=9104 [11:34:30] RECOVERY - MariaDB sustained replica lag on s8 on db1214 is OK: (C)2 ge (W)1 ge 0.6 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1214&var-port=9104 [12:18:30] PROBLEM - MariaDB sustained replica lag on s8 on db1177 is CRITICAL: 3 ge 2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1177&var-port=9104 [12:19:30] RECOVERY - MariaDB sustained replica lag on s8 on db1177 is OK: (C)2 ge (W)1 ge 0.2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1177&var-port=9104 [14:24:30] PROBLEM - MariaDB sustained replica lag on s8 on db1214 is CRITICAL: 2.2 ge 2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1214&var-port=9104 [14:27:30] RECOVERY - MariaDB sustained replica lag on s8 on db1214 is OK: (C)2 ge (W)1 ge 0.6 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1214&var-port=9104 [15:30:30] PROBLEM - MariaDB sustained replica lag on s8 on db1214 is CRITICAL: 2.4 ge 2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1214&var-port=9104 [15:34:30] PROBLEM - MariaDB sustained replica lag on s8 on db1214 is CRITICAL: 3.6 ge 2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1214&var-port=9104 [15:43:30] PROBLEM - MariaDB sustained replica lag on s8 on db1214 is CRITICAL: 2.8 ge 2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1214&var-port=9104 [15:52:30] RECOVERY - MariaDB sustained replica lag on s8 on db1214 is OK: (C)2 ge (W)1 ge 0.6 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1214&var-port=9104 [16:59:24] PROBLEM - MariaDB sustained replica lag on s8 on db1192 is CRITICAL: 2.4 ge 2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1192&var-port=9104 [17:00:24] RECOVERY - MariaDB sustained replica lag on s8 on db1192 is OK: (C)2 ge (W)1 ge 0.8 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1192&var-port=9104 [17:07:30] PROBLEM - MariaDB sustained replica lag on s8 on db1214 is CRITICAL: 2.2 ge 2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1214&var-port=9104 [17:14:30] PROBLEM - MariaDB sustained replica lag on s8 on db1214 is CRITICAL: 3 ge 2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1214&var-port=9104 [17:16:30] PROBLEM - MariaDB sustained replica lag on s8 on db1214 is CRITICAL: 2.6 ge 2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1214&var-port=9104 [17:18:30] PROBLEM - MariaDB sustained replica lag on s8 on db1214 is CRITICAL: 2.4 ge 2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1214&var-port=9104 [17:25:30] PROBLEM - MariaDB sustained replica lag on s8 on db1214 is CRITICAL: 2.2 ge 2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1214&var-port=9104 [17:36:30] PROBLEM - MariaDB sustained replica lag on s8 on db1214 is CRITICAL: 2.2 ge 2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1214&var-port=9104 [17:41:32] RECOVERY - MariaDB sustained replica lag on s8 on db1214 is OK: (C)2 ge (W)1 ge 0.6 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1214&var-port=9104 [19:07:30] PROBLEM - MariaDB sustained replica lag on s8 on db1214 is CRITICAL: 4 ge 2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1214&var-port=9104 [19:10:30] RECOVERY - MariaDB sustained replica lag on s8 on db1214 is OK: (C)2 ge (W)1 ge 0.4 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1214&var-port=9104 [19:15:30] PROBLEM - MariaDB sustained replica lag on s8 on db1214 is CRITICAL: 3.6 ge 2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1214&var-port=9104 [19:24:30] RECOVERY - MariaDB sustained replica lag on s8 on db1214 is OK: (C)2 ge (W)1 ge 0.6 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1214&var-port=9104