[00:27:43] PROBLEM - MariaDB sustained replica lag on s3 on db1175 is CRITICAL: 34 ge 10 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1175&var-port=9104 [00:28:05] PROBLEM - MariaDB sustained replica lag on s3 on db2209 is CRITICAL: 30.6 ge 10 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db2209&var-port=9104 [00:28:19] PROBLEM - MariaDB sustained replica lag on s3 on db1212 is CRITICAL: 24.4 ge 10 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1212&var-port=9104 [00:28:19] PROBLEM - MariaDB sustained replica lag on s3 on db1166 is CRITICAL: 14.2 ge 10 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1166&var-port=9104 [00:28:19] PROBLEM - MariaDB sustained replica lag on s3 on db1189 is CRITICAL: 21.4 ge 10 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1189&var-port=9104 [00:28:19] PROBLEM - MariaDB sustained replica lag on s3 on db1198 is CRITICAL: 32.8 ge 10 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1198&var-port=9104 [00:28:25] PROBLEM - MariaDB sustained replica lag on s3 on db1157 is CRITICAL: 30.2 ge 10 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1157&var-port=9104 [00:29:19] RECOVERY - MariaDB sustained replica lag on s3 on db1166 is OK: (C)10 ge (W)5 ge 0 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1166&var-port=9104 [00:30:05] RECOVERY - MariaDB sustained replica lag on s3 on db2209 is OK: (C)10 ge (W)5 ge 0 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db2209&var-port=9104 [00:30:19] RECOVERY - MariaDB sustained replica lag on s3 on db1212 is OK: (C)10 ge (W)5 ge 0 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1212&var-port=9104 [00:30:19] RECOVERY - MariaDB sustained replica lag on s3 on db1198 is OK: (C)10 ge (W)5 ge 0 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1198&var-port=9104 [00:30:19] RECOVERY - MariaDB sustained replica lag on s3 on db1189 is OK: (C)10 ge (W)5 ge 0 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1189&var-port=9104 [00:30:25] RECOVERY - MariaDB sustained replica lag on s3 on db1157 is OK: (C)10 ge (W)5 ge 0 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1157&var-port=9104 [00:30:43] RECOVERY - MariaDB sustained replica lag on s3 on db1175 is OK: (C)10 ge (W)5 ge 0 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1175&var-port=9104