[01:08:31] <icinga-wm>	 PROBLEM - MariaDB sustained replica lag on m1 on db1217 is CRITICAL: 5 ge 2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1217&var-port=13321
[01:10:01] <icinga-wm>	 RECOVERY - MariaDB sustained replica lag on m1 on db1217 is OK: (C)2 ge (W)1 ge 0 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1217&var-port=13321
[09:11:51] <Amir1>	 s4 schema change will probably take a full week
[14:33:42] <Amir1>	 50GB is gone from s3 because of ruwikinews externallinks drop https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&var-server=db2109&var-datasource=thanos&var-cluster=wmcs&from=1691502390075&to=1691505167218&viewPanel=28
[14:33:49] <Amir1>	 https://phabricator.wikimedia.org/T312666#9077350
[16:10:36] <icinga-wm>	 PROBLEM - MariaDB sustained replica lag on s6 on db1213 is CRITICAL: 2.2 ge 2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1213&var-port=13316
[16:13:08] <icinga-wm>	 RECOVERY - MariaDB sustained replica lag on s6 on db1213 is OK: (C)2 ge (W)1 ge 0 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1213&var-port=13316