[14:54:02] mysql can't take a default value for blobs, that makes extlinks migration quite more complicated (the columns I want to drop are blobs). Thankfully Mariadb claims that it takes default (haven't tested yet) but still a lot more work than it needs to be [14:56:55] if you need to be backwards compatible, a one-time, temporary trigger could do the thing [14:57:40] yeah but still :( [14:58:22] I think I'll make the columns default in production given that it's not an issue for third parties (by the time of the release, it'll be dropped directly) [15:03:16] ah, maybe I can make it nullable [15:04:25] ms-be1064 alert is an expected restart? [15:08:26] it was on SAL [15:08:34] why it alerted dunno [15:11:01] I've been rebooting backends all day (T335838) but using the cookbook, so they shouldn't alert [15:12:19] ok, as long as it is not a crash or urgency not worries on my side [15:12:27] is it possible ms-be1064 crashed after the planned reboot? [15:14:14] I did have to reboot it twice (stupid drive ordering), but that was while the reboot cookbook was still running (waiting for successful puppet run) [20:57:19] PROBLEM - MariaDB sustained replica lag on s4 on db1221 is CRITICAL: 4.4 ge 2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1221&var-port=9104 [21:00:17] RECOVERY - MariaDB sustained replica lag on s4 on db1221 is OK: (C)2 ge (W)1 ge 0 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1221&var-port=9104