[09:05:18] marostegui: the good news is that s2 pk change drops 140GB per relica, the bad news is that it's taking 10 hours, meaning to finish codfw, it'll need 80 more hours [09:05:44] nice one [09:05:53] It doesn't matter, I am not in a rush for s2 codfw switch [09:06:32] I am switching s8 primary eqiad tomorrow morning though [09:07:40] That'd be nice. I'll be able to start dropping stuff from s8 [09:08:10] yep, I need to migrate s8 master and run a few schema changes too [09:32:27] https://phabricator.wikimedia.org/T307501#9841193 [09:37:20] wohooo [10:31:58] Sigh, disks have started failing in this Ceph cluster and it's still in development [11:44:02] This is such a gotcha https://phabricator.wikimedia.org/T366167 [12:30:21] oh, ouch, yes [16:31:46] in theory all mariadb backups are not happening on 10.6 hosts [16:31:53] s/not/now/ [16:32:22] but leaving some leftover 10.4 hosts for now, in case we need to revert [16:32:43] plus a few other tasks missing (cleanups, decoms, os upgrades)