[07:29:44] puppet needs to know because it manages fstab and handles disk replacement (by mkfsing etc. the new device) [07:30:29] [which is why we had problems with inconsistent device names - puppet would want to make swift-sda3 on the disk that had come up as sda3 this time and so on] [07:30:56] Amir1: can I start a schema change on s7? [07:32:24] Amir1: I see your schema change on s7 ended yesterday ( https://phabricator.wikimedia.org/T400854 ) so I can start mine in a bit [12:56:51] Emperor: that's right, "automating disk replacement" [13:15:07] Emperor: I guess the `exp` value is the SAS expander port? Is there any danger of the expander getting replaced, and having the same disk present with a different ID? [13:16:40] I think replacing the disk controller would likely require a reimage [13:17:39] I guess I'm wondering whether that is entirely stable. I realize the by-path entries are specifically meant to be stable across reboots, but are there other scenarios? [13:18:04] oh, a disk controller would be a reimage? [13:20:08] automating management of the fstab through puppet wasn't something I could bring myself to do for Cassandra [13:20:48] I think part of that is that the hardware is less consistent, and we're doing software raid on the same devices as the JBOD [13:21:05] (because it's inherited hardware configuration) [13:21:23] and I don't think disk failures are as routine as they seem to be in swift [13:22:04] and there is more to do after a disk failure, and that couldn't similarly be automated through puppet [13:22:12] *but*, it also felt brittle [13:22:23] I'm currently failing to drive puppet (cf #wikimedia-sre) with a change-set to try and make this work