[07:16:55] good morning [07:20:02] hi [07:55:17] bit concerned about the power work proposal - AIUI we have 3-way replication in ms-swift, which means that turning off 5 nodes at once (codfw A2 contains ms-be20{28,29,40,44,51}) means there could be availability issues. And even though eqiad is currently active DC (and we could depool codfw-swift for reads) MW will try and write to both swift clusters. Is that all correct, godog? [07:57:18] Emperor: data is replicated across rows, one rack (or one row) shouldn't cause availability issues [08:00:47] godog: Ah, I'd missed that detail; how is that set up? the rings not explicitly have row in them, do they? Or is that reflected in IP address somehow (and thus zone)? [08:02:03] Emperor: yes that's correct, each row has its own subnet and thus reflected in the ring as zones [08:23:14] TIL, thank you :) [08:28:37] for sure! [08:52:17] that does mean we'll want to make sure e.g. all the A7 servers are back up and happy before they start on the row B PDUs (some of which are also due the same day). I'll write something up for Papaul