[09:04:11] RhinosF1|Away: yeah, I plan to make the document public in wikitech today. It's been internal for the last couple of months, getting progressively reviewed by WMF departments. I 'd love to hear your input in the talk page if you have any. [09:12:45] btw, all upcoming (and some past) switchover dates are pre-calculated (up to 2050) and displayed here (including the code used to calculate them): https://wikitech.wikimedia.org/wiki/Switch_Datacenter/Switchover_Dates [09:13:23] and linked from the main switchover page (https://wikitech.wikimedia.org/wiki/Switch_Datacenter#Upcoming_Switches) [09:43:57] RhinosF1|Away: https://wikitech.wikimedia.org/wiki/Switch_Datacenter/Recurring,_Equinox-based,_Data_Center_Switchovers [09:55:01] RhinosF1|Away, taavi: I will be reading wikitech-l, at least around the switchover times [10:21:02] akosiaris: thanks [10:23:40] * RhinosF1|Away will read when through passport control [16:00:30] jelto or sobanski: are you planning to do anything user-visible (e.g. gitlab unavailability) during the switchover? the current banner mentions gitlab unavailability but that might just be historical, so I wanted to ask [16:02:36] kamila_: Yes, the switchover requires a downtime of ~2h. We are not planning to execute on the same day as the Mediawiki switchover (exact date TBD, tasks coming up tomorrow). FYI eoghan. [16:07:34] kamila_: Which banner are you referring to? [16:08:31] eoghan: https://meta.wikimedia.org/wiki/Tech/Server_switch [16:09:58] * kamila_ context is https://phabricator.wikimedia.org/T345265 [16:12:16] I'm not sure that anything in the DC switchover affects Gitlab, we'll be doing that a few days before/after the swithc (as sobanski said, dates TBC in the next few days). [16:12:54] So I think it can be removed if you want to be super accurate, but maybe it can be a happy surprise for people that gitlab continues working (: [16:13:09] +1 [16:31:56] :D so I'll leave it as a happy surprise then, ok? [16:31:59] thanks for the info [16:59:15] we clearly need an XKCD 303 for "gitlab is down" ;-) [17:05:11] kamila_: "The Wikimedia Foundation tests the switch between its first and secondary data centers." - is that true anymore as they are now equal? [17:05:23] I read through Alex's wikitech doc, seems like a nice plan [17:23:40] there continues to be a lot of things where the DCs aren't equal. [17:24:12] even for mediawiki, of course, there's a primary site for writes, and only reads are balanced across the two. Some other services are also either active/active or active/passive across the two. [17:25:14] but then there's some "other" bits that aren't dual-site at all (like most of WMCS last I checked, and probably still some of operational bits, grafana-like things (although I think that one in particular is now at least a/p across both) [17:25:24] some analytics things too? [17:25:28] Y [17:25:30] bblack: equal in terms of there is an aim to be no primary / secondary DC. Just the active & passive DC. [17:25:51] And yea WMCS is eqiad for actual use and codfw for test [17:25:54] yeah but some things can't even fail over active/passive yet. they're just not things that are in the path of "service the main wikis and the bulk of the traffic" [17:26:04] true [19:02:31] jbond or anyone else, is cergen the recommended way to generate new TLS certs or is it CFSSL now? we need one for a discovery.wmnet domain [19:59:02] inflatador: waaay out of my depth here, but I know Xio.NoX automated TLS certs for out network gear recently and used CFSSL [19:59:14] so that's probably the answer but I'm not sure [20:00:14] topranks thanks, I was looking at that network TLS cookbook and the puppet repo. We went with cergen for the short term since it seems more common in the puppet repo [20:01:11] yeah might be the safer option [20:03:17] We got it to work, so should be good 'till 2028...at which time we'll probably have something totally different ;P [21:25:01] RhinosF1|Away: re "first datacenter", I've told them to replace it with primary for exactly that reason [21:25:59] would it be better to rewrite it to active/passive? [21:26:53] I'm not really sure how much it's worth it to faff with the terminology given that (as others have pointed out) different systems treat the DCs differently [21:27:29] there is no single word that will be correct anyway :D [21:27:54] IMHO, "primary" is less-confusing. active/passive vs active/active is kind of a finer level of detail that varies across all kinds of clusters and has different meanings. [21:28:41] agreed, I'll stick with primary/secondary then