[08:04:21] <dcausse>	 remote cluster config seems to have broken
[08:08:20] <dcausse>	 yeah it still has elastic1034.eqiad.wmnet:9500 for chi -> omega :/
[08:08:30] <dcausse>	 the restart probably dropped some transient settings
[08:33:39] <dcausse>	 ah no chi still has the old .persistent.search.remote layout in addition to .persistent.*cluster*.remote 
[08:41:22] <dcausse>	 wow I can't seem to drop it
[08:45:46] <dcausse>	 re-uploaded the seeds and errors are dropping
[08:45:54] <dcausse>	 seems like a bug
[08:46:17] <dcausse>	 elastic is probably loading these old compat settings overridding the new ones
[08:46:38] <dcausse>	 but if we can't drop the old ones we're stuck :/
[08:46:52] <dcausse>	 will file a task
[09:09:51] <gehel>	 Another quick run to the hospital with Oscar, back as soon as I can.
[09:10:06] <gehel>	 Just another check, nothing to worry about.
[11:28:04] <dcausse>	 lunch
[14:47:59] <inflatador>	 greetings
[14:49:43] <inflatador>	 dcausse link me that task when you get make it, I know e-bernhardson discovered that the CCS settings aren't in puppet (and were missing from some hosts), we probably want to fix that
[14:50:39] <dcausse>	 inflatador: filed T305689 but unsure how to fix this yet, seems like there's some bug in elastic
[14:50:39] <stashbot>	 T305689: Elasticsearch chi@eqiad cluster contains invalid cross cluster settings - https://phabricator.wikimedia.org/T305689
[14:52:31] <inflatador>	 SMH at the automated bot reply ;(
[15:05:25] <dcausse>	 errand
[15:06:47] <inflatador>	 ^^ I asked our Elastic instructor about this one via LinkedIn. Not holding out too much hope for a reply, but who knows
[15:11:57] <gehel>	 I have a late time interview tonight. I'll take a long break before that to compensate. Back later or good weekend to you all!
[15:17:24] <dcausse>	 inflatador: thanks!
[15:19:27] <inflatador>	 Skipping the gaming mtg, I have a pretty bad headache
[15:27:18] <dcausse>	 take care!
[15:27:33] <dcausse>	 skipping it too and going offline, have a nice week-end
[15:30:13] <inflatador>	 Thanks, you too!
[16:33:27] <inflatador>	 taking off, have a great weekend all!