[11:09:30] Hi all. Can anyone explain why only Elasticsearch 6.x is supported for CirrusSearch? Is it that significant of a difference to support v7.x? I had a meeting with an AWS analytics specialist solution architect on Friday and this came up. [12:50:17] Another question: In reading through the CirrusSearch README and UPGRADE docs, for relatively large wikis (how are we defining large here?), how significant of a CirrusSearch upgrade should warrant a reindexing of the wiki's data? The only time I upgrade extensions is when upgrading MW, either through a point release or a major release (1.35.2 -> [12:50:17] .3 or 1.35 to 1.36), by just grabbing all new code from the appropriate branch out of gerrit, e.g. REL1_35. [12:50:42] Just trying to figure out these operational considerations before deploying CirrusSearch and AWS Elasticsearch to my production wikis. [15:00:00] I also noticed in the UPGRADE document that it recommends running UpdateSearchIndexConfig.php when upgrading CirrusSearch, but in the README, recommended step 2B uses the same UpdateSearchIndexConfig.php followed by ForceSearchIndex.php. What's the difference in leaving out the second command? [15:00:15] I also noticed in the UPGRADE document that it recommends running UpdateSearchIndexConfig.php when upgrading CirrusSearch, but in the README, recommended upgrade procedure 2B uses the same UpdateSearchIndexConfig.php followed by ForceSearchIndex.php. What's the difference in leaving out the second command? [21:26:22] justinl: heads up America is out on holiday today, so I'd follow up tomorrow with any questions you still have [21:27:20] I can't handle the latter questions, but as for ES 7.x, yes it's a significant difference; any time there's a major version upgrade that usually requires downstream dependencies and the like to be bumped. Upgrading to ES 7 is a goal of ours and on the roadmap, but it won't be done for a while. https://phabricator.wikimedia.org/T263142 here's the phab epic tracking that [21:31:41] Ah sweet, thanks for that link. At least now I have something I can point to when AWS says "but whyyyy?" :) I figured it's just a lot of work to make such a major version change, which this confirms. It sure would be nice to be able to use the new Graviton-based AWS Elasticsearch servers, though. :P [21:45:36] justinl: AWS elasticsearch (assuming you mean the managed service) is great until something goes wrong...no way to ssh into nodes, which would be less of a problem if they didn't block off important chunks of the elasticsearch API (no `reroute`, etc) [21:46:41] and the worst is if you realize you're under provisioned and try to scale up your cluster, they do a red/green deployment and only cut over when ready...which sounds great in theory, in practice that means your already overloaded cluster now has >= 2x the number of nodes it had previously while it's trying to cut over, and then it hits some race condition and gets stuck until AWS support resets it 2 weeks later [21:46:46] (cue vietnam war flashbacks gif) [21:47:32] But yeah the TL;DR is, we want to be on 7 too, but it's not an incredibly pressing need so other work (new wikidata query service streaming updater, etc) has taken precedence thus far