[07:47:06] re '-local:example' probably an oversight... testing the negation is completely ignored... the only valid use-cases that I can think of is searching for images that are not local, not sure that's very interesting, at least I don't recall anyone requesting this... [10:51:39] lunch [12:46:58] \o [12:58:27] o/ [13:14:39] * ebernhardson is up to 14 commits and probably needs to squash a number of these... [13:15:35] :) [13:28:04] I seem to get parallel (2) features runs with cindy passing OK but only if when things are preloaded (bumping special:undelete browser timeout from 2 to 3min and completely skipping BeforeOnce inits via an env var) [13:28:44] ~2m16 for a run [13:28:59] 2m16s, for the whole suite? wow! [13:29:34] bumping to 4 in parallel I'm down to 1m30 but this gets flaky on some tests that still test updates [13:30:08] preloading from an existing db takes ~1m30 [13:30:28] so that sounds like basically it needs some pre-stage the runs all the hooks instead of trying to run hooks-on-demand? That honestly sounds simpler than the current tag tracker [13:30:34] but yes all this seems to suggest that loading the data live during the tests is what makes the system slow and flaky [13:31:20] potentially whatever currently inits the tag tracker before forking could instead initialize the indexes? [13:35:15] it also sounds like we kindof have two different suites here, although not sure what do with that. One suite is focused on "can we query docs back out of search", the other is "can we observe the updates we expect on edit" [13:36:02] yes.. and since tests are run alphabetically I suspect the update_* features runs in parallel when concurrency is enabled [13:37:06] I'm not sure how to improve the pre-loading tho... will have to change quite a few things [13:37:40] it's a bit of a half measure, but maybe combine a bunch of hooks that just create pages? [13:38:47] Could we capture writes to OS like WireMock (and other mock servers in proxy-mode) allow? We did that for the SUP integration tests too, where you can run a test in capturing mode and in pre-initialized mode. [13:39:21] yes possibly but still running with the tracker I worry of possible races, I saw that there's a pending state but this gets harder to debug if something's fragile there [13:40:02] pfischer: possibly but ideally we also want to test what happens in mediawiki and also the fact that the mw db is in sync with the search index [13:40:55] I think ideally we would have a pre-load phase pushing data to MW from a king of dump format [13:41:05] s/king/kind [13:41:54] but the dump format must be human readable so that we can adapt it when needed [13:42:13] dcausse: So restore a maria dump and then bulk-update the index from the db-state? [13:42:55] mw has an import format, but i've never really looked at it. A single giant array of mwbot edit's seems plausible to me, but does probably run a bunch of code the import might skip [13:42:56] yes that's exactly what I do when I run the second time for opensearch 2, I destroy the opensearch container and re-index from the existing mediawiki db [13:43:48] if we could have a single mw maint script run to push the data that'd be ideal, no api/nodejs overhead [13:44:49] dcausse: Are the indexes name-spaced per test or how did you avoid collisions? Did you use isolated infra per test? [13:45:53] we don't have strong isolation, tests tend to use a specific set of pages [13:47:11] eventhough we have @Before hooks that push the data for a specific test the reality is that we have more of a global test corpus [13:47:36] yea very much so, if anything the hooks imply more isolation than exists. [13:49:04] ok I think I'll push what I have, to at least make the second run with opensearch 2 faster (verify that it's not too flaky in real life) [13:49:29] then we'll have to think about how to improve that pre-loading phase [14:34:02] got it down to 7, but 3 are pre-patches that clean things up before the impl [14:37:50] nice! [14:40:12] mostly done on the cirrus side, although i probably missed a thing or three. But going to start working out the streaming updater side and hopefully come to a conclusion about how much info sup needs, i'm optimistic it's only a small schema change for the new doc fields, but also have to consider how cirrus now updates 2 pages for 1 redirect edit [14:40:32] and reviewing where we recognize/restrict redirects already [14:43:28] sure, possible annoyance (can't remember when the decision is made) is cross-namespace redirects and the bits that detect the target index [18:05:03] dinner