[15:59:05] Pchelolo: :((( [15:59:19] Amir1: no worries. I'll make a new version in no time [15:59:28] let me know [15:59:49] the fact that cache message is just added to ParserOutput::mText is not amazing either [16:00:14] will make it write to a separate variable [16:00:48] will have more flexibility this way [20:52:43] duesen: can you take a look at https://gerrit.wikimedia.org/r/c/mediawiki/core/+/736306 to see if anything comes to mind? [21:12:56] How do I bypass MediaWikiIntegrationTestCase's "cleverness" and access the actual wiki's DB? [21:14:24] James_F: short of writing mysqli_open with hardcoded credentials, I dont think there's meant to be a way. [21:14:41] Perhaps I can help think of another solution to the underlying issue :) [21:15:35] Krinkle: Our code spends ~60s in update.php populating the wiki with content. Then the @group database flag 'helpfully' creates new tables without the content we just spent a minute injecting into the wiki. I really want to stop having CI inject all the content again. [21:15:59] (Dropping the @group doesn't help, as the cleverness is extra-clever to stop disasters.) [21:16:24] By our, you mean WikiLambda default content pages? [21:17:29] >Software version (if not a Wikimedia wiki), browser information, screenshots, other information, etc: web 2.0 [21:17:31] * Reedy cringes [21:17:45] Krinkle: Yes. [21:18:29] You should really speed up these slow tests (>50ms)... [21:18:34] 1. 64475ms to run MediaWiki\Extension\WikiLambda\Tests\Integration\HooksTest:testCreateInitialContent [21:18:36] Ouch. :-( [21:20:44] 1.6ms? [21:21:58] hehe [21:21:59] if only [21:22:52] The next slowest is "only" 300ms. [21:23:11] James_F: Are there multiple separate phpunit test class hiearchies that need the same sample content, or are/can the integration tests that execise/depend on that logic being present (vs mocked) be under the same overall test suite? [21:23:43] Krinkle: The whole suite needs them, as does the production code. This isn't test data, this is real data and fundamental to the operation of the extension. [21:23:51] If the latter, then one way around it might be to no-op the update hook, and run it as a beforeClass step once only. [21:23:52] It's not "sample content". [21:24:25] Ack, but depending on how low level it is, it might then make sense to not insert it as edits but as table defaults, imported as such in a bit more low level manner [21:24:43] kind likek how interwiki defaults are inserted as a single db transaction, and not as 200 API or SpeicalPage edits [21:24:48] (if that exension were in core) [21:25:23] that would speed it up to 0.5s and then you can forgot the need for optimising it beyond beforeClassOnce etc. [21:25:24] The point of this test is to assert that all the data was injected. We just read the titles from the DB and check that each of the expected items is there. [21:25:38] ack, the assertions are good [21:25:47] but the insertion logic, I'm guessing, is currently going through user journeys [21:26:08] or at least not as low-level as it could be [21:26:19] Hmm. A bulk DB write isn't lovely, but we could do it. [21:26:28] We'd need to re-implement all the SecondaryDataUpdater stuff though. [21:26:34] Which would be very not-lovely. [21:27:10] (Eventually all MW code re-implements EditPage, it seems.) [21:33:19] James_F: Ack, couple things come to mind. 1) how much secondary data does it use. are we talking standard pagelinks/templatelinks/search, or more? 2) how efficient is our existing import.php/WikiImporter logic for batching xml imports by comparison? Might be worth checking if that can be utilized somehow. Those do support secondarydataupdates which appear to be batched somehow, with revisions inserted in a simplified manner as well [21:33:19] (fewer hooks, fewer transactions). 3) alternatively, if it's relatively self-contained it might work to write a script that inserts it the "tranditional" way and then snapshots it into a sql dump and then import that, so you'd have e.g. a sql dump to import with a sceript to periodically update it if the default content has to change and/or if the secondary updates change. this dump would be literal rows to insert into [21:33:19] page/logging/revision/linksupdate etc. That'd trade off speed for potential staleness/need for updating it. [21:35:22] if all the data is in non-core tables only, it might be possible to opt-out of clone logic for those - if and only if the tests are safe to run against any existing wiki db locally regardless of what content it might already have and so it would also have to clean up reliably after itself, which is very hard in practice given during devleopment youd likely to run tests that will fatal every once in a while when code is unfinished so [21:35:23] disabling clone logic isn't really an option even then, but even less so for core tables like page/revision as core's own tests wouldn't pass that way. [21:37:50] another option, outside the box, might me for the data to never exist in the database in the first place. This woudl be a larger change, but e.g. a way to make the data appear existent to MW without ever having to sit in the database. Not unlike some parts of wikidata, and MediaWiki-namespace/Commons/GlobalUserPage. The Z-ids would be reserved and made existent through other hooks etc. This would mean they don't participate in link [21:37:50] tables and search for outgoing links (incomign links is fine). [21:37:53] might be* [23:09:30] Yeah, sadly we need them to be user-editable (with restrictions) so they need to be 'real' wiki pages, and they appear in revision/page as well as our own custom link tables etc. :-( Will have a think.