[00:07:45] Krinkle: do you know, how is the xhgui mode in production (X-Wikimedia-Debug) so fast? [00:08:07] I am trying to use something similar locally and the function call overhead is enormous [00:08:27] 10x slower on a realistic benchmark and 100x slower on a tight loop of function calls [00:08:50] in production with X-Wikimedia-Debug it doesn't seem so bad, at least it doesn't test my patience so much [00:10:09] and unlike the old MW-native profiler, xhprof doesn't subtract its own overhead, so function calls appear 100x more significant than they really are for optimisation [00:15:14] TimStarling: hm.. you mean php72 with tideways in prod is comparable to regular prod on the same mwdebug, but locally it slows you down 10-100x? [00:16:14] I haven't really thought about it, but indeed it seems tideways' overhead is not 10x for typical requests at least. [00:17:58] I should just do a proper benchmark in production so that I can be sure of what I'm suggesting here [00:18:25] locally a lot of the time is in syscalls [00:18:49] the clock_gettime() syscalls persist even if I use rdtsc mode, but in rdtsc mode I get garbage for results [00:19:07] even with CPU affinity [00:22:11] ok never mind, it sucks in production too [00:22:27] new question: how do you do profiling when the tools all suck? [00:25:21] not totally sure I'm measuring this right, I should stop blurting [00:25:30] TimStarling: right, for when absolute numbers matter, I wouldn't use tideways in general. I'd either write the benchmark on the CLI and run it on mwmaint or some such, or hotpatch mwdebug for end to end things from a browser. [00:26:14] Maybe with some local time spans in an ad-hoc log. It's silly I guess, but I still do that from time to time. [00:26:53] the flame graphs can help to get a break down that isn't biased by func call overhead [00:26:56] I did some profiling for my CommentFormatter patch because the benchmark indicated it was surprisingly slow and I had some trouble figuring out why [00:27:02] I did eventually figure it out [00:27:04] to orient the optimising work going in [00:27:29] https://grafana.wikimedia.org/d/QLtC93rMz/backend-pageview-response-time [00:27:33] might be of interest. [00:27:42] I was surprised we didn't have a dash for it yet [00:27:56] I mostly used xhgui and excimer with flame graphs [00:28:14] right, flame graphs for the time numbers [00:28:18] I've got a very basic flame graph generator for excimer, a few lines of config [00:28:39] xhgui for the break down. that seems sound as well. [00:28:49] I used that approach when optimising the wgConf/SiteConfiguration class [00:29:05] which reminds me, I should take another look at removming the cache file we use in prod. [00:29:56] InitialiseSettings.php has an odd structure, with $logos + [ huge static array ] [00:30:11] if it was all one static array then it would probably be faster [00:30:13] it seems plausible for the cost of reading the file, parsing json, allocating all values etc might be higher than just running InitialiseSettings and extractAll() [00:30:31] given exctratAll does relatively little in practice. [00:30:43] and IS.php presumably mostly in opcache [00:31:33] benchmarks did support the cache when I first introduced it, but that was a very very long time ago now [00:34:14] yeah, it also used serialized php at the time. [00:34:38] when I tested it 1-2 years ago, most time was spent in reading dblists more than anything else. [00:34:59] the wgConf logic became rather marginal in comparison, at least after my optimisations there. [06:11:45] TimStarling: I split logos into a separate file so it would be easier to autogenerate. I can merge it back all into one if there's a benefit to doing so [06:15:22] we'd want to do a benchmark before doing anything, it's quite possible that it doesn't matter [06:16:13] I'd expect $hugeArray += $logos would be faster than $logos + $hugeArray, assuming those operations are implemented efficiently [06:16:38] so maybe that's the simplest thing to try [06:17:30] I mean, PHP has to copy $hugeArray either way if there is another reference to it, but if the refcount is 1 then maybe it can be done in place with += [07:26:57] the order of what gets added to what should be irrelevant in this case [18:20:56] addshore: should we use wdwb-techfocus instead of wikidata-campsite when filing prod errors for WikibaseClient/Repo? [18:21:13] It's unclear to me how/where we're expected to put new tasks of that kind going forward [18:26:25] bd808: seeing a couple of these in the logs: [18:26:27] > MWUnknownContentModelException: The content model 'yaml' is not registered on this wiki. [18:26:31] from labstestwiki [18:42:17] Krinkle: that sounds like something crawling old hiera settings pages [18:42:41] indeed, Hiera pages [18:42:56] I'll see if I can just nuke them [18:43:11] and it looks like MW content handler wants to believe its not possible to create pages with an unknown conten tmodel and fails for that at runtime (e.g. server error, not user error) [18:43:22] I'm guessing the way this can happen is if you create them with the extension installed [18:43:26] and then uninstall the extension [18:44:17] yeah, or in this case you rip support for that old cruft out of the extension :) [19:09:25] Krinkle: well this is fun. deleteBatch.php says that the pages don't exist. I'm guessing that has something to do with the busted content-type [19:12:55] oops. that was a caused by my bad spelling. But with spelling fixed deleteBatch fails with errors about the content-type not being supported. hmmmm [19:14:58] bd808: hm.. got a stack trace for that one? [19:15:13] not seeing it in logstash [19:15:31] s/it/yours/ [19:15:54] I see the action=delete attempt but not the cli one [19:16:10] Krinkle: https://phabricator.wikimedia.org/P16959 -- but it looks like the delete actually happens so I just need to do them one at a time [19:16:43] The action=delete one seems to be failing because we're trying to honour a hook contract that overrides which actions exist by content type [19:17:01] MediaWiki.php(176): Action::getActionName(RequestContext) ->Article->getActionOverrides() -> WikiPage->getContentHandler() [19:17:27] so that wikidata and LQT can render a different deletion handler [19:22:52] I've also gone ahead and added `cloudweb2001-dev` to the "Debugging/Testing" filter from which we also exclude mwdebug* and shell/eval.php [19:23:08] +1 [19:23:44] Krinkle: the bad content-type pages should all be gone now -- https://labtestwikitech.wikimedia.org/wiki/Special:PrefixIndex?prefix=&namespace=666 [19:23:54] <3 [19:24:02] Yuvi had fun picking that namespace id [19:24:45] hehe [19:25:30] btw, regarding P16959, so that's kind of for the same reason. During the deletion, it eventually wants to call the content handler, since custom contnet types can have their own secondary data clean up routines (like how wikitext has link tables to clean up) [19:25:56] I thought maybe it was an avoidable call but seems not in this case [19:26:22] I guess the CLI doesn't have a transaction and maybe no atomic section, so the partial delete just goes through [19:26:37] it might be that labtestwiki db now has some bad references to delted page ids somewhere.. [19:26:47] anyway :) [19:28:13] as an alternative to deletion, we could've aliased the 'yaml' content type to plain text [19:46:21] Krinkle: yes please. I'll see both, but wdwb-tech would be the current correct place for them to land [21:02:16] SameSite cookies https://toot.cafe/@baldur/106697197988699865