[15:47:04] Krinkle: I know I already asked you that long time ago, but I lost my notes on that, apologies. What's performance team doing with the parser limit report, the one in the comment in parser output, not in JS vars? [15:47:40] I want to move it to post-parse transform - that will allow us to use parser cache much more, and will make canonical and non-canonical parser options equivalent [15:48:05] but I wanna test the change with whatever this report is used for [16:04:20] Neither of them are used in any significant sense with automation. There's a few gadgets that read them some of which we use ourselves. So long as the data remains measured in PHP and available to all users in view source in some way I don't think the details matter [16:05:08] What kind of change would improve cache use around that? [16:10:47] Krinkle: Something like this: https://gerrit.wikimedia.org/r/c/mediawiki/core/+/727361 - very very WIP [16:11:14] basically now if you parse with non-canonical parser options (e.g. ParserOptions::newFromAnon instead of ParserOptions::newCanonical) [16:11:21] the parser cache is skipped [16:11:28] because of limit report. [16:12:34] and on top of that people need to think whether they need canonical or non-canonical options. Now both will be the same thing. [16:16:48] Pchelolo: I think some history analysis might be useful here if you haven't already. I don't understand why this would be conditional or related to canonical-ness in any way. [16:17:23] My sense is that this is always enabled in practice and there isn't a reason not to include it ever other than by site configuration [16:18:16] yeah, I'm still experimenting, T269234 has a little context, but I'll have more context when I'm done experimenting. [16:18:26] T269234: Eliminate the split between canonical and non-canonical ParserOptions - https://phabricator.wikimedia.org/T269234 [16:19:36] I suspect the difference is accidental and not intended that way, the some refactoring may be cause it to be skipped in some cases and someone tried to preserve that [16:20:29] The notion of canonical options seems important though. We need that for link tables and to generate canonical entries for users with custom preferences [16:20:41] But that isn't related to perf limit [16:22:07] Maybe it would be simpler to generate it always in the parser based on then-current site config and remove all the rest complexity options etc unless history shows a use case that needs it [16:22:30] So it would just be part of the html blob and nothing else aware if it [16:25:05] we need to not have this report in html more then we need to have it. For example - parsing messages, parsing for transclusion, etc, etc [16:26:06] James_F, legoktm : I just came across ConfigRepository. It's not used anywhere, and while the idea sounds nice enough, I'm not sure why we have it on core. Can you tell me [16:26:30] duesen: It's work in progress. [16:27:03] James_F, legoktm: I just came across ConfigRepository. It's not used anywhere, and while the idea sounds nice enough, I'm not sure why we have it on core. Can you tell me more about why this exists? I'm asking because in the context of the move to kubernetes, we are currently trying to re-think how configuration works. [16:27:20] James_F: oops, double-send. [16:27:42] James_F: yea, but it has been in progress for three years, and I don't see any tickets/plan/discussion around it. [16:29:30] duesen: Most of the discussion was at Hackathons over the years, I think? I forget, sorry. [16:30:23] hm, ok. right now, it looks like dead wood. [16:30:35] The principal objective was to have fewer production config deployments and let things be set on-wiki. [16:30:47] But Kunal's much more of an expert on it than I am. [16:30:55] before we can think about managing config options via a web interface, we have to have a good think about how we structure and store config. [16:31:23] ok. I expect we'll be touching on this when we work on config loading in november. [16:31:24] This is that thinking. [16:31:33] "We"? [16:31:46] As part of the pipeline work? [16:32:05] Yes, together with RelEng. I have been meeting with Dan to discuss it. [16:32:37] PET together with RelEng [16:33:20] The short term idea is basically to load from JSON into globals variables. But the slightly longer term goal is to, well, not do that. [16:38:19] James_F: here's the problem statement: https://phabricator.wikimedia.org/T292402 [16:40:36] ...we'll work on the technical solution in November. [16:43:08] Pchelolo: ah, right. So from that POV, page views are the minority. Got it :) [16:53:08] duesen: Oh, interesting. I've not seen that TDMP thing yet? [16:53:59] duesen: And yeah, the JSON into global variables thing for k8s is my idea from years ago, very glad it's being worked on as the interim solution. :-) [16:55:31] James_F: it's supposed to start phase 1 on the forum next week, after initial review by the chairs. And yes, I saw the tickets you wrote :) [16:55:49] Excellent. [16:56:14] By the way, moving away from global variables entirely may actually easier than one might thinks. I want to make that a stretch goal for the Q2. [16:56:57] And I just sent you and Kunal an invite to chat about this next week. [17:00:29] James_F: working on this proposal i came to realize something: currently, we structure configuration by what it is used for (which extension or subsystem). But the decision of where we want to load that config from is based on entirely different criteria, e.g. does it differ per wiki? does it differ per data center? do we need to override it quickly to respond to incidents? how bis is the impact of misconfiguration? can it easily be undone? [17:01:34] These are all orthogonal to the question of where the config is used. Having the ability to configure different config sources per extension or other component doesn't seem very useful any more. [17:01:39] duesen: Yeah, agreed. It's a real mess. [17:04:51] What I envision for the future is basically: 1. load config trees from different sources 2. ????? 3. have config trees for different components [17:40:05] James_F: oh btw, you may find this amusing: https://phabricator.wikimedia.org/P17424 [18:00:07] addshore: hey, could you review or get someone to review? I'm kind of stuck on this: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/722427 [18:02:13] that doesn't look too scary (trying not to forshadow) [18:02:42] +2ed [18:13:11] duesen: Oh yeah, I saw. It reminded me just how much some extensions (e.g. Cirrus) really have huge numbers of config flags, whereas 'modern' ones mostly compact that into a single settings object. Bit of a mess. [18:14:15] Of course, using complex arrays adds more complexity [18:19:37] True. [19:51:30] duesen, James_F: the idea behind ConfigRegistry (and most of it was Florian's work) was a place to store *metadata* about the config settings (e.g. description, public/private, eventually types, etc.) rather than just the plain value...but the work was never finished partly because I didn't have enough time to review it all [19:57:03] I'm a bit behind on the phab tickets, but I'll try to review everything before the meeting next week [20:00:01] addshore: ty! [22:59:11] > The principal objective was to have fewer production config deployments and let things be set on-wiki. [22:59:53] we wrote an on-wiki config registry for GrowthExperiments, weren't aware of ConfigRegistry [23:00:32] is it in a usable state? would be interesting to see if the two are compatible [23:00:55] currently the meta-configuration for variable type, help text etc. is a bit disorganized [23:01:44] https://www.mediawiki.org/wiki/Growth/Community_configuration has some high-level ideas