[10:05:38] in looking at the resourceloader grafana dashboard I see that on around Oct 20 there was both a large increase in the map-js rate and in the map-js cache hit ratio, which have stuck since then. curious if anyone knows what those are related to [11:49:46] apergos: guessing from “map”, source maps? https://sal.toolforge.org/log/TnkUS4sBxE1_1c7s8y51 [11:50:17] huh. wonder what those are, guess I'll find out [11:51:29] you should be able to see unminified sources in browser dev tools even without RL debug mode, basically [11:51:33] interesting. [11:51:37] it’s pretty cool [11:51:55] that seems a huge increase for the small number of people who would be using this feature, no? [11:52:17] * Lucas_WMDE looks up the dashboard [11:53:03] I quote from the task: "for people that have them enabled (and are debugging)" [11:55:50] I mean, not sure how meaningful “increase” is for a new feature [11:55:55] anything is going to be an increase over 0 ;) [11:56:07] the absolute rate looks reasonable to me, comparing to minify-js and minify-css [11:56:32] their Y axes are measured in kiloops/s, map-js’ Y axis is missing the “K” ^^ [11:58:21] well if there was 1 ops/s and now it's 6, that's a large leap [11:58:47] I'm looking for reference at https://grafana.wikimedia.org/d/000000066/resourceloader?orgId=1&from=1696334312628&to=1698926312628 [11:58:50] 30 day range [11:59:10] I think before October 20 it was only enabled on mediawikiwiki [11:59:19] but let’s check the git history instead of my partial memory ;) [11:59:34] ahhhh that's the missing piece, that it was already partially deployed. that would make sense [11:59:35] ah, no, all of group0 [11:59:41] hmm [12:00:10] re the task quote – I’m actually not sure what “have them enabled” refers to… I guess it’s possible to disable source maps? but AFAIK they’re enabled by default in browsers [12:00:16] (I haven’t had to enable them, at least) [12:00:27] yeah group0 isn't liable to have a bunch of traffic to it by debuggers except for mediawikiwiki I would think. checks out [12:00:28] so “and [who] are debugging” will be the main filter [12:00:50] uh huh [12:01:02] also, I’m assuming the minify-js and -css rates are still lower than the “real” rates because they don’t take the client-side cache (local storage?) into account [12:01:12] and possibly not even the front / HTTP-level cache? no idea [12:01:18] me neither [12:01:28] (because otherwise, “1 in 1000 requests uses dev tools” would still be suspiciously high tbh) [12:01:32] but I don't feel the need to go down that rabbit hole. not today at least :-) [12:01:35] ^^ [12:04:24] (thasnks for shedding some light on this, really appreciate it!) [12:13:06] np :) [20:10:55] apergos: the map-js rate of ~1-5 per second is fine yeah. It's out of 32,000 requests at the top of the dashboard in terms of Varnish frontend http requests. [20:12:37] the minify metrics are tehre mainly for diagnostic purposes, they are not themselves tied to an alert and don't have a problematic threshold level per-se, besides that large changes are always worth understanding I suppose, which is exactlty what you did :) [20:14:16] we do have an alert on the "Minify calls per backend request" metric for example (on the https://grafana.wikimedia.org/d/000000402/resourceloader-alerts dashboard). If that fires, the lower level metric on the bottom of the RL dash can help figure out why it might be high. E.g. is there a drop in cache hits, or a rise in extra traffic that tends to cache-miss. [20:18:04] of the 32,000 req/s, about 100 reach MW as a cache miss, and of those ~5 seem to be source maps requests. There is likely slight bias on backend reqs toward debug requests than regular requests, given that debug requests are more unlikely to stick in Varnish. [21:13:59] thanks for the extra info, very useful! I assumed there was nothing actually broken (because if it's been that way from oct 20 and there was something broken, others than me would have noticed and changed it) but did want to understand the source, a learning opportunity you might say :-)