[05:39:42] ^ cleaned up git message slightly. i had some general questions about the preseed, but mostly my only actionable comment is keeping `A:wdqs-test` included in `A:wdqs-public` [09:55:45] lunch [13:18:04] o/ [13:59:39] \o [14:13:22] o/ [15:37:13] workout, back in ~40 [16:05:21] wonderful historical abandoned queries (what does this mean?): tayps of wlding difacts [16:06:09] at the time, google could correct it but bing couldn't. Looks like bing can fix it today [16:10:25] i modified T376161 and T375554 to ensure that the identification of the apparent component(s) of abandonment are in the 8-pointer (i think that was what our discussion was around, but if i misunderstood and it should go back to the 5-pointer, please lmk). i also moved the namespace 0 requirement to the 5-pointer (i *believe* that part isn't too bad). [16:10:30] T376161: Classify fulltext search abandonment: sampling - https://phabricator.wikimedia.org/T376161 [16:10:30] T375554: Classify fulltext search abandonment: English, French, Spanish - https://phabricator.wikimedia.org/T375554 [16:23:22] back [16:35:56] ryankemper if you have time, just updated https://gerrit.wikimedia.org/r/c/operations/puppet/+/1076841 w/your suggestion [16:40:11] inflatador: change looks good but you hadn't pulled down my patchset so my beautiful git commit formatting changes aren't there anymore D: [16:40:22] ez fix, lemme upload a patch rq [16:40:55] wait maybe my changes never uploaded, or gerrit is bugging for me [16:42:23] Ah sort of gerrit bug...so if you go to e.g. https://gerrit.wikimedia.org/r/c/operations/puppet/+/1076841/4 it won't show the commit message for that patch, but rather the latest patchset. I can't imagine why that would be the case [16:43:12] ryankemper: you can select different patchset's at https://gerrit.wikimedia.org/r/c/operations/puppet/+/1076841/5//COMMIT_MSG [16:44:37] ebernhardson: yeah I know you can click the file manually but it just seems odd to me that the commit message that shows up top is the latest patchset [16:45:52] (shows up top if not manually clicking the commit msg file) [16:46:05] I guess that's if you rebase when you push? [16:46:31] inflatador: I updated the commit msg btw so it should be all good [16:46:41] need to take the dog out, he's protesting right now [16:48:44] ryankemper ACK, thanks [16:49:38] ebernhardson or anyone else, do you know what the CirrusSearch doc size metrics look like in Prometheus? I can't seem to find them w/explorer. I think the metrics are generated here: https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/CirrusSearch/+/refs/heads/master/includes/DataSender.php#701 [16:55:16] inflatador: they are on a dashboard, sec [16:56:11] ACK...maybe someone's already done the work for me then ;P . Was looking at updating https://grafana.wikimedia.org/goto/Va6cNSkNg?orgId=1 and friends to use prom [16:56:15] inflatador: they are found here, but some of the data looks suspicious https://grafana.wikimedia.org/d/JLK3I_siz/elasticsearch-indexing?orgId=1&viewPanel=58 [16:56:44] oh, thats still using graphite [16:56:45] hmm [16:59:25] inflatador: in terms of a bare query, something like: histogram_quantile(0.5, sum by (search_cluster, le) (mediawiki_CirrusSearch_update_doc_size_kb_bucket)) [16:59:37] inflatador: but i'm not sure why it only has cloudelastic [17:00:13] hmm, also being in DataSender i think that isn't invoked when SUP is being used, so that data might only be wikitech [17:00:21] (the only remaining wiki on old updater, afaik) [17:00:34] not sure if SUP has a comparable metric, hmm [17:02:20] Ah, interesting. I've just been doing rough find/replace on the dashboards, sounds like this needs a bit more consideration [17:02:23] we return document length with the api request as `size_limiter_stats`, but i don't see it used anywhere. Open question i suppose is if SUP should report those, or if the cirrus bits should record it earlier than in data sender during doc building [17:03:08] yea, perhaps open up a ticket. We can query it from prometheus but the data is less meaningful these days because it's collected in the "wrong" place [17:04:28] oh i guess the size limiter stats are also a little different, data sender is looking at the total json encoded doc, instead of per-field [17:05:19] prometheus data is also oddly only available for the last few days :S [17:08:22] oh, i bet codfw / eqiad are such tiny numbers there because they are still recording archive index updates, and those are tiny [17:18:41] * ebernhardson ponders how terrible it would be to try and join virtual pageviews to search satisfaction using actor id's and query strings [17:23:03] the other way around, probably more reliable, would be to subscribe to the virtualpageview events and re-emit them to search satisfaction with our extra data [17:24:06] hmm, i suppose thats probably a much better idea, join in the browser instead of trying to do backend data magic [17:28:03] ebernhardson created T376189 for further discussion on this, feel free to fix the description as I'm sure I missed something ;P [17:28:04] T376189: Replace the "CirrusSearch.$cluster.updates.all.doc_size" with a new metric that works with SUP - https://phabricator.wikimedia.org/T376189 [17:31:03] added some notes that are hopefully clearer than my mess above :) [17:42:10] * ebernhardson realizes schemas on mediawiki.org/wiki/Schema:??? are all locked for editing...i wonder if they were supposed to migrate somewhere [18:09:26] dinner [18:59:54] ebernhardson they are doing that SUL/SSO migration thing today. Might be related [19:00:04] https://wikitech.wikimedia.org/wiki/Wikitech/SUL-migration [19:03:06] ryankemper updated https://gerrit.wikimedia.org/r/c/operations/puppet/+/1076841 if you have time to take a look again [19:03:25] inflatador: nah, i'm still logged in. The schemas were indeed moved to a git repo at some point instead of being on-wiki. Might have been helpful if the pages could say that, but since they are stored as json pages and not wikitext there isn't really a place to stuff an informative template or something [19:03:35] also i was mistaken, they are on meta.wikimedia.org and not mediawiki.org [19:03:42] inflatador: okay now you're just trolling me with the commit message at this point :P [19:03:49] * ryankemper is onto your games [19:04:11] he's just editing the local version and not pulling down the updated patch [19:04:20] ryankemper oops, I should have pulled down the updated patch. will fix [19:04:44] i like `git review -d ` for pulling it down. But i also mostly ignore all local branching so that might not work as well depending on your process [19:06:20] Yeah the way it handles local branching is so silly [19:06:29] yeah, I typically do that, just forgot to before making my last change [19:11:53] ebernhardson: any chance you're available to pair starting in 5-10ish mins? the 2pm time might not work for me today depending on when the dog wakes up [19:12:07] works for i.nflatador if it works for you. otherwise the normal time is fine there's just a chance i'll miss it [19:13:22] ryankemper: ya i can [19:13:33] i do have another meeting at 1 though [19:13:52] ebernhardson: cool, i'll move time to 12:15. figure sooner we start the better so you have some time before next meeting [19:17:06] inflatador: in https://meet.google.com/eki-rafx-cxi now [20:54:41] small CR for adding kube_env config if anyone has time to look https://gerrit.wikimedia.org/r/c/operations/puppet/+/1077096 [21:02:18] * ebernhardson wonders how we are supposed to represent a gerrit patch that Depends-On a gitlab MR