[07:51:30] o/ [08:00:10] o/ [08:51:32] hiya! [10:13:35] hi! [10:13:36] lunch [12:59:08] o/ [13:03:26] \o [13:06:09] o/ [13:25:53] ebernhardson: to make sure I get the concept right, the notion of 'redirect-scope' means we now have redirects in the search space? [13:27:47] mainly asking because this is what I thought but reading https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CirrusSearch/+/1298808/8/includes/Search/Filters.php#26 [13:28:26] dcausse: i guess my idea is there are two ways to represent redirects, they are either their own pages, or they are a property of the target page. The idea is that in redirect scope everything behaves as if the redirects are their own dedicated page [13:28:27] the method is "isRedirectScopedField" but is actually select fields that would be used when the redirect-scope is disabled [13:29:05] oh, yea i suppose that could use a better name [13:29:20] ok got it, thanks! [13:47:47] hmm, did-you-mean metrics are very different now vs last test (~6 months?). Manual selection rate increased from 0.6% to >4%. Clickthrough rate on those manually selected rewrites was 30% (down from ~42% before) [13:48:40] so about 5x as many selected did-you-mean rewrites, but second-click (to an article) decreases ~25% [13:48:57] * ebernhardson has no clue what that means... [13:51:57] if it's 5x more overall that's more clicks? but the 5x brings in more precision issues I guess? [13:53:34] its like we used to have say 600 user-selected did-you-means, increased to 4k. But before we have 252 article click throughs from the 400, now it's 1200 from the 4k. More in the funnel, so more overall coming out [13:54:54] yes... these 5x more did-you-means suggestion were kind of tempting but the actual results they showed not really [13:55:38] really hard to judge what's better... we might solve more information needs in the end but at the cost of an extra "useless" dym click for others [13:55:53] oh, realizing now i'm looking at WIKIS=enwiki for the historical doc, but my analysis is all wikis :P [13:55:59] one sec... [13:56:01] oh [14:00:10] oh and i should clarify, those numbers were control vs control, just now vs early october. Will see what it spits out for enwiki only, takes a few [14:03:50] ah so with no underlying changes... yes that would have been curious [14:05:00] still similar, selection rate is still 5% [14:06:10] sec i'll gen the full html report bit [14:08:48] should be comparable, athough the second (T407432) covers two weeks instead of one: old https://people.wikimedia.org/~ebernhardson/T390858-prefix_len-enwiki.html vs new: https://people.wikimedia.org/~ebernhardson/T407432-prefix_len-all_wikis.html [14:08:49] T407432: Follow-up AB test of dym language model variants - https://phabricator.wikimedia.org/T407432 [14:09:15] the nominal difference is the variant field here has title+redirect.title+opening_text, where the old variant was only opening text [14:09:54] control = 2 char prefix, title+redirect.title. d1 = control with 1 char prefix, d1v = modified suggest language field, 1 char prefix [14:10:44] doh lemme fix name...the T407432 is enwiki only not all wikis (i wrote that command line earlier :P) [14:13:14] correctd (also not a full publish, just a draft): https://people.wikimedia.org/~ebernhardson/T407432-prefix_len-enwiki.html [14:22:43] the "Display Rate" does not include automatic rewrites I suppose? [14:23:11] dcausse: only when the auto-rewrite gets a suggestion of it's own :) [14:23:21] ack [14:24:50] default_1v is *only* opening_text? [14:25:18] dcausse: in the old test yes. In the new test its titles+opening [14:25:38] i almost called it d_1v.v2 but seemed worse :P [14:25:58] :) [14:26:36] https://people.wikimedia.org/~ebernhardson/T407432-prefix_len-enwiki.html is the new one? [14:26:45] dcausse: yes, the newer T number [14:27:09] ignoring that big shift, this mostly says to me that titles vs titles+opening_text isn't much different. [14:27:46] old & new are both english [14:27:52] ? [14:28:03] yes, both enwiki. old is 1 wk of data new is 2 [14:28:58] something's odd, selection rate is 0.5% on the old one vs 5% on the new one [14:29:54] indeed [14:30:24] you regenerated the old report perhaps? [14:30:42] no, the old report is the original from october [14:31:40] selection rate has always been low, historically i mostly ignored the user-selected and looked at the auto-rewrites. But this is showing much higher numbers now [14:35:36] this is on the same number of days? [14:35:48] dcausse: twice as many on the new test, two weeks vs 1 [14:35:52] ok [14:36:41] but yes unless I'm reading the report the wrong way it does confirm yet again that our intuition on this opening_text was not correct [14:37:27] yea i agree. I don't understand why, but it doesn't appear to help [14:37:55] and we're sure control is prefix 1 [14:37:58] I mean 2 [14:38:16] yea, 2 char for control [14:40:25] hipothesis: opening_text is perhaps adding more noise making the typos found with prefix 1 treated as false positives by the phrase suggester [14:42:32] hm perhaps no, true for display rate which goes back to be equal to control but still higher than control in auto rewrites for both 1 and 1v [14:44:30] perhaps time to think about some else than the phrase suggester if we want to improve search on typos :) [14:45:01] s/some/something [14:45:43] yea, i imagine a more ideal solution would be query suggestions against a corpus of "good" queries that have previously satisfied users or some such [14:46:12] or maybe we don't worry about suggestions too much...never clear [14:59:00] might poke through some data later today and try and understand that lift from 0.5% to 5% on user-selected rewrites. Curious if at last a sampling of rewrites and final clickthroughs look sane and human-ish [16:54:40] what do y'all think is a good `Xms` number for the 8 GB beta cluster instance? I was gonna go with 6 GB [17:19:42] inflatador: rule of thumb is generally half (4gb) [17:20:15] dinner [17:27:52] dcausse ACK, I'll update the hiera value then