[11:12:34] lunch [13:38:02] \o [13:53:11] .o/ [14:00:06] o/ [14:21:21] computed jina embeddings for the latest partition and the 8 wikis we download enterprise dumps, took ~4days, much faster than qwen3 [14:22:37] ebernhardson: when you have a moment could you take a look at https://gitlab.wikimedia.org/repos/search-platform/discolytics/-/merge_requests/82 and possibly at https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/merge_requests/2621 [14:23:30] if I could deploy them before the week-end they could pick-up from the job I started manually to initiate the first partition [14:31:48] sure i'll look [14:32:18] also thats fast! 4 days is not bad at all [14:33:18] yes, I think that was the time for only enwiki with qwen3 [14:34:33] could be faster I guess with more careful tuning but well... it did not fail and I think used half of the mem I gave to qwen3 [14:35:05] always hard to say how much careful tuning is necessary, but it sounds like this already runs fast enough for purpose [14:37:58] patch also doesn't look too bad, a bit of volume but not particularly complex. i guess HF is doing most of it [14:55:18] I packaged the hf model under hdfs:///user/analytics-search/hf_models/jina-embeddings-v5-text-nano.zip, asked claude to generate a fake one in test/fixtures to have a bit of testing [14:58:02] ebernhardson: would you happen to have the code for load-testing semantic search (T414623) somewhere? I am trying to figure out the CPU utilization so I can estimate, how many OpenSearch nodes could operate on a single K8S worker. [14:58:02] T414623: [Vector Search] Estimate resource consumption at scale - https://phabricator.wikimedia.org/T414623 [14:58:47] pfischer: hmm, david has a locust bit that can do the gpu side, but i don't know we have anything end-to-end [15:28:32] pfischer: poked around, i had this on stat1009 which seems plausibly what i used: https://gitlab.wikimedia.org/ebernhardson/semsearch_loadtest# [15:28:46] i think that's cribbed from david's earlier work [15:29:54] files are at least dated right, i reported around march 2 and these files are ~feb 27 through mar 9 [15:34:24] building a release for discolitycs, crossing fingers that it does not blow up the size of the venv because of torch :/ [15:34:34] oh, indeed :S [15:35:31] I pinned the cpu version which I hope will still be under 1g, the cuda one is about 5g on disk... [15:36:30] sigh ofcourse gitlab token expired... [15:36:53] should we look into how the auto-renew works some day? I've seen there is something but never looked into details [15:37:02] we indeed have an annoying amount of tokens, and i think they cap at 1 year [15:39:01] yes... would be great if they could be auto-renwed... [15:39:16] could create one that expires in 2028, not sure where the limit [15:44:57] claude claims some release things can use the auto-populated CI_JOB_TOKEN, i don't know i've ever tried that. Claims it started working in gitlab 18.4 (we are on 19.1.7) [15:45:27] otherwise, it suggests we could create a group access token on search-platform instead of per-project, then it's at least one token [16:03:01] was afraid of group access tokens because a leak could be a bit more dramatic, I'm probably a bit too paranoid, esp. for search-platform repos, we have nothing that auto-deploys [16:04:34] discolitycs venv size almost doubled from 375Mb to 672Mb, not great but smaller than what I anaticipated [16:05:04] should probably fork this embedding thing out of discolitycs at some points... [16:06:23] sounds manageable, it will slightly slow anything that starts it, but compared to runtimes that should still be ok [16:40:30] errand [16:52:31] annoying maven errors that somehow only error when doing `release:prepare release:perform` : Javadocs must be provided but not found in entries [16:55:30] huh, just noticed they turned auto mode back off on claude. for a few weeks it was on by default [16:55:41] guess they aren't as confident as they were before [16:57:19] and the answer is...we only build javadocs on release. fixing [17:33:22] meh, same error in search-extra-analysis...at least this time i know what to fix :)