[06:45:06] Good morning! [07:48:26] 10Machine-Learning-Team: Support building and running of langid model-server via Makefile - https://phabricator.wikimedia.org/T357382 (10kevinbazira) While testing the locally-built langid model-server, I queried the inference service and received some interesting results. I tested three languages (English, Fren... [07:58:23] isaranto: o/ [07:58:23] https://phabricator.wikimedia.org/T357382#9549568 [07:58:23] ^-- as I was testing the langid model-server build locally, I noticed the isvc struggles to predict English when the input is a short sentence (with about 4 words): [07:58:23] ``` [07:58:23] $ curl https://api.wikimedia.org/service/lw/inference/v1/models/langid:predict -X POST -d '{"text": "My name is Ilias"}' -H "Content-type: application/json" [07:58:23] {"language":"afr_Latn","wikicode":"af","languagename":"Afrikaans","score":0.818276584148407} [07:58:23] ``` [07:59:04] hey Kevin! [08:02:26] I'm aware and if I recall correctly Santhosh is as well. This came up while developing the model server but it doesnt seem like a big issue since our use cases are going to be with longer sentences [08:03:04] Good thing that you brought it up though cause it should be mentioned in the model card. I'm going to ping Santhosh and ask his opinion [08:03:42] ok, great. thank you for sharing more context. [08:03:52] +1 on adding it to the model card. [08:06:17] 10Machine-Learning-Team: Support building and running of langid model-server via Makefile - https://phabricator.wikimedia.org/T357382 (10isarantopoulos) Good catch! Perhaps we could add a small note on the [[ https://meta.wikimedia.org/wiki/Machine_learning_models/Proposed/Language_Identification | model card ]... [08:06:34] thanks for diving a bit deeper Kevin! [11:58:26] this seems pretty close to what we are trying to achieve in kserve https://github.com/kserve/kserve/blob/d3ec987f14310fec61efc57a8c908c854d914c32/python/huggingfaceserver/huggingfaceserver/model.py [11:58:56] the huggingface model server. And it dropped in the last 2-3 weeks [12:14:33] seems nice and does a better generalization than what I was trying to achieve [12:14:36] * isaranto lunch [12:29:24] wow that's cool!! [13:26:31] btw if you've missed this piece of new https://openai.com/sora [14:46:01] Morning all [14:46:14] Sora looks wild [14:52:30] Morning Chris o/ [14:52:39] can't wait to create the movies I'd like to watch [14:53:16] I can really also see how that can go totally wrong as well [15:04:37] isaranto: do you know if the HF model server is more suitable to adding arbitrary endpoints (e.g. /metrics for Prometheus)? [15:06:17] klausman: it is just a custom model server (like the ones we create) based on kserve. It doesn't have additional endpoints other than the ones already available [15:06:52] that said, a `generate` endpoint is being made available alognside predict and explain [15:07:05] Well, that's at least something [15:07:22] I'll throw "see if I can make kserve provide extra endpoints" on my todo pile [15:09:27] and generate seems to only be available in v2 of the api so we'll need to start using that one in a while [15:11:09] the only (proper) way to add extra endpoints is by adding them here https://github.com/kserve/kserve/blob/master/python/kserve/kserve/protocol/rest/server.py#L83 [15:11:21] btw there is a /metrics route available [15:58:32] oh good to know [15:58:42] going afk folks, have a nice weekend! (I may drop by later to respond to update some tasks) [16:07:32] \o [16:07:38] o/ [16:18:01] 10Machine-Learning-Team, 10Wikimedia Enterprise, 10Data-Engineering (Sprint 9), 10Epic, 10Event-Platform: [Event Platform] Implement PoC Event-Driven Data Pipeline for Revert Risk Model Scores using Event Platform Capabilities - https://phabricator.wikimedia.org/T338792 (10lbowmaker) [17:07:50] 10Machine-Learning-Team: Add a script for running the Revert Risk model server locally - https://phabricator.wikimedia.org/T352689 (10kevinbazira) [17:07:52] 10Machine-Learning-Team: Support building and running of langid model-server via Makefile - https://phabricator.wikimedia.org/T357382 (10kevinbazira) 05In progress→03Resolved Support for building the langid model-server using the Makefile was added and it can be tested using: ` # first terminal $ make langua... [19:08:38] 10artificial-intelligence: Google, Microsoft, Meta, X & Others Pledge To Prevent AI Election Interference - https://phabricator.wikimedia.org/T357810 (10Robertonovoski)