[08:40:10] Guten tag!! [09:08:19] Καλημέρα! [10:03:50] :D [10:23:45] aiko: if you want we can discuss about pydantic after u wath some material [10:25:08] you could try to do something "hands on" e.g. add it to a model server. You can check how we have used it in ores legacy. Also the pydantic docs are really good ! https://docs.pydantic.dev/latest/ [10:34:09] 10Lift-Wing, 10Machine-Learning-Team: Investigate increase p99 latencies in ml-serve-eqiad - https://phabricator.wikimedia.org/T352958 (10isarantopoulos) | {F41649526} | {F41649528} | On the left are the p99 latencies before the improvements and the redirect fixes. On the right are the latencies after that.... [10:38:17] isaranto: thank you for the pointers! I found it very useful, and muniza is also gonna use it in knowledge integrity. I'll try it hands-on :) [10:57:33] 10Lift-Wing, 10Machine-Learning-Team: Investigate increase p99 latencies in ml-serve-eqiad - https://phabricator.wikimedia.org/T352958 (10isarantopoulos) 05Open→03Resolved [11:11:57] ---^ nice! [11:55:22] 10Machine-Learning-Team, 10Patch-For-Review: Upgrade the readability model server to KServe 0.11.1 - https://phabricator.wikimedia.org/T348664 (10isarantopoulos) [[ https://github.com/catboost/catboost/commit/2b48e42aceff9ef434ad7dd6aec355514c77bb35 | The PR fo Cgroups ]] v2 support in catboost has been merge... [11:58:33] 10Machine-Learning-Team: Investigate inference optimization frameworks for Large models - https://phabricator.wikimedia.org/T354257 (10isarantopoulos) [12:03:27] * isaranto lunch [12:36:34] * aiko lunch 2 [14:09:22] 10Machine-Learning-Team: Optimize response performance for the article-descriptions model-server - https://phabricator.wikimedia.org/T353127 (10Isaac) > is there a dataset/set of requests you we can use to run load tests? @isarantopoulos I didn't do anything special in selecting articles for load-testing beyond... [14:39:59] 10Machine-Learning-Team: Investigate inference optimization frameworks for Large Language Models (LLMs) - https://phabricator.wikimedia.org/T354257 (10isarantopoulos) [14:40:43] 10Machine-Learning-Team: Investigate inference optimization frameworks for Large Language Models (LLMs) - https://phabricator.wikimedia.org/T354257 (10isarantopoulos) [14:40:46] 10Machine-Learning-Team, 10Goal: Goal: Inference Optimization for Hugging face/Pytorch models - https://phabricator.wikimedia.org/T353337 (10isarantopoulos) [14:41:51] 10Machine-Learning-Team: Investigate inference optimization frameworks for Large Language Models (LLMs) - https://phabricator.wikimedia.org/T354257 (10isarantopoulos) [14:47:55] 10Machine-Learning-Team: Investigate inference optimization frameworks for Large Language Models (LLMs) - https://phabricator.wikimedia.org/T354257 (10isarantopoulos) [15:59:36] I've been doing some reading on inference optimization from the resources I shared on the task above. I need to write some descriptions more properly on what we need to do [16:38:54] 10Machine-Learning-Team, 10ORES, 10MW-1.42-notes (1.42.0-wmf.13; 2024-01-09 ): Replace use of $wgCommandLineMode in ORES - https://phabricator.wikimedia.org/T353750 (10matmarex) 05Open→03Resolved [16:50:46] * isaranto afk