[05:05:54] (03PS1) 10MPGuy2824: Replace addQuotes() with expression builder [extensions/ORES] - 10https://gerrit.wikimedia.org/r/1004348 (https://phabricator.wikimedia.org/T350986) [06:55:00] (03PS6) 10MPGuy2824: Replace addQuotes() with expression builder [extensions/ORES] - 10https://gerrit.wikimedia.org/r/1004348 (https://phabricator.wikimedia.org/T350986) [07:22:15] Good morning! [08:48:05] good morning! [09:00:18] I have an appointment for registration today (finally) [09:00:31] afk from 11am, back in afternoon [09:03:41] Morning Aiko! [09:03:50] registration (?) [09:05:14] it is 11 am for me 😝 [09:05:26] registration of address, something I need to do after moving to berlin [09:05:27] but I haven't done yet ^^" [09:05:43] ah nice! [09:06:52] it is 10am for me haha :D [11:08:30] * isaranto lunch; [13:10:56] 10Machine-Learning-Team: Investigate inference optimization frameworks for Large Language Models (LLMs) - https://phabricator.wikimedia.org/T354257#9554921 (10isarantopoulos) There is a plan to include a prebuilt model server for LLMs very close to what we were discussing which is also based on vllm (kserve alre... [15:32:02] 10Machine-Learning-Team: Support building and running of readability model-server via Makefile - https://phabricator.wikimedia.org/T357913#9555361 (10kevinbazira) [15:35:01] 10Machine-Learning-Team: Add a script for running the Revert Risk model server locally - https://phabricator.wikimedia.org/T352689#9555387 (10kevinbazira) [15:35:03] 10Machine-Learning-Team: Support building and running of readability model-server via Makefile - https://phabricator.wikimedia.org/T357913#9555384 (10kevinbazira) 05Open→03In progress p:05Triage→03Medium [15:49:09] 10Machine-Learning-Team, 10Patch-For-Review: Deploy 7b parameter models from HF - https://phabricator.wikimedia.org/T354870#9555468 (10isarantopoulos) At the moment I'm working on utlizing quantization on our AMD GPUs. AutoGPTQ seems like a prominent solution for rocm. However I'm facing issues with dependenc... [15:53:25] (03CR) 10Ilias Sarantopoulos: "Still havent merged this. I'm marking it as WIP due to the dependency issues experienced while testing the docker image." [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1003488 (https://phabricator.wikimedia.org/T354870) (owner: 10Ilias Sarantopoulos) [15:54:01] 10Machine-Learning-Team, 10Patch-For-Review: Deploy 7b parameter models from HF - https://phabricator.wikimedia.org/T354870#9555471 (10isarantopoulos) From what I've faced so far it seems that there are a lot of moving parts in the area of LLM deployments. Things are moving fast but at the same time they aren'... [15:58:44] (03PS1) 10Kevin Bazira: readability: make model path configurable [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1004692 (https://phabricator.wikimedia.org/T357913) [17:09:57] (03CR) 10Ilias Sarantopoulos: [C: 03+1] "LGTM!" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1004692 (https://phabricator.wikimedia.org/T357913) (owner: 10Kevin Bazira) [17:10:11] going afk folks, cu tomorrow! [17:53:41] (03CR) 10Kevin Bazira: [C: 03+2] "Thanks for the review :)" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1004692 (https://phabricator.wikimedia.org/T357913) (owner: 10Kevin Bazira) [17:54:26] (03Merged) 10jenkins-bot: readability: make model path configurable [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1004692 (https://phabricator.wikimedia.org/T357913) (owner: 10Kevin Bazira)