[07:47:56] <elukey>	 hello folks
[07:48:02] <elukey>	 very nice new website for kserve: https://kserve.github.io/website/0.7/modelserving/control_plane/
[09:03:48] <elukey>	 left a note in the kserve slack channel about our doubts for tornado/asyncio/etc..
[09:03:56] <elukey>	 let's see if a merciful soul answers :D
[10:04:21] <wikibugs>	 10Machine-Learning-Team, 10observability: Improve ORES observability - https://phabricator.wikimedia.org/T299137 (10elukey) @Halfak thanks a lot for the detailed summary, it helps a lot! I would definitely like to have some logs indicating what happened during a certain timeframe, to be able to correlate spike...
[11:25:02] <wikibugs>	 10Lift-Wing, 10Machine-Learning-Team (Active Tasks): Load test the Lift Wing cluster - https://phabricator.wikimedia.org/T296173 (10elukey) There was a constant throttling (cpu) for the queue-proxy containers, now it should be gone.  The following link can be good to see how cpu/memory wise are doing with revs...
[11:27:03] <wikibugs>	 10Lift-Wing, 10Machine-Learning-Team (Active Tasks): Load test the Lift Wing cluster - https://phabricator.wikimedia.org/T296173 (10elukey) I also left some questions in Kserve's upstream slack channel about how tornato/asyncio/etc.. are working and if we should follow specific guidelines to make our code as p...
[11:28:59] * elukey lunch!
[16:16:58] <elukey>	 the kserve upstream folks gave me this link https://kserve.github.io/website/master/modelserving/v1beta1/custom/custom_model/
[16:17:08] <elukey>	 so the TL;DR seems to be:
[16:17:43] <elukey>	 - the async workers are used only if we mark code as co-routine etc.. and it seems mostly helping in the transformer, when calling external http services etcc.
[16:18:09] <elukey>	 - the tornado workers have an async loop, and code runs in it
[16:18:27] <elukey>	 so blocking/cpu-bound code reduces parallelism a lot
[16:18:53] <elukey>	 - we could deploy ray workers (not still super sure how) in order to have a separate pool for models
[16:19:02] <elukey>	 so the tornado loop would only dispatch calls to the models
[16:20:20] <elukey>	 it may be as simple as adding the annotation to the code (last famous words)
[17:33:08] <wikibugs>	 10Lift-Wing, 10Machine-Learning-Team (Active Tasks): Load test the Lift Wing cluster - https://phabricator.wikimedia.org/T296173 (10elukey) From IRC:  ` the kserve upstream folks gave me this link  https://kserve.github.io/website/master/modelserving/v1beta1/custom/custom_model/ so the TL;DR seems to be: - the...