[00:07:38] 10Lift-Wing: Factor out feature retrieve functionality to a transformer - https://phabricator.wikimedia.org/T294419 (10ACraze) @kevinbazira & I discussed doing something like this to fetch article text for articlequality models in T285909 in order to get feature parity with ORES, however, I think we could do thi... [06:26:21] 10Lift-Wing, 10Machine-Learning-Team, 10Patch-For-Review: Create a LB service for inference.discovery.wmnet - https://phabricator.wikimedia.org/T289835 (10elukey) Things to do before closing: * evaluate running Istio gateway as Daemonset (https://wikitech.wikimedia.org/wiki/User:JMeybohm/Kubernetes/Ingress#... [06:27:08] 10Lift-Wing: Factor out feature retrieve functionality to a transformer - https://phabricator.wikimedia.org/T294419 (10elukey) We can close this task and use T285909, lemme know what you prefer :) [11:03:12] o/ [11:03:35] I finally found a working patch for the storage-initializer issue, sent it to upstream, let's see [11:04:04] in theory it should allow us to use Andy's original uri path (the one that specified blabla/blabla/model.bin) [11:04:16] and also to fail if no model is returned etc.. [11:04:28] (and have better logging on what gets downloaded and where) [11:06:19] (pr is https://github.com/kserve/kserve/pull/1883) [11:10:08] I have also found https://github.com/kserve/modelmesh-serving [11:10:19] but it seems more complicated [11:10:30] (than our current stack) [11:12:07] https://www.youtube.com/watch?v=la3Y0lXuKRM - there was a talk about it at Kubecon :) [11:48:00] * elukey lunch [14:42:29] created also https://grafana-rw.wikimedia.org/d/Rvs1p4K7k/kserve [14:42:48] not super useful but good to have it [14:43:53] 10Lift-Wing, 10Machine-Learning-Team: Add prometheus metrics collection for Istio and Knative - https://phabricator.wikimedia.org/T289841 (10elukey) Added https://grafana-rw.wikimedia.org/d/Rvs1p4K7k/kserve We now have most of the dashboards that we need: - istio gateway - istio control plane - knative-servi... [16:53:34] Worked with Hugh on https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/730966, in theory now we should have a way to change Host headers in the api-gateway [17:20:32] going afk, have a good (rest of the) day folks :) [17:39:36] Thanks Luca! [20:13:59] 10Machine-Learning-Team, 10artificial-intelligence, 10editquality-modeling, 10Hindi-Sites: Train and test editquality models for Hindi Wikipedia - https://phabricator.wikimedia.org/T252581 (10Halfak) @calbon, waiting on this one for a couple of weeks. Any chance y'all can take a look?