[08:11:36] 10Machine-Learning-Team, 10API Platform, 10API-Portal, 10Platform Team Initiatives (API Gateway Roadmap): Add documentation about LiftWing to the API Portal - https://phabricator.wikimedia.org/T325759 (10elukey) >>! In T325759#8739389, @apaskulin wrote: > @elukey, I added a section for the [[ https://api.w... [08:42:36] isaranto: o/ [08:42:45] ok if we check the helm chart in the afteroon?? [08:48:21] Sure! [08:48:25] Thanks! [08:49:55] Ping me when u are available [09:24:14] 10Machine-Learning-Team, 10Epic: Migrate ORES clients to LiftWing - https://phabricator.wikimedia.org/T312518 (10isarantopoulos) [10:14:50] * elukey lunch! [10:47:47] * isaranto lunch [12:41:54] 10Machine-Learning-Team: Define SLI/SLO for Lift Wing - https://phabricator.wikimedia.org/T327620 (10klausman) https://docs.google.com/document/d/1NspQtkfyuD_kiYCgms1gRZeFFiAaetnk/edit <- My thoughts so far, comments here or on the doc welcome. There is nothing in the way of concrete numbers on the doc yet. Mai... [12:58:14] 10Machine-Learning-Team: Define SLI/SLO for Lift Wing - https://phabricator.wikimedia.org/T327620 (10elukey) I think that we could start comparing what others did before us, see https://wikitech.wikimedia.org/wiki/SLO#Published_SLOs (maybe the API Gateway one?) We could also involve more specifically SRE, in or... [13:18:45] o/ context: me and Luca discussed that it would be better to try to create the helm chart for the fastapi app using sextant https://gitlab.wikimedia.org/repos/sre/sextant/-/merge_requests/4 [13:19:20] to basically replace the one I have created with helm create here -> https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/904178 [13:21:14] so I'm having a bit of difficulty understanding how I would use it. If I undestand correctrly it is a prerequisite that we have some models/ directory which has some templates that we need to use [13:26:26] isaranto: there is some explanation in the repo's readme [13:26:35] I think it is more or less to customize the scaffolding [13:26:58] if you kick it off without the "models" bit, does it work? Just to understand the baseline of what it creates [13:28:06] I understand the following - correct me if I;m wrong : I create a `fastapi` model and inside it I add the _skel things I require (e.g. service, deployment, ingress) liek `# replace: ingress` [13:28:32] without the models thing it doesnt work. [13:29:29] ok, by looking thoroughly in this patch I think I got it https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/885281 [13:30:21] we have to manually create a model in our case for fastapi applications and ask questions to the user so that they can use the wizard to create a helm chart for their application [13:54:20] ah okok [13:54:59] ahhh wait now I understand, so this is something for us to bootstrap generic ml services [13:55:13] oook sorry I thought it was a little different :( [13:55:46] so we should probably use https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/885281 isaranto [13:56:09] sorry for the confusion [13:57:37] I used that one, but I think I am still confused [13:58:05] it adds a lot of boilerplate stuff we dont need [13:58:35] or at least the values are too many [13:58:53] I'll try to deploy an app with this on minikube to see if ti works then [14:01:01] isaranto: the number of modules that we'll need are not a lot, so the boiler plate shouldn't be that much [14:58:30] isaranto: I was checking the available solutions to customize the uvicorn's access logs for kserve, and found https://stackoverflow.com/questions/62894952/fastapi-gunicon-uvicorn-access-log-format-customization [14:58:52] maybe we can use something similar for the ores legacy endpoint as well, without explicit logging.info(etc..) [14:59:17] for example I see [14:59:18] [formatter_AccessFormatter] [14:59:18] format={"event":"access_log","ip":"%(h)s","status":"%(s)s","method":"%(m)s","path":"%(U)s","referer":"%(f)s","x_session_id":"%(x-session-id)s","x_google_id":"%(x-google-id)s","x_server_time":"%(x-server-time)s","agent":"%(a)s"} [14:59:37] looks nice [14:59:40] u mean add a logging.conf? [15:00:00] I was planning to look into it but initially wanted to have some custom logging [15:00:13] IIUC atm we don't customize the uvicorn's access log, but we just use explicit logging.etc.. right? [15:00:22] right! [15:00:23] yes yes no problem, it is perfectly fine for the initial step [15:00:34] but it makes more sense especially for access logging [15:00:37] I just dropped the link for reference [15:00:41] will add it on a ticket [15:00:42] thanks! [15:00:52] kserve hardcodes their config, and there seems to be no easy way to replace it [15:01:30] I'll open a github issue to ask for some guidance [15:01:44] btw I created a chart with sextant. the wizard doesnt work as it supposed to (was never asked for image +tag ) -which is minor [15:01:55] wow nice! [15:01:58] thanks a lot :) [15:02:09] if you want to provide feedback in #wikimedia-serviceops please do [15:02:10] but I also got a wrong template with an additional {{end}} [15:02:18] it will be very well accepted [15:02:48] I will try to deploy it on minikube and I'll take a look at the MR + patch first [15:30:29] created https://github.com/kserve/kserve/issues/2778 [15:32:28] going afk for today folks! Have a nice rest of the day [15:40:37] bye Luca :) [15:42:27] hi luca [15:42:50] onwenfoewnf [15:42:53] wefewf [15:42:54] wefwef [15:42:54] wefwef [15:42:55] wefwefw [15:42:55] fwef [15:42:56] wefwefwe [15:42:56] fab, ew [15:42:58] fwe [15:43:00] f [15:43:02] wef [15:43:04] wef [15:43:06] we [15:43:08] fwe [16:31:13] heading out now \o [17:12:01] 10Machine-Learning-Team, 10ORES, 10Advanced-Search, 10All-and-every-Wikisource, and 66 others: Remove unnecessary targets definitions - https://phabricator.wikimedia.org/T328497 (10EBernhardson) [18:41:14] 10Machine-Learning-Team, 10API Platform, 10API-Portal, 10Platform Team Initiatives (API Gateway Roadmap): Add documentation about LiftWing to the API Portal - https://phabricator.wikimedia.org/T325759 (10apaskulin) > we are going to add them in Wikitech for the moment (in https://wikitech.wikimedia.org/wik... [20:58:58] hey peeps! i'm going to hire a Data Scientist for a 1-year Wikipedia-based ML project. if you know people who would be a good fit for it, please let them know and/or send them my way. https://wikiedu.org/careers/