[07:38:01] Guten tag o/ [08:44:35] (03CR) 10Ilias Sarantopoulos: [C: 03+1] Add deprecation warning for ORES [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/988488 (https://phabricator.wikimedia.org/T349632) (owner: 10Klausman) [08:46:43] (03CR) 10Ilias Sarantopoulos: revertrisk: add initial script for local-run (031 comment) [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/988666 (https://phabricator.wikimedia.org/T352689) (owner: 10AikoChou) [08:52:29] * isaranto running an errand be back in 30' [08:54:53] (03CR) 10Nikerabbit: [V: 03+2] Localisation updates from https://translatewiki.net. [research/recommendation-api] - 10https://gerrit.wikimedia.org/r/988466 (owner: 10L10n-bot) [09:13:45] 10Machine-Learning-Team: Fix rec-api-ng relative paths handling - https://phabricator.wikimedia.org/T354601 (10kevinbazira) [09:18:31] Morning! [09:19:31] (03CR) 10Klausman: [C: 03+2] Add deprecation warning for ORES [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/988488 (https://phabricator.wikimedia.org/T349632) (owner: 10Klausman) [09:19:36] (03CR) 10Klausman: [V: 03+2 C: 03+2] Add deprecation warning for ORES [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/988488 (https://phabricator.wikimedia.org/T349632) (owner: 10Klausman) [09:23:01] 10Machine-Learning-Team, 10ORES, 10Patch-For-Review: Add deprecation warnings to ORES-related repositories on Github - https://phabricator.wikimedia.org/T349632 (10klausman) Amir set the mentioned repo to archived, so I think we're now done here. [09:43:22] Back! [09:43:27] o/ Tobias! [09:48:07] heyo Ilias. [10:17:35] morning! [10:20:29] hey aiko :) [10:34:35] (03PS2) 10AikoChou: revert-risk: add missing force_http arg in batch model [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/985347 (https://phabricator.wikimedia.org/T353622) [10:51:38] klausman: is this sth you can help with or shall we wait for other folks that deal with access requests ? https://phabricator.wikimedia.org/T354516 [10:53:07] I think the current setup of deployment does not have the required granularity. I will confer with the other k8s folks to see what the easiest way is that we can make that happen. I think namespace-level permissions etc should be easy from a technical side, but the details can be tricky. [10:53:19] I will update the ticket accordingly [11:00:41] ack, thanks! [11:01:16] (03PS2) 10AikoChou: revert-risk: refactor checking empty result for wikidata model [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/985349 [11:17:34] 10Machine-Learning-Team, 10SRE, 10SRE-Access-Requests: Requesting write access to ml-staging-codfw for ML team - https://phabricator.wikimedia.org/T354516 (10MatthewVernon) Hi. I'm the clinician on duty this week. I'm afraid I'm not quite clear what sort of access you are requesting here (ml-staging-codfw is... [11:20:59] 10Machine-Learning-Team: Fix rec-api-ng relative paths handling - https://phabricator.wikimedia.org/T354601 (10kevinbazira) I was able to reproduce this issue locally. 1.Query strings that start after a slash return an API response: ` $ time curl -s "127.0.0.1/api/?s=en&t=fr&n=3&article=Apple" [{"pageviews": 0,... [11:30:33] * klausman lunch [11:30:52] 10Machine-Learning-Team: Fix rec-api-ng relative paths handling - https://phabricator.wikimedia.org/T354601 (10kevinbazira) Following the second example above, I ran the same command with the "-L" flag to automate redirection and the API was able to return an API response: ` $ time curl -sL "127.0.0.1/api?s=en&t... [11:35:58] * isaranto lunch! [11:38:23] * aiko lunch 2 [11:52:13] 10Machine-Learning-Team: Fix rec-api-ng relative paths handling - https://phabricator.wikimedia.org/T354601 (10kevinbazira) The solution above works for people using `curl` or tools that are willing to implement an "-L" flag-like feature that automates the redirection. This may not produce a very good user exper... [11:57:05] (03PS1) 10Kevin Bazira: Disable strict slashes to support relative paths [research/recommendation-api] - 10https://gerrit.wikimedia.org/r/988245 (https://phabricator.wikimedia.org/T354601) [14:12:35] 10Machine-Learning-Team: Set SLO for the recommendation-api-ng service hosted on LiftWing - https://phabricator.wikimedia.org/T347262 (10klausman) Unfortunately, Istio does not export a le=7500 bucket, but only these (the base unit is milliseconds) ` 0.5 1 5 10 25 50 100 250 500s 1'000 (1s) 2'500 5'000 10'000 3... [14:18:53] Morning all [14:22:31] ehyo Chris [14:38:26] kevinbazira: say, do we have any sizable traffic on the LW rec-api-ng yet? [14:43:25] klausman: nope. users are yet to migrate to it. [14:46:12] Ah, that explains the SLO dashboard preview, then :) [14:48:55] 10Machine-Learning-Team: Fix rec-api-ng relative paths handling - https://phabricator.wikimedia.org/T354601 (10kevinbazira) We have deployed this solution in prod. Users can now use query strings on the rec-api-ng hosted on LiftWing and it will return an API response on both relative and absolute paths as shown... [14:49:48] btw, can confirm that queries without / are working via the API GW [14:50:24] e.g. https://api.wikimedia.org/service/lw/recommendation/v1/api?s=de&t=en&n=3&article=Paris [14:52:06] 10Machine-Learning-Team: Create external endpoint for recommendation-api-ng hosted on LiftWing - https://phabricator.wikimedia.org/T347263 (10kevinbazira) >>! In T347263#9392641, @klausman wrote: > Things that still need improvement: > > - the slash after `api` is mandatory, i.e. `https://api.wikimedia.org/serv... [14:52:15] yey o/ [14:52:30] thanks for the reviews klausman :) [14:52:53] thanks for the coding bits :) [15:19:12] 10Machine-Learning-Team, 10SRE, 10SRE-Access-Requests: Requesting write access to ml-staging-codfw for ML team - https://phabricator.wikimedia.org/T354516 (10calbon) a:03klausman [15:23:42] 10Machine-Learning-Team: Investigate inference optimization frameworks for Large Language Models (LLMs) - https://phabricator.wikimedia.org/T354257 (10calbon) a:03isarantopoulos [15:25:06] 10Machine-Learning-Team, 10SRE: Requesting write access to ml-staging-codfw for ML team - https://phabricator.wikimedia.org/T354516 (10klausman) I think we can remove the SRE-Access-Requests tag, since this likely can be entirely covered on the k8s permission level. [15:26:45] 10Machine-Learning-Team, 10artificial-intelligence: LLM that specializes in assisting Wikimedia/MediaWiki technical contributors - https://phabricator.wikimedia.org/T353974 (10calbon) a:03calbon [15:27:00] 10Machine-Learning-Team, 10artificial-intelligence: LLM that specializes in assisting Wikimedia/MediaWiki technical contributors - https://phabricator.wikimedia.org/T353974 (10calbon) [15:36:26] 10Machine-Learning-Team, 10Project-Admins: Archive #Test-Grounds Phabricator project? - https://phabricator.wikimedia.org/T353764 (10calbon) @Aklapper confirmed, you can archive it. [15:40:51] 10Machine-Learning-Team, 10Project-Admins: Archive #Test-Grounds Phabricator project? - https://phabricator.wikimedia.org/T353764 (10Aklapper) 05Open→03Resolved Done by following https://www.mediawiki.org/wiki/Phabricator/Project_management#Archiving_a_project [15:41:23] 10Machine-Learning-Team: Investigate how to improve model card integration with existing user flows - https://phabricator.wikimedia.org/T353025 (10calbon) a:03calbon [15:44:19] 10Machine-Learning-Team, 10SRE: Requesting write access to ml-staging-codfw for ML team - https://phabricator.wikimedia.org/T354516 (10isarantopoulos) [15:44:21] 10Machine-Learning-Team, 10Goal: Goal: Inference Optimization for Hugging face/Pytorch models - https://phabricator.wikimedia.org/T353337 (10isarantopoulos) [16:15:19] (03PS2) 10AikoChou: revertrisk-batch: async fetching data and batch prediction [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/985350 (https://phabricator.wikimedia.org/T352987) [16:29:46] kevinbazira: I'll run a script that slowly asks the rec api about random words and lang-pairs overnight, just to have some data for the SLO page. At most 3r/s. wdyt? [16:30:49] klausman: that's fine. please go ahead. I can't wait to see the results :) [16:44:44] heading out now, see ya tomorrow, everyone [16:45:34] o/ [16:47:28] bye Tobias! have a nice evening :) [16:53:22] night all! [17:21:11] (03PS2) 10AikoChou: Add Makefile for local-run [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/988666 (https://phabricator.wikimedia.org/T352689) [17:30:55] aiko: I'm heading out, I'll review --^ in the morning! [17:32:00] (03CR) 10AikoChou: Add Makefile for local-run (032 comments) [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/988666 (https://phabricator.wikimedia.org/T352689) (owner: 10AikoChou) [17:54:58] isaranto: thanks! :) [18:03:38] heading out as well o/ [21:06:18] (03PS2) 10Kosta Harlan: WIP revertrisk: Add thresholds for likely/very likely bad [extensions/ORES] - 10https://gerrit.wikimedia.org/r/988048 (https://phabricator.wikimedia.org/T348298) [21:26:36] 10Machine-Learning-Team, 10artificial-intelligence, 10Bad-Words-Detection-System, 10revscoring: Gather language assets for Occitan - https://phabricator.wikimedia.org/T354702 (10Lhanars) [21:27:05] 10Machine-Learning-Team, 10artificial-intelligence, 10Bad-Words-Detection-System, 10revscoring: Gather language assets for Occitan - https://phabricator.wikimedia.org/T354702 (10Lhanars) p:05Triage→03High