[04:14:55] (03PS1) 10Ilias Sarantopoulos: remove precision/recall from Special page [extensions/ORES] - 10https://gerrit.wikimedia.org/r/955417 (https://phabricator.wikimedia.org/T345407) [04:20:44] 10Machine-Learning-Team, 10MediaWiki-extensions-ORES, 10Patch-For-Review: Fix ORES Special page - https://phabricator.wikimedia.org/T345407 (10isarantopoulos) Since we have removed the ORES requests and we keep the threshold interval in the config, precision and recall don't exist anywhere. My suggestion for... [07:51:44] 10Machine-Learning-Team, 10Foundational Technology Requests: Content Translation Recommendations API - https://phabricator.wikimedia.org/T293648 (10santhosh) > We are already working on deploying this flask app as a service on the Lift Wing @isarantopoulos Could you please update the status on this work? [08:29:05] o/ [08:29:11] isaranto: kalimera! [08:29:18] I am reviewing code reviews, and I noticed https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/940945 [08:29:23] do we have to deploy it? [08:30:49] Kalimera! [08:31:15] W8 let me check. If I'm not mistaken these are the old ones which we want to remove [08:31:46] yeah I don't recall 100% [08:31:54] kevinbazira: o/ [08:31:57] around for a chat? [08:42:01] (added comments to the code review) [08:42:51] isaranto: one thing that I noticed is that the recommendation-api-ng chart is almost identical to the fastapi one [08:43:06] what varies is the command etc.. [08:43:34] so I am wondering if we shouldn't just create a more generic chart, called python-webapp or similar [08:43:44] to avoid duplicating efforts [08:43:47] am I crazy? [08:44:59] I agree.. [08:45:44] if we dont thing they would diverge in any way we should do it. Thinking about it they are both python web applications so I don't think that one would need something different than the other [08:46:02] perhaps some config map now or in the future but that could be done within the same chart [08:46:53] exactly yes [08:48:03] basically cp -r of fast-api to python-webapp (or similar), deploy the new chart and configure ores-legacy to use it [08:48:08] (03CR) 10Ilias Sarantopoulos: "I added a comment on the task with a screenshot of how it looks." [extensions/ORES] - 10https://gerrit.wikimedia.org/r/955417 (https://phabricator.wikimedia.org/T345407) (owner: 10Ilias Sarantopoulos) [08:48:19] then remove fast-api, and use the same chart for recommendation-api [08:50:01] (03CR) 10Ladsgroup: [C: 03+1] "I'm inclined to merge it but just giving a bit of time if anyone would object to it." [extensions/ORES] - 10https://gerrit.wikimedia.org/r/955417 (https://phabricator.wikimedia.org/T345407) (owner: 10Ilias Sarantopoulos) [08:51:36] 10Machine-Learning-Team, 10MediaWiki-extensions-ORES, 10Patch-For-Review: Fix ORES Special page - https://phabricator.wikimedia.org/T345407 (10Ladsgroup) For precision and recall, I think we should find a way to show it from an outside service. Doesn't LW provide that somehow? [08:54:07] the only different thing is that in recommendation-api we'll need to allow connetions to thanos-swift [08:54:14] but we can configure them in helfile.d [08:54:58] 10Machine-Learning-Team, 10MediaWiki-extensions-ORES, 10Patch-For-Review: Fix ORES Special page - https://phabricator.wikimedia.org/T345407 (10isarantopoulos) >>! In T345407#9148568, @Ladsgroup wrote: > For precision and recall, I think we should find a way to show it from an outside service. Doesn't LW prov... [08:58:19] * isaranto commuting to coworking. back online in 30' [08:59:04] isaranto, kevinbazira - created https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/955584/ [09:04:00] (03CR) 10AikoChou: [C: 03+1] "+1 print the thresholds instead" [extensions/ORES] - 10https://gerrit.wikimedia.org/r/955417 (https://phabricator.wikimedia.org/T345407) (owner: 10Ilias Sarantopoulos) [09:04:23] 10Machine-Learning-Team, 10MediaWiki-extensions-ORES, 10Patch-For-Review: Fix ORES Special page - https://phabricator.wikimedia.org/T345407 (10Ladsgroup) Yeah makes sense. We could link to the model card or something like that in the special page. Anyway, off-topic :D [09:30:32] commuting aborted due to rain :( back home now [09:47:05] ouch [09:47:09] so heavy? [09:50:33] kevinbazira: I'll wait for serviceops' approval, after that we can probably add the helmfile.d bits for recommendation-api and see how it goes :) [09:50:45] 10Machine-Learning-Team, 10MinT, 10SRE, 10serviceops, and 2 others: New Service Deployment Request: NNLB-200 for machine translation - https://phabricator.wikimedia.org/T329971 (10Pginer-WMF) 05Openā†’03Resolved a:03Pginer-WMF Since MinT [was launched](https://diff.wikimedia.org/2023/06/13/mint-support... [09:51:00] great. thanks elukey! [10:02:45] * aiko lunch [10:15:55] elukey: regarding https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/940945: [10:15:55] The change we want to introduce is that not all wiki ids contain the `wiki` suffix. This means that we want to expose eswikibooks-damaging:predict instead of eswikibookswiki-damaging-predict. To do the switch we want zero traffic but SWViewer is using it. [10:18:11] perhaps we can do the following: add swikibooks-damaging:predict to the API gateway, file a patch for SWViewer and then remove the old deployment from API gateway and inf service [10:18:26] at the moment we have 2 deployments for eswikibooks and eswikiquote for this reason [10:21:02] isaranto: makes sense yes! [10:39:34] (03CR) 10Ladsgroup: [C: 03+2] remove precision/recall from Special page [extensions/ORES] - 10https://gerrit.wikimedia.org/r/955417 (https://phabricator.wikimedia.org/T345407) (owner: 10Ilias Sarantopoulos) [10:47:17] (03Merged) 10jenkins-bot: remove precision/recall from Special page [extensions/ORES] - 10https://gerrit.wikimedia.org/r/955417 (https://phabricator.wikimedia.org/T345407) (owner: 10Ilias Sarantopoulos) [10:53:51] * elukey lunch! [10:54:06] * isaranto same [11:48:17] 10Machine-Learning-Team, 10Add-Link, 10Growth-Team (Current Sprint), 10User-notice: Deploy "add a link" to 13th round of wikis - https://phabricator.wikimedia.org/T308138 (10Sgs) [11:49:43] 10Machine-Learning-Team, 10Add-Link, 10Growth-Team (Current Sprint), 10Malayalam-Sites, 10User-notice: Deploy "add a link" to 12th round of wikis - https://phabricator.wikimedia.org/T308137 (10Sgs) a:05Trizek-WMFā†’03Sgs [11:54:54] elukey: I'm having a bit of a difficulty understanding what I need to change in order to expose eswikibooks and eswikiquote instead of eswikibookswiki and eswikiquotewiki [12:42:33] isaranto: back! Let's check [12:49:34] actually, do you mind if I fix the python-webapp's code change first? [12:52:49] go ahead! the other thing is not urgent at all [12:53:51] I'll also found what needs to change on SWViewer side so I'll send a PR once we have it [12:58:11] <3 [13:14:39] isaranto: where do we have the wikibooks/quote pods? [13:14:50] if you have it handy, otherwise I'll check on k8s [13:14:53] 10Machine-Learning-Team, 10MediaWiki-extensions-ORES, 10MW-1.41-notes (1.41.0-wmf.26; 2023-09-12): Fix ORES Special page - https://phabricator.wikimedia.org/T345407 (10isarantopoulos) this works now -> https://en.wikipedia.beta.wmflabs.org/wiki/Special:ORESModels [13:15:09] revscoring-damaging namespace [13:15:21] revscoring-editquality-damaging [13:16:56] elukey: sry it is in goodfaith [13:17:11] ah! [13:17:13] ok [13:17:55] w8 it should be both, but I cant fine one of the deployments in dep-charts [13:19:11] isaranto: https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/955728/ [13:19:14] this should work in theory [13:19:40] sorry I missed one thing [13:19:52] fixed [13:20:05] aa nice [13:21:46] please disregard my comments above about deployments. I hadn't pulled the latest code šŸ¤¦ [13:25:56] I'll wait for Hugh or Kamila to +1 and I'll deploy [13:31:47] ack [13:34:39] btw we're not going to have correct user-agents until next Mediawiki train deployment on tuesday [13:43:16] that's super fine [13:43:33] so we move enwiki and wikidata on monday/ture? [13:43:35] *tue? [13:56:17] it is a yes from my side. Amir1: wdyt? about enabling LW for enwiki and wikidata on monday/tuesday [13:59:20] Sounds good [14:12:02] isaranto: I am deploying the api-gateway change rn [14:12:15] ack, thanks! [14:15:55] done! [14:21:47] 10Machine-Learning-Team: Remove traffic from old eswikibooks and eswikiquote deployments - https://phabricator.wikimedia.org/T345850 (10isarantopoulos) [14:22:09] I added a task --^ in order to track this work in case it doesnt finish today/tomorrow [14:23:46] 10Machine-Learning-Team, 10Patch-For-Review: use wikiID in inference name on LW for revscoring models - https://phabricator.wikimedia.org/T342266 (10isarantopoulos) 05Resolvedā†’03Open [[ https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/955728 | This is the patch ]] about the API Gateway changes [14:24:50] 10Machine-Learning-Team: Remove traffic from old eswikibooks and eswikiquote deployments - https://phabricator.wikimedia.org/T345850 (10isarantopoulos) [[ https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/955728 | This is the patch ]] about the API Gateway changes [14:25:11] accidentally changed wrong task above plz disregard [14:26:22] elukey: do I need to do sth in order to access the new endpoints? [14:26:39] I want to make this request `curl https://api.wikimedia.org/service/lw/inference/v1/models/eswikibooks-damaging:predict -X POST -d '{"rev_id": 1234}'` [14:29:45] in theory no [14:31:19] ah of course the regex is wrong [14:31:41] no wait, mmmm [14:32:04] does it work via discovery? [14:32:27] ah yes isaranto, goodfaith not damaging [14:32:31] just realized :D [14:32:50] yeah it works :D [14:34:27] šŸ¤¦ā€ā™‚ļø sry elukey: I never noticed that the patch was only for goodfaith [14:34:37] I'm sending a new one [14:34:45] but IIUC it was only good faith no? [14:35:46] nope also damaging, just checked [14:35:57] it is in both but I confused you while trying to find out (was long time ago couldn't remember) [14:37:11] I should've checked as well [14:37:49] sending the patch now! [14:46:59] 10Machine-Learning-Team, 10Patch-For-Review: Remove traffic from old eswikibooks and eswikiquote deployments - https://phabricator.wikimedia.org/T345850 (10isarantopoulos) Added a [[ https://github.com/SWViewer/swviewer-service/pull/2 | Pull Request ]] to adopt the above changes for SWViewer [14:51:40] 10Machine-Learning-Team, 10Patch-For-Review: use wikiID in inference name on LW for revscoring models - https://phabricator.wikimedia.org/T342266 (10isarantopoulos) 05Openā†’03Resolved [14:54:06] elukey: let me check if enwiktionary is affected as well before we merge the new patch [14:58:45] ofc it was! i double checked the pods and all of them had requests. now the patch is ready [15:00:57] ok perfect, let's wait for Kamila or Hugh [15:26:11] I am trying to utilizing ChatGPT to categorize and extract unstructured metadata from filenames and description texts of files that my bot uploads to Commons. It performs effectively, although it may not be accurate all the time. Given that I am exclusively processing files uploaded by my own bot, I believe it is acceptable and better than nothing. Actually, I used to spend much time to compose complicated RegExps that are also [15:26:11] inaccurate sometimes. However, I am not inclined to personally cover the API fee. Would it be possible for the foundation to offer free access to ChatGPT or other LLM APIs, just like Toolforge or other cloud services? [15:30:06] Hoi: Hi! For ChatGPT I'd say definitely no, since it is a paid non-open source endpoint, and giving free access to the community would sadly burn very fast our limited budget. We are experimenting with LLMs self-hosted on our platform (Lift Wing) but we are still in early stages (see https://phabricator.wikimedia.org/T334583 and https://phabricator.wikimedia.org/T341695 for example) [15:31:00] and we'll surely not be able to offer something as complex as ChatGPT on our infra [15:31:19] (maybe some LLM more targeted to specific use cases, but it is still all to be decided) [15:31:47] you can open task in phabricator explaining your use case if you have time, adding the Machine-Learning-team tag will land in our queue for review :) [15:33:40] Thanks the info. [15:35:31] https://www.irccloud.com/pastebin/ODmzaGAL/ [15:35:36] > not be able to offer something as complex as ChatGPT on our infra [15:35:36] Does it mean there will only be smaller models with relatively limited size of params? [15:36:23] (Plz ignore the pastebin link. I clicked a button by mistake.) [15:36:50] yeah I'd say so, it is still all in flux (we didn't get the time to figure out what models to target) [15:37:07] for example, from our tests, not using GPUs makes response time go up to minutes [15:37:39] and with our 16G GPUs, a model like Falcon 7B doesn't work out of the box (due to not fitting in VRAM etc.) [15:38:37] we are thinking to get 32GB GPUs, but we won't surely be able to host gigantic chat bots [15:38:54] (GPUs with more ram cost a fortune) [15:39:00] elukey: I supposed we have much more hardware resources, while it appears not to be so... [15:41:16] Hoi: they cost a lot, for example a Nvidia A100 with 80G of VRAM is around 20/25k dollars IIRC [15:41:55] our goal will probably be to support ad-hoc use cases, rather than a complete ChatGPT-like solution [15:49:05] going afk folks! [15:49:09] have a nice rest of the day [15:55:59] Good afternoon Luca! Thanks for the above response. Totally agree. [15:56:05] Going afk as well! [15:56:47] elukey: Nice day. Iā€™m off to dreamland..