[08:31:45] 10Lift-Wing, 10SRE-swift-storage, 10Machine-Learning-Team (Active Tasks): Create Swift account for readonly access to ML models - https://phabricator.wikimedia.org/T311628 (10elukey) Filippo applied the following rule and everything now works: ` swift post wmf-ml-models+segments -r 'mlserve:ro' ` I had pre... [08:34:33] 10Lift-Wing, 10SRE-swift-storage, 10Machine-Learning-Team (Active Tasks): Create Swift account for readonly access to ML models - https://phabricator.wikimedia.org/T311628 (10elukey) Tried to upload a model with the new read only account and I got access denied (good): ` elukey@stat1004:~$ sudo s3cmd -c tes... [08:46:27] article quality pods on ml-staging-codfw in running state :) [08:47:31] 10Lift-Wing, 10SRE-swift-storage, 10Machine-Learning-Team (Active Tasks): Create Swift account for readonly access to ML models - https://phabricator.wikimedia.org/T311628 (10elukey) 05Open→03Resolved a:03elukey All working! Added the new account to the ML staging cluster, and it worked nicely. We'll m... [08:48:27] 10Lift-Wing, 10Epic, 10Machine-Learning-Team (Active Tasks), 10Patch-For-Review: Create the ml-serve-staging k8s cluster - https://phabricator.wikimedia.org/T302195 (10elukey) articlequality pods up and running! The swift credentials are working as expected. Next steps: add some editquality etc.. pods as... [08:49:32] nice \o/ [08:54:13] 10Lift-Wing, 10Epic, 10Machine-Learning-Team (Active Tasks), 10Patch-For-Review: Create the ml-serve-staging k8s cluster - https://phabricator.wikimedia.org/T302195 (10elukey) ` elukey@ml-serve-ctrl1001:~$ curl "https://inference-staging.svc.codfw.wmnet:30443/v1/models/enwiki-articlequality:predict" -X POS... [08:54:55] going afk, ttl! [10:39:55] elukey: nice work on Staging. I'll make some changes for editquality et al [12:10:49] <- lunch [12:47:42] 10Lift-Wing, 10Epic, 10Machine-Learning-Team (Active Tasks), 10Patch-For-Review: Send score to eventgate when requested - https://phabricator.wikimedia.org/T301878 (10Ottomata) One thing to note: it is a bit difficult to change the schema a stream uses after the fact. So maybe while doing the test and dev... [12:54:37] 10Lift-Wing, 10Machine-Learning-Team (Active Tasks), 10Patch-For-Review: Test async preprocess on kserve - https://phabricator.wikimedia.org/T309623 (10achou) I investigated the [[ https://pypi.org/project/async-mediawiki/ | async-mediawiki (aiowiki) ]] library. I feel it is not a suitable replacement for th... [13:38:02] 10Lift-Wing, 10Epic, 10Machine-Learning-Team (Active Tasks), 10Patch-For-Review: Create the ml-serve-staging k8s cluster - https://phabricator.wikimedia.org/T302195 (10klausman) Now also running draftquality for enwiki: ` ml-serve-ctrl1001 ~ $ curl "https://inference-staging.svc.codfw.wmnet:30443/v1/mod... [14:32:35] 10Lift-Wing, 10Epic, 10Machine-Learning-Team (Active Tasks), 10Patch-For-Review: Set up the ml-cache clusters - https://phabricator.wikimedia.org/T302232 (10calbon) a:05elukey→03klausman [14:53:50] 10Lift-Wing, 10Epic, 10Machine-Learning-Team (Active Tasks), 10Patch-For-Review: Send score to eventgate when requested - https://phabricator.wikimedia.org/T301878 (10elukey) Ah ok I was assuming that the revision score streams would have changed their name anyway after T308017, but it may not be the case... [14:55:40] klausman: I dumped all ideas in https://phabricator.wikimedia.org/T311982 for kserve 0.8, what changes have you in mind that could be disrupive? [14:55:49] (the plan would be to test it on staging first) [14:56:59] 10Lift-Wing, 10Epic, 10Machine-Learning-Team (Active Tasks), 10Patch-For-Review: Send score to eventgate when requested - https://phabricator.wikimedia.org/T301878 (10Ottomata) > Ah ok I was assuming that the revision score streams would have changed their name anyway after T308017 Yeah they might, so may... [14:59:36] 10Lift-Wing, 10Epic, 10Machine-Learning-Team (Active Tasks), 10Patch-For-Review: Set up the ml-cache clusters - https://phabricator.wikimedia.org/T302232 (10elukey) >>! In T302232#8042485, @Eevans wrote: > Just to reiterate for posterity sake. The Platform team's Generated Data Platform (aka AQS) isn't th... [15:18:37] elukey: it was more of a "there _may be_ disruptive things I am not aware of" comment [21:14:19] (03PS3) 10Umherirrender: Convert to abstract schema [extensions/ORES] - 10https://gerrit.wikimedia.org/r/757989 (https://phabricator.wikimedia.org/T268566)