[08:34:20] <elukey>	 hello folks
[08:37:24] <isaranto>	 good morning!
[08:46:01] <wikibugs>	 10Machine-Learning-Team: Fix translatewiki-reverted and frwikisource-articlequality isvcs - https://phabricator.wikimedia.org/T324567 (10elukey) @achou there is one problem with the model rename to "itemquality":  ` root@deploy1002:/srv/deployment-charts/helmfile.d/ml-services/revscoring-articlequality# kubectl...
[08:56:20] <wikibugs>	 10Machine-Learning-Team: Fix translatewiki-reverted and frwikisource-articlequality isvcs - https://phabricator.wikimedia.org/T324567 (10elukey) ` elukey@stat1004:~$ sudo s3cmd -c /etc/s3cmd/cfg.d/ml-team.cfg mv s3://wmf-ml-models/articletopic/wikidatawiki/20220720074925/model.bin s3://wmf-ml-models/itemtopic/wi...
[08:57:46] <elukey>	 isaranto: o/ is https://phabricator.wikimedia.org/T323586 deployed to all clusters?
[08:59:15] <isaranto>	 elukey: yes on all prod ones. I haven't deployed it to staging yet- since it would increase the resources. I will issue a new patch to reduce the resources on staging.. I also started taking a look at the revscoring package
[09:00:36] <elukey>	 isaranto: ah ok I am rolling it out since I had to test Aiko's patch to remove some isvcs, no problem to allocate more resources. Feel free to change them later on!
[09:02:25] <isaranto>	 ack
[09:05:33] <wikibugs>	 10Machine-Learning-Team, 10Add-Link, 10Growth-Team, 10User-notice: Investigate `UnicodeEncodeError` thrown by Add-A-Link training pipeline for fywiki model - https://phabricator.wikimedia.org/T325521 (10kevinbazira)
[09:14:45] <wikibugs>	 10Machine-Learning-Team: Fix translatewiki-reverted and frwikisource-articlequality isvcs - https://phabricator.wikimedia.org/T324567 (10elukey) The change has been rolled out :)
[09:43:25] <elukey>	 revscoring tests passed, I am testing the revscoring docker image now (soooo nice to have only one!)
[09:59:00] <elukey>	 kserve 0.8.0 depends on numpy~=1.19.2
[09:59:05] <elukey>	 uff
[10:03:28] <elukey>	 of course it was fixed in 0.9 (we run 0.8)
[10:03:35] <elukey>	 so for the moment we cannot really upgrade revscoring
[10:05:13] <elukey>	 https://github.com/wikimedia/revscoring/pull/526 is basically ready, but we'll need to wait to deploy it on lift wing
[10:06:56] <wikibugs>	 10Machine-Learning-Team, 10artificial-intelligence, 10revscoring: Update revscoring dependencies to fix security reports - https://phabricator.wikimedia.org/T325366 (10elukey) Ran `pytests` on stat1004 and all tests passed, except `tests/languages/test_indonesian.py` (but this was already failing before).  B...
[10:13:37] <wikibugs>	 10Machine-Learning-Team: Upgrade ml clusters to kserve 0.9 - https://phabricator.wikimedia.org/T325528 (10elukey)
[10:18:04] <elukey>	 also opened https://github.com/wikimedia/drafttopic/pull/66 to bump drafttopic to 0.4
[10:18:18] <elukey>	 not super urgent but the git commit in revscoring's requirements.txt is not great
[10:25:05] <isaranto>	 aa nice! and I was wondering about that...
[11:41:35] <wikibugs>	 10Machine-Learning-Team: Fix translatewiki-reverted and frwikisource-articlequality isvcs - https://phabricator.wikimedia.org/T324567 (10achou) @elukey Changing model path is good, thanks for your help! :)
[11:41:44] * elukey lunch
[12:34:50] * isaranto lunch
[13:19:29] <isaranto>	 hey, I am trying to run kinit  but I get `kinit: Client 'isaranto@WIKIMEDIA' not found in Kerberos database while getting initial credentials`. any pointers?
[13:19:52] <isaranto>	 according to this ticket it should be ok (?) https://phabricator.wikimedia.org/T322350
[13:24:54] <wikibugs>	 10Lift-Wing, 10Machine-Learning-Team: Test batch prediction for revert-risk model - https://phabricator.wikimedia.org/T323023 (10achou) Update: * we fixed the issue regarding "missing" responses from MW API and deployed a new image (`2022-12-14-170742-publish`, corresponding to knowledge integrity **v0.1**) to...
[13:41:55] <moritzm>	 isaranto: you don't have a Kerberos user principal, this seems to have fallen through the cracks in T322350
[13:41:58] <moritzm>	 let me fix that
[13:42:50] <isaranto>	 thank u!
[13:43:24] <moritzm>	 I created your account with a temp password, you should have gotten a mail with instructions how to change it
[13:48:35] <isaranto>	 worked great, I changed the password already, thanks!
[13:57:26] <moritzm>	 great :-)
[14:21:07] <elukey>	 thanks moritzm!
[14:29:26] <isaacj>	 fyi elukey regarding T325349, diego was presenting at a conference last week but hopefully will see it this week
[14:30:00] <isaacj>	 https://phabricator.wikimedia.org/T325349
[14:42:56] <chrisalbon>	 Morning all!
[14:55:25] <isaranto>	 morning! o/ 
[14:59:43] <elukey>	 isaranto: thanks Muniza already helped a lot!
[14:59:48] <elukey>	 err isaacj --^
[14:59:50] <elukey>	 sorry :)
[15:00:22] <isaacj>	 oh awesome, I should have guessed :)
[15:34:11] <elukey>	 just pushed https://pypi.org/project/drafttopic/0.4.0/
[15:35:55] <wikibugs>	 (03PS1) 10Elukey: revscoring: update the drafttopic's version to 0.4.0 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/869232
[15:41:16] <isaranto>	 elukey: how is the pypi repository updated? I see it on pypi but I dont see any github actions or such on github so there is a missing link for me
[15:41:38] <wikibugs>	 (03CR) 10Ilias Sarantopoulos: [C: 03+1] revscoring: update the drafttopic's version to 0.4.0 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/869232 (owner: 10Elukey)
[15:43:37] <isaranto>	 or is it manual?
[15:44:38] <elukey>	 isaranto: yeah it is still manual, I had to learn how we do it from the previous team
[15:45:01] <elukey>	 so in revscoring and drafttopic's repo there is a script/deploy.sh 
[15:45:13] <isaranto>	 now it's clear, thanks!
[15:45:38] <elukey>	 I basically follow https://wikitech.wikimedia.org/wiki/ORES/Deployment#Update_revscoring_in_PyPI
[15:45:51] <elukey>	 the pass for Pypi is in pwstore (SREs can access it etc..)
[15:46:05] <elukey>	 not ideal I know :(
[15:46:15] <wikibugs>	 (03CR) 10Elukey: [C: 03+2] revscoring: update the drafttopic's version to 0.4.0 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/869232 (owner: 10Elukey)
[15:46:41] <wikibugs>	 (03CR) 10Elukey: [C: 03+2] "Tested locally with Docker, worked fine." [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/869232 (owner: 10Elukey)
[15:47:04] <wikibugs>	 (03Merged) 10jenkins-bot: revscoring: update the drafttopic's version to 0.4.0 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/869232 (owner: 10Elukey)
[15:47:46] <isaranto>	 we could automate that in the repo with github actions. is that something we would want to avoid for some reason , or shall I create a ticket for it?
[15:49:40] <isaacj>	 i don't know that it's all best practices but FYI I have a repo that automatically publishes to PyPi when I create a new release on Github and I've found it to work very nicely. feel free to borrow: https://github.com/geohci/edit-types/blob/main/.github/workflows/release.yml
[15:50:05] <isaacj>	 details: https://github.com/geohci/edit-types/blob/main/README.md#releases
[15:50:39] <isaranto>	 thanks isaac , this is what I had in mind, either on merge or on release
[15:51:43] <elukey>	 isaacj: interesting! Where do you keep PYPI_API_TOKEN ?
[15:52:28] <isaacj>	 you presumably can't access but just stored under github settings: https://github.com/geohci/edit-types/settings/secrets/actions
[15:52:39] <elukey>	 super didn't know it
[15:53:24] <isaacj>	 https://usercontent.irccloud-cdn.com/file/mmoPpIDY/Screen%20Shot%202022-12-19%20at%2010.52.47%20AM.png
[15:53:27] <elukey>	 isaranto: I think we can open a task, I never really did it since we were planning to not touch revscoring/etc.. a lot, but I think that we'll release new versions so worth to have some automation
[16:18:24] <wikibugs>	 10Machine-Learning-Team: Automate publishing python packages to PyPI - https://phabricator.wikimedia.org/T325561 (10isarantopoulos)
[16:20:59] <isaranto>	 I've used github actions quite extensively so we can do pretty much anything we want. as long as we have the credentials :D 
[16:22:42] <elukey>	 we do have them yes :)
[16:31:17] <wikibugs>	 10Machine-Learning-Team, 10Add-Link, 10Growth-Team: Investigate `UnicodeEncodeError` thrown by Add-A-Link training pipeline for fywiki model - https://phabricator.wikimedia.org/T325521 (10JJMC89)
[17:02:39] <elukey>	 going afk for a little bit to buy groceries
[18:10:19] <wikibugs>	 10Machine-Learning-Team: Automate publishing python packages to PyPI - https://phabricator.wikimedia.org/T325561 (10elukey) The revscoring and {article,draft}{quality,topic} repos all use a `about.py` file with the version hardcoded in it (and then sourced to Pypi). We could very well introduce tags and push to...
[18:11:54] <elukey>	 going afk for the evening folks, have a nice rest of the day :)
[19:29:42] <wikibugs>	 10Machine-Learning-Team, 10Bad-Words-Detection-System, 10revscoring: Add language support for Esperanto (eo) - https://phabricator.wikimedia.org/T325577 (10Robin_van_der_Vliet)
[19:33:19] <wikibugs>	 10Machine-Learning-Team, 10Bad-Words-Detection-System, 10revscoring: Add language support for Esperanto (eo) - https://phabricator.wikimedia.org/T325577 (10Robin_van_der_Vliet) I noticed Revision Scoring supports a lot of languages, but not Esperanto. I'd like to add support for Esperanto, so we can also che...