[08:23:31] hello folks [08:25:26] o/ [08:25:45] 10Machine-Learning-Team: Upgrade python from 3.7 to 3.9 in docker images - https://phabricator.wikimedia.org/T325657 (10elukey) I agree yes! I think that we could simply bump the base image to Bullseye, rebuild and test if everything runs fine. I did a quick test for the revscoring image and some errors popped u... [08:39:16] (03PS1) 10Elukey: WIP - Upgrade the revscoring model server to Python 3.9 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/870517 (https://phabricator.wikimedia.org/T325657) [08:40:32] (03CR) 10CI reject: [V: 04-1] WIP - Upgrade the revscoring model server to Python 3.9 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/870517 (https://phabricator.wikimedia.org/T325657) (owner: 10Elukey) [08:49:33] isaranto: o/ [08:49:40] 10Machine-Learning-Team, 10Patch-For-Review: Upgrade python from 3.7 to 3.9 in docker images - https://phabricator.wikimedia.org/T325657 (10elukey) After some tries I ended up in this situation: ` #24 167.0 The conflict is caused by: #24 167.0 The user requested pyenchant==3.2.2 #24 167.0 revscoring 2... [08:50:00] after playing a bit with --^ I think that the most benefits from the automatic github actions would be for the revscoring repo [08:50:06] isaranto: --^ [08:50:16] (forgot to tag you sorry) [08:54:22] I read it. we'll have to bump the versions of some packages (perhaps some others along with pyenchant ) as they don't exist for newer python versions. e.g. [08:55:06] nice work! [08:56:24] I wonder what if we relax all the requirements (or most of them) and install the latest package versions - as some of them are quite old.. [08:57:34] they are yes.. but I am wondering if we end up in having to deal with major versions changes like deprecations etc.. [08:58:02] I'd prefer to release more and do little steps.. (so we don't invest a ton of time into revscoring) [09:00:20] agreedo! [09:30:14] I was reading some interesting things about 3.11, the microsoft team led by Guido released a ton of improvements in the interpreter itself [09:30:28] some people say even that it is a 2x gain from 3.10 [09:32:04] * elukey little break [09:43:58] (03PS1) 10AikoChou: revertrisk: install torch==1.13.1+cpu [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/870520 (https://phabricator.wikimedia.org/T325349) [09:51:07] yeah they say new versions of python are blazing fast! [09:51:19] but as i understand they are not supported by us yet, right? [09:59:33] in theory no, we don't have it running on any debian release yet (I suppose bookworm will get 3.11 before becoming stable) [10:00:27] https://release.debian.org/ shows that the freeze period may start early next year [10:02:27] (03CR) 10Elukey: "Looks very good! One suggestion - I'd add a comment about why we are doing this in blubber.yaml, so that we'll remember in 6 months why we" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/870520 (https://phabricator.wikimedia.org/T325349) (owner: 10AikoChou) [10:09:23] isaranto: if you want I can add secrets to the revscoring repo [10:09:43] (if you need to test etc..) [10:09:50] (03CR) 10Ilias Sarantopoulos: [C: 03+1] "Nice work! instead of having a separate requirements file though we could add this dependency to the existing requirements file. We need t" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/870520 (https://phabricator.wikimedia.org/T325349) (owner: 10AikoChou) [10:10:47] please do, if you need help let me know [10:11:26] the script adds PYPI_USER and PYPI_PASS, are those ok? [10:12:39] (only the pass sorry) [10:13:54] added PYPI_PASS [10:14:07] (repository secrets) [10:16:34] there are two ways to authenticate to PYPI, either via user +pass or via an API token. we use the first one right? (with user `scoring-internal`) [10:17:00] if so, just PYPI_PASS will do and I can hardcode the user [10:21:13] exactly yes [10:37:49] there is not user account for scoring-internal in the test pypi index https://test.pypi.org/. It is a good approach to use that repository especially to try out CI/CD as it states `a separate instance of the Python Package Index that allows you to try distribution tools and processes without affecting the real index.` [10:38:34] we could do the following: create a scoring-internal user in the test pypi or just create another user we can user to try out stuff. wdyt? [10:41:17] +1 didn't know it [10:43:15] (03PS2) 10AikoChou: revertrisk: add torch==1.13.1+cpu [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/870520 (https://phabricator.wikimedia.org/T325349) [10:44:36] aiko: I hope my suggestion works, don't swear plz if it doesnt :D [10:46:25] isaranto: I tested and it worked like a charm!! :D [10:50:36] elukey: who maintains the scoring-internal user? shall I create a test_user or sth? [10:50:57] I could also just add a token from my account that we can use [10:53:35] isaranto: in theory it is us, we can change it.. I can share the password with you if you want [10:54:01] or I can add another test user if you want, not sure what the procedure is (never done it) [10:54:16] (03PS3) 10AikoChou: revertrisk: add torch==1.13.1+cpu [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/870520 (https://phabricator.wikimedia.org/T325349) [10:54:59] (03CR) 10Elukey: [C: 03+1] "Nice work folks!" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/870520 (https://phabricator.wikimedia.org/T325349) (owner: 10AikoChou) [10:56:44] (03CR) 10AikoChou: [C: 03+2] revertrisk: add torch==1.13.1+cpu (032 comments) [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/870520 (https://phabricator.wikimedia.org/T325349) (owner: 10AikoChou) [10:58:25] (03CR) 10CI reject: [V: 04-1] revertrisk: add torch==1.13.1+cpu [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/870520 (https://phabricator.wikimedia.org/T325349) (owner: 10AikoChou) [10:59:18] PR is ready https://github.com/wikimedia/revscoring/pull/528 [11:00:22] i also tested with the TEST PYPI index and it worked https://github.com/wikimedia/revscoring/actions/runs/3748645114/jobs/6366217345 [11:00:22] however for the scoring-internal we'll have to wait for it to be merged [11:03:10] isaranto: to understand - if we change __author__ in about.py, we'll trigger a publish right? [11:03:45] yes.. the action will fail though as the version already exists [11:04:12] alternatively I could add another action that creates release+tag in github and triggers this one [11:04:43] hmm but the question still remains, what is the trigger point.. [11:05:04] 10Machine-Learning-Team: Automate publishing python packages to PyPI - https://phabricator.wikimedia.org/T325561 (10isarantopoulos) Done it the first way `publish a package whenever we merge a new version of about.py to master` https://github.com/wikimedia/revscoring/pull/528 [11:07:00] isaranto: nono I think this one is good enough, it was just to understand :) [11:07:32] isaranto: shall I merge?? [11:08:01] feel free 🤞 [11:10:14] :facepalm [11:10:31] i left this line in `if: startsWith(github.ref, 'refs/tags')` which means it searches for a tag.. [11:11:05] ah snap totally missed as well :( [11:11:15] do you want to send another pull request? [11:13:02] https://github.com/wikimedia/revscoring/pull/529 [11:16:25] isaranto: mmm it failed to push to test pypi IIUC [11:19:36] yeah it says that it already exists (it was pushed with the previous merge). but I manually deleted it before we merged this one [11:19:52] perhaps it was still showed in the index. shall we try version 2.11.8? [11:25:31] https://github.com/wikimedia/revscoring/pull/530 [11:28:25] isaranto: can we try to relax the enchant dependency as well? Also if you could add a note in the changelog [11:28:32] basically a proper release [11:28:40] or better to test only? [11:29:23] maybe better test only, but if you could update the changelog it would be more clear [11:29:39] (like "test release for github actions, no relevant changes" [11:36:05] elukey: done [11:37:55] isaranto: done! You can try to merge (so you can see if it works) [11:39:14] * elukey lunch! [11:39:54] it worked ! https://pypi.org/project/revscoring/ [11:39:58] wooooowwwww [11:40:04] \o/ [11:40:04] things are never supposed to work with first try right? [11:40:12] really nice work :) [11:40:17] thanks for the quick review Luca [11:40:18] I'll test later on 2.11.9 :) [11:57:05] * isaranto afk lunch [12:17:10] https://integration.wikimedia.org/ci/job/inference-services-pipeline-revertrisk/66/console [12:17:36] hmm.. no idea what caused the failure [13:15:42] elukey: can u add the PYPI pass to the drafttopic repo as well? [13:24:33] 10Machine-Learning-Team: Automate publishing python packages to PyPI - https://phabricator.wikimedia.org/T325561 (10isarantopoulos) same for drafttopic repo https://github.com/wikimedia/drafttopic/pull/67 [13:47:13] aiko: I was able to build the revertrisk image locally , so dont really know what happened. I rerun the jenkins build and it succeeded https://integration.wikimedia.org/ci/job/inference-services-pipeline-revertrisk/67/ [13:50:57] successful build doesnt report back status on the patch so just try to rebase or issue an empty commit [13:54:49] (03PS4) 10AikoChou: revertrisk: add torch==1.13.1+cpu [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/870520 (https://phabricator.wikimedia.org/T325349) [13:56:12] looks like it's not working. can I manually give Verified +2? [13:56:30] oh it works! [14:09:12] folks if you want to re-trigger a CI run: add a comment with "recheck" in gerrit [14:09:24] and it should just work :) [14:10:31] isaranto: done! (re: drafttopic) [14:16:30] elukey: nice! that's useful :) [14:19:23] useful hint! [14:21:50] this is ready as well https://github.com/wikimedia/drafttopic/pull/67 [14:22:48] same as the revscoring one [14:23:28] isaranto: I forgot to add 4.0 in the changelog! /o\ I can you add a note for me? Lemme see what we should add [14:24:04] sure I think it is just release of version 0.4.0 [14:24:35] a no there were some old commits [14:25:26] yeah we can say something like "Rebuild to add compatibility for revscoring 2.11" [14:25:36] there are other things but they shouldn't matter a lot [14:26:25] done! [14:27:19] approved :) [14:27:41] any other repo that we should add this thingy or are we ok with these two for now? [14:27:47] we have other repos like drafttopic, not sure if it is worth to add gh actions to all [14:27:50] https://github.com/wikimedia/articlequality [14:28:00] https://github.com/wikimedia/draftquality [14:28:32] https://github.com/wikimedia/editquality [14:29:53] what shall we do with this ticket then? [14:29:53] A) add it in the above repos now [14:29:53] B) add it whenever and if we make a change to any of them [14:31:19] 10Machine-Learning-Team, 10Add-Link, 10Growth-Team: Investigate `UnicodeEncodeError` thrown by Add-A-Link training pipeline for fywiki model - https://phabricator.wikimedia.org/T325521 (10MGerlach) Just want to add that this error is not unique to fywiki but will likely appear when working with other wikis a... [14:35:08] isaranto: I think that we'll likely forget B), maybe better A)? [14:35:16] but with low pace, we can do it in 2023 [14:36:52] I can just submit them now it is the same thing [14:37:00] and we close the ticket :) [14:37:06] thanks for the patience then :) [14:37:20] <3 [14:38:06] u just will have to put the secret in the other repos as well . whenever u feel like it, just let me know [14:40:27] isaranto: done [14:41:18] wow! [14:41:20] thanks [14:54:03] done :) [14:54:03] https://github.com/wikimedia/draftquality/pull/44 [14:54:03] https://github.com/wikimedia/articlequality/pull/175 [14:54:04] https://github.com/wikimedia/editquality/pull/238 [14:54:48] 10Machine-Learning-Team: Automate publishing python packages to PyPI - https://phabricator.wikimedia.org/T325561 (10isarantopoulos) Same for other revscoring model repos https://github.com/wikimedia/draftquality/pull/44 https://github.com/wikimedia/articlequality/pull/175 https://github.com/wikimedia/editquality... [14:55:25] elukey: ready for approval [14:56:59] thanks! [14:58:18] all approved! [14:58:20] thanks a lot [14:59:37] thank uu [14:59:51] I had an extra cup of coffee accidentally today :P [15:01:22] 10Machine-Learning-Team: Automate publishing python packages to PyPI - https://phabricator.wikimedia.org/T325561 (10isarantopoulos) a:03isarantopoulos [15:58:56] 10Machine-Learning-Team: Automate publishing python packages to PyPI - https://phabricator.wikimedia.org/T325561 (10elukey) Updated the pwstore repo (for SREs) where we store the Pypi password with a note (to remember changing the pass also in the aforementioned repos). [16:06:21] 10Machine-Learning-Team, 10Research, 10Patch-For-Review: Update torch's settings in the Knowledge Integrity repo - https://phabricator.wikimedia.org/T325349 (10achou) a:03achou [16:07:00] 10Machine-Learning-Team: Enrich revertrisk image tag with model's package version - https://phabricator.wikimedia.org/T325295 (10achou) a:05AikoChou→03achou [16:30:47] 10Machine-Learning-Team: Add documentation about LiftWing to the API Portal - https://phabricator.wikimedia.org/T325759 (10elukey) [16:35:43] isaranto: so upgrading revscoring to run it on debian bullseye is not... straightforward :D [16:35:49] I'll add some comments to the task [16:36:40] ack [16:42:14] 10Machine-Learning-Team, 10Patch-For-Review: Upgrade python from 3.7 to 3.9 in docker images - https://phabricator.wikimedia.org/T325657 (10elukey) I tried to: * Run a bullseye docker image * Checkout the revscoring repo (better: mount it over from my local checkout) * Install all the packages listed in blubb... [16:47:35] 10Machine-Learning-Team: Review ORES traffic to better understand Lift Wing's requirements - https://phabricator.wikimedia.org/T325763 (10elukey) [16:47:38] opened --^ [16:47:45] this is what we discussed during the meeting [16:48:30] thanks Luca! [16:49:49] I tried to remove all the internal traffic, and for the past 15 days I see few requests from [16:49:52] SeroBOT - an ORES counter vandalism tool [16:49:58] SWViewer/1.4+; swviewer@tools.wmflabs.org; SSE-proxy, TheTalk; NodeJS: request [16:50:53] python-requests/2.4.3 CPython/3.7.15 Linux/REDACTED [16:51:06] that is a little depressing, maybe something is missing [16:55:17] chrisalbon: I created two tasks after the team's chat: [16:55:28] 1) https://phabricator.wikimedia.org/T325763 [16:55:44] 2) https://phabricator.wikimedia.org/T325759 [16:55:47] anything missing? [16:55:55] (also added them to the gdoc) [16:56:25] (will read later :) [16:56:31] going afk folks! Havea nice rest of the day [17:03:42] going afk as well! [17:03:58] elukey: I added this PR if you want to check revscoring deps https://github.com/wikimedia/revscoring/pull/527 [17:05:41] we can sync tomorrow. It installs all the requirements and runs tests. I had to change some tests as the language dictionaries seem to have changed. or we could use the buster image which would result in exactly the same build as we have in inference-services