[05:17:00] 10Machine-Learning-Team, 10Add-Link, 10Growth-Team, 10User-notice: Deploy "add a link" to 9th round of wikis - https://phabricator.wikimedia.org/T308134 (10kevinbazira) [05:30:50] 10Machine-Learning-Team, 10Add-Link, 10Growth-Team: Fix Armenian sentence tokenization bug in the link recommendation algorithm - https://phabricator.wikimedia.org/T327371 (10kevinbazira) [05:36:13] 10Machine-Learning-Team, 10Add-Link, 10Growth-Team: Fix Armenian sentence tokenization bug in the link recommendation algorithm - https://phabricator.wikimedia.org/T327371 (10kevinbazira) [05:38:41] 10Machine-Learning-Team, 10Add-Link, 10Growth-Team, 10User-notice: Deploy "add a link" to 9th round of wikis - https://phabricator.wikimedia.org/T308134 (10kevinbazira) 20/21 models were trained successfully in the 9th round of wikis. The Armenian Wikipedia (hywiki) pipeline did not complete successfully... [10:17:26] I am reading https://phabricator.wikimedia.org/diffusion/MSCP/browse/master/config.example.wikimedia.yaml$545 [10:17:36] and changeprop looks not super difficult afaics [10:17:51] we may not even need to code any extra module in .js [10:19:05] a simple implementation for revision-score-goodfaith could be to have a regex that matches on the wiki [10:19:13] interesting [10:28:11] sounds nice! [10:54:44] I reopened this PR - it is ready for review https://github.com/wikimedia/revscoring/pull/531 [10:55:03] (03CR) 10Ilias Sarantopoulos: "recheck" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/870517 (https://phabricator.wikimedia.org/T325657) (owner: 10Elukey) [11:04:46] (03PS17) 10Ilias Sarantopoulos: Upgrade the revscoring model server to Python 3.9 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/870517 (https://phabricator.wikimedia.org/T325657) (owner: 10Elukey) [11:05:55] (03CR) 10CI reject: [V: 04-1] Upgrade the revscoring model server to Python 3.9 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/870517 (https://phabricator.wikimedia.org/T325657) (owner: 10Elukey) [11:08:53] isaranto: nice! Going to review it after lunch [11:09:57] whenever u feel like it :) . I am now giving a fight with Jenkins in inference-services. he changed his mind about some stuff 😂 [11:10:35] *she changed [11:12:29] (03PS18) 10Ilias Sarantopoulos: Upgrade the revscoring model server to Python 3.9 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/870517 (https://phabricator.wikimedia.org/T325657) (owner: 10Elukey) [11:13:15] (03CR) 10CI reject: [V: 04-1] Upgrade the revscoring model server to Python 3.9 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/870517 (https://phabricator.wikimedia.org/T325657) (owner: 10Elukey) [11:15:37] (03PS19) 10Ilias Sarantopoulos: Upgrade the revscoring model server to Python 3.9 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/870517 (https://phabricator.wikimedia.org/T325657) (owner: 10Elukey) [11:16:42] (03CR) 10CI reject: [V: 04-1] Upgrade the revscoring model server to Python 3.9 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/870517 (https://phabricator.wikimedia.org/T325657) (owner: 10Elukey) [11:18:04] this is very interesting: https://toolhub.wikimedia.org/search?q=vandalism&ordering=-score&page=1&page_size=12 [11:18:36] cc: chrisalbon: --^ [11:23:19] (03PS1) 10Ilias Sarantopoulos: pre-commit: make required changes [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/881574 (https://phabricator.wikimedia.org/T325198) [11:39:15] isaranto: While I approve of the changes in 881574 in principle, I caution against wide-area cleanups like that, because they tend to create "watershed commits", i.e. points in history where tracking back changes/finding out the history of a piece of code becomes very hard. Ideally, stuff like this (execept e.g. EOL at EOF) shoudl go into changes that already touch the relevant (or [11:39:17] logically-adjacent) code. Its fine for this one, just a remark. [11:39:21] (03CR) 10Klausman: [C: 03+1] pre-commit: make required changes [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/881574 (https://phabricator.wikimedia.org/T325198) (owner: 10Ilias Sarantopoulos) [11:41:15] totally agree! I am against these kind of massive refactorings but I made this patch in order to include the checks in our CI. I couldn't think of any other way at the moment [12:00:53] (03CR) 10Elukey: [C: 03+1] pre-commit: make required changes [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/881574 (https://phabricator.wikimedia.org/T325198) (owner: 10Ilias Sarantopoulos) [12:33:50] totally ok for me to proceed isaranto, great work [12:34:17] we are finally doing all the nice tidy/cleanup steps in our code that will make our life easier [12:34:20] thanks! [12:35:34] 🤗 [12:36:17] I filed https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/881594 as poc for changeprop [12:36:52] nice! will review after lunch [12:37:08] not super ready yet, but it gives an idea [12:39:01] * elukey lunch! [14:48:28] elukey: I checked the changeprop patch. great work! [14:49:33] isaranto: thanks! I need to refine a couple of things and ask platform folks to review as well, but it looks really really easy [14:49:56] I have one question regarding the part after the call to lifwing [14:49:56] query: [14:49:56] postevent: true [14:49:56] body: '{ revision_create_event: {{ `{{globals.message}}` }} }' [14:49:56] Does this send to eventgate? [14:52:10] ah nono I need to keep only the "body" bit, the query part was a leftover from the ores_cache entry [14:52:23] so IIUC "body" is the json payload of the post [14:53:07] ack [14:53:16] in our case this should be enough to trigger lift wing to send an event to eventgate [14:53:35] now there are multiple questions related to schemas and how many streams we want to support [14:59:54] elukey: fwiw, the page change schema model we've been working on is mostly finalized. so if you want to base your stuff on that you can. We have as a goal to fully finalize it by the end of the quarter [15:03:36] ottomata: o/ o/ I was thinking to base the revscoring-based streams on revision-create, and the newer ones (like reverted risk etc..) on page change. What do you think? [15:03:47] (I'll also ask to research what they think about) [15:12:05] sure i'm not familiar wiith the differences. i guess revscoring base ones are == ores and would use a similiar data model? [15:12:08] addinig the scores map field? [15:12:40] or, actually,i guess is the same but with one model score per eventstream? [15:12:43] instead of multiple? [15:12:59] elukey: FYI one day we are hoping to deprecate revision create with page change [15:13:19] so revision-create, page-create, page-delete, page-undelete (maybe a few more) are all superseded by page-change [15:53:47] ottomata: yeah I hope to deprecate the revision-score ones too as well, this is why I'd like to keep the ones that are currently based on revision-create as they are, and use page change for the new models [15:53:56] so deprecate everything in one go :D [15:57:38] got it. sounds good [15:57:47] elukey: why not just keep revision-score as is then? why make new streams? [15:58:31] ottomata: second problem - we need to deprecate ORES :D [15:58:57] and we don't have (in liftwing) a single api that scores the same rev-id for multiple models [15:59:29] (it is a weird way to pack things anyway so we decided to split the streams into more granular ones) [15:59:39] revision-score will hopefully be deprecated with ORES [16:06:17] k, sounds good. ty! [16:06:23] <3 [16:06:28] thanks for the brainbounce (as always) [16:09:28] thanks otto! [16:10:19] hey team, any way I can get the Dockerfile that blubber creates? I see the commands on build but I want to extract it [16:10:40] I'm having some issues with tox and python 3.9 in our test images and trying to investigate it [16:12:16] you can send a post to blubberoid.wikimedia.org [16:13:31] I have this alias in my bash config [16:13:42] blubber() { [16:13:42] if [ $# -lt 2 ]; then [16:13:42] echo 'Usage: blubber config.yaml variant' [16:13:42] return 1 [16:13:42] fi [16:13:44] curl -s -H 'content-type: application/yaml' --data-binary @"$1" https://blubberoid.wikimedia.org/v1/"$2" [16:13:47] } [16:13:56] and then I use it like `blubber .pipeline/revscoring/blubber.yaml production` [16:14:12] (from the inference-services repo) [16:15:47] aaa yes. thank u :) [16:18:38] saved me [16:18:50] * elukey sends the bill to Ilias [16:25:05] (03PS20) 10Ilias Sarantopoulos: Upgrade the revscoring model server to Python 3.9 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/870517 (https://phabricator.wikimedia.org/T325657) (owner: 10Elukey) [16:25:07] One of my gaming group's long-time members would often give really good advice on game matters, and then end with "That will be €4.99" [16:25:22] I see Luca has had the same idea. [16:25:58] Ilias didn't really answer so I think my proposal went nowhere :D [16:26:22] (03CR) 10CI reject: [V: 04-1] Upgrade the revscoring model server to Python 3.9 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/870517 (https://phabricator.wikimedia.org/T325657) (owner: 10Elukey) [16:27:08] He thanked you, which implies acceptance of contract :-P [16:27:24] I'll pay def [16:27:58] piece of mind is expensive :) [16:28:25] I am sure an appropriate beverage will be appreciated next time we meet. [16:30:43] (03PS21) 10Ilias Sarantopoulos: Upgrade the revscoring model server to Python 3.9 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/870517 (https://phabricator.wikimedia.org/T325657) (owner: 10Elukey) [16:31:27] ok so https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/881594 and next should in theory be a good baseline for changeprop [16:31:34] lemme know your thoughts :) [16:31:41] (03CR) 10CI reject: [V: 04-1] Upgrade the revscoring model server to Python 3.9 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/870517 (https://phabricator.wikimedia.org/T325657) (owner: 10Elukey) [16:34:58] (03PS22) 10Ilias Sarantopoulos: Upgrade the revscoring model server to Python 3.9 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/870517 (https://phabricator.wikimedia.org/T325657) (owner: 10Elukey) [16:38:52] elukey: my mind is stuck on the ports it seems :) [16:40:29] isaranto: I removed the port from the chart in theory [16:41:06] now there is only the uri field, much better (I didn't notice before your first comment the port hardcoded) [16:41:21] aa my bad [16:41:37] I checked the previous version. all good then nevermind [16:43:26] elukey: had one somewhat general question about the changeprop change [16:43:43] yeah I saw it, tried to answer, it is a good point [16:43:49] not sure what's best [16:43:57] (we have the same problem with ORES as well) [16:46:21] (03PS23) 10Ilias Sarantopoulos: Upgrade the revscoring model server to Python 3.9 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/870517 (https://phabricator.wikimedia.org/T325657) (owner: 10Elukey) [16:58:52] (03CR) 10Ilias Sarantopoulos: [C: 03+2] pre-commit: make required changes [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/881574 (https://phabricator.wikimedia.org/T325198) (owner: 10Ilias Sarantopoulos) [16:59:21] I merged all the formatting related changes in the patch above [17:04:33] also, I figured out the blubber issues I had. pre-commit run via tox tries to install packages but it doesn't find anything as it doesnt have access to the pypi index on runtime (blubber thingy). So for now I manually added these requirements in the requirements-test.txt file. [17:05:00] elukey: LGTM'd [17:05:17] (03Merged) 10jenkins-bot: pre-commit: make required changes [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/881574 (https://phabricator.wikimedia.org/T325198) (owner: 10Ilias Sarantopoulos) [17:05:31] Going to merge the revscoring PR first and I'll spend tomorrow testing the model servers once again [17:06:02] (03PS24) 10Ilias Sarantopoulos: Upgrade the revscoring model server to Python 3.9 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/870517 (https://phabricator.wikimedia.org/T325657) (owner: 10Elukey) [17:06:21] thanks klausman! [17:06:32] we'll see what Hugh thinks about the patch :) [17:06:53] Might take til Monday, since he's usually out on Fridays (I think) [17:07:04] ah snap okok [17:07:08] makes sense [17:07:14] mayeb give him a friendly ping? [17:07:20] we will not merge+deploy tomorrow anyway [17:07:25] (even if it is staging) [17:07:28] There's that [17:08:12] elukey: all good from my side as well. sry for the wrong comments. things are getting clearer every day for me :) [17:08:27] cya tomorrow folks! [17:08:50] isaranto: please comment anytime, it is always good to check assumptions etc.. [17:09:02] never say sorry if you took the time to review my horrors :) [17:13:53] * elukey afk! [17:14:02] have a good evening / rest of the day folks!