[08:23:37] hello folks [08:28:47] while I was checking wikidata's pods I realized one thing [08:29:16] we are fetching features from api-ro.discovery.wmnet (internal endpoint of the mw api) using the hostname wikidata.wikipedia.org [08:29:32] that is totally wrong [08:29:43] we should query wikidata.org [08:29:56] but everything works, I get scores [08:30:21] that is very weird [08:31:42] yeah reproduced locally, it works [08:31:43] lol [08:33:40] even goodfaith works [08:36:42] 10Machine-Learning-Team, 10Data Engineering Planning, 10Research, 10Shared-Data-Infrastructure: Proposal: deprecate the mediawiki.revision-score stream in favour of more streams like mediawiki-revision-score- - https://phabricator.wikimedia.org/T317768 (10EChetty) [08:39:25] (03PS1) 10AikoChou: articlequality: use the fetch_features shared function [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/842695 [08:39:58] elukey: ^^^ I have a big finding! [08:41:44] elukey: and good morning :) [08:43:10] aiko: ah nice going to check it in a minute! [08:43:18] I am still a little confused about the wikidata thing [08:43:43] so yeah my bad, it seems that api-ro servers the wikidata api [08:43:55] friday's confusion [08:44:27] but the wikidata.wikipedia.org hostname still doesn't look right to me [08:45:26] it should be www.wikidata.org [08:48:35] (03CR) 10Elukey: [C: 03+1] "LGTM! Can you add Bug: T320374 so we can tie this code change to the task? Thanks!" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/842695 (owner: 10AikoChou) [08:48:44] nice finding aiko! [08:51:05] this is great so we can share the same code and fix the last corner case for wikidata and articlequality [08:51:37] (03PS2) 10AikoChou: articlequality: use the fetch_features shared function [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/842695 (https://phabricator.wikimedia.org/T320374) [08:53:27] (03CR) 10AikoChou: [C: 03+2] articlequality: use the fetch_features shared function [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/842695 (https://phabricator.wikimedia.org/T320374) (owner: 10AikoChou) [08:55:19] elukey: that's what I thought as well! [09:00:01] (03Merged) 10jenkins-bot: articlequality: use the fetch_features shared function [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/842695 (https://phabricator.wikimedia.org/T320374) (owner: 10AikoChou) [09:06:04] I am a little confused, I am trying to curl api-ro.discovery.wmnet with a host header "wikidata.wikipedia.org" and it works [09:06:39] there is probably a redirect or similar happening [09:08:18] ahhhh okok so httpd on the mw servers accept *.wikipedia.org basically [09:08:27] port 80 namevhost wikipedia (/etc/apache2/sites-available/wikipedia.org.conf:1) [09:08:30] wild alias *.wikipedia.org [09:09:09] okok [09:09:34] so we can probably leave our settings as they are, even if they are not 100% correct in my opinion [09:09:43] ok now I am less confused :D [09:12:46] aiko: I've manually set the new docker image on ml-serve nodes, so we can keep testing [09:13:15] I think that we should probably add wikidata as special use case in staging [09:14:17] running Benthos now [09:19:04] elukey: that's a good idea [09:23:26] aiko: created https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/842697/ to update all the docker images (again sigh) [09:23:40] it also includes your recent fix for articlequality [09:55:41] aiko: thanks for the review :) [09:56:15] aiko: I was wondering one thing - at line 89 of extractor_utils.py we don't really check the HTTP return code [09:56:26] should we do it? And error out early if needed? [10:36:16] * elukey lunch! [13:47:08] Hey all just an update, my youngest kid is still sick and now my other kid hurt her leg. I think I will have OOO for today as well. My apologies to you all [13:49:00] chrisalbon: sorry to hear that, no problem at all! Please take care of your kids :) [13:50:13] I want to hear what happened in the research meeting but i assume that can wait until next week [13:55:07] chrisalbon: nothing big we just had a quick chat about feature datasets, most of the discussion is in https://docs.google.com/document/d/1O5tb9U4EgL7r2Ypl-4jAyjTciAgaKAld1sP_xCpV9Zs/edit [14:06:47] rolling out the new docker images [14:41:07] (03CR) 10Klausman: [C: 03+1] topic: align ORES prediction output with Lift Wing's one [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/842360 (https://phabricator.wikimedia.org/T318932) (owner: 10AikoChou) [14:41:09] the amount of logs that knative/istio produce is huge, sigh [14:41:20] (03CR) 10Klausman: [C: 03+1] draftquality: align ORES prediction output with Lift Wing's one [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/842358 (https://phabricator.wikimedia.org/T318932) (owner: 10AikoChou) [14:41:50] Sorry for being so late with the code reviews. The doc kept me around for longer than I expected [14:48:32] super fine, Aiko will probably rollout the changes next week [14:48:46] I am currently rolling out the last changes for all the revscoring models [14:51:36] done [15:04:50] found a way to reduce the amount of logs from knative [15:11:07] code review incoming :) [15:11:23] Ready :) [15:12:23] this is only for the autoscaler in ml-serve-codfw https://logstash.wikimedia.org/goto/7c7aecbf0cfdc1f5222ac46b6f2caf46 [15:12:46] I think that warn is enough for us [15:12:52] info is definitely too much [15:13:29] Agreed [15:13:51] https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/842829 [15:18:06] lint is unhappy, but I have to yet find the error [15:24:14] checking [15:32:51] let's see if it is better now [15:35:52] lgtm [15:37:01] danke :) [15:43:41] (03PS3) 10AikoChou: topic: align ORES prediction output with Lift Wing's one [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/842360 (https://phabricator.wikimedia.org/T318932) [15:44:14] (03PS3) 10AikoChou: draftquality: align ORES prediction output with Lift Wing's one [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/842358 (https://phabricator.wikimedia.org/T318932) [15:44:59] (03CR) 10AikoChou: [C: 03+2] "Thanks for the review :)" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/842360 (https://phabricator.wikimedia.org/T318932) (owner: 10AikoChou) [15:45:20] (03CR) 10AikoChou: [C: 03+2] draftquality: align ORES prediction output with Lift Wing's one [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/842358 (https://phabricator.wikimedia.org/T318932) (owner: 10AikoChou) [15:57:47] logs rolled out [15:57:57] err log levels for knative [16:02:33] (03Merged) 10jenkins-bot: topic: align ORES prediction output with Lift Wing's one [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/842360 (https://phabricator.wikimedia.org/T318932) (owner: 10AikoChou) [16:08:27] going afk for the weekend folks! [16:08:37] have a nice rest of the day + weekend [16:09:59] \o [16:22:30] bye Luca! :) [16:23:02] (03PS4) 10AikoChou: draftquality: align ORES prediction output with Lift Wing's one [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/842358 (https://phabricator.wikimedia.org/T318932) [16:23:12] heading out as well \o have a nice Friday and weekend, everyone [16:23:40] bye Tobias :) [16:26:10] (03CR) 10AikoChou: draftquality: align ORES prediction output with Lift Wing's one [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/842358 (https://phabricator.wikimedia.org/T318932) (owner: 10AikoChou) [16:31:46] (03Merged) 10jenkins-bot: draftquality: align ORES prediction output with Lift Wing's one [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/842358 (https://phabricator.wikimedia.org/T318932) (owner: 10AikoChou) [18:10:28] (03PS3) 10AikoChou: events.py: fix prediction type in the revision_score_event [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/842414