[04:24:03] <wikibugs>	 10Machine-Learning-Team, 10Add-Link, 10Growth-Team, 10Chinese-Sites, 10User-notice: Deploy "add a link" to 14th round of wikis - https://phabricator.wikimedia.org/T308139 (10kevinbazira)
[04:25:38] <wikibugs>	 10Machine-Learning-Team, 10Add-Link, 10Growth-Team, 10Chinese-Sites, 10User-notice: Deploy "add a link" to 14th round of wikis - https://phabricator.wikimedia.org/T308139 (10kevinbazira) Model evaluation has been completed and below are the backtesting results: | | Precision@0.5 | Recall@0.5 |wawiki | 0....
[11:14:13] <wikibugs>	 (03PS1) 10Elukey: events.py: prioritize the excp handling of ClientResponseError [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/902689 (https://phabricator.wikimedia.org/T328576)
[11:15:35] <elukey>	 aiko: o/
[11:15:56] <klausman>	 \o
[11:16:06] <elukey>	 hello :)
[11:16:09] <klausman>	 Github changed their SSH host key https://github.blog/2023-03-23-we-updated-our-rsa-ssh-host-key/ 
[11:16:27] <klausman>	 So you'll likely see scary errors if you fecth from repos using ssh URLs
[11:20:22] <elukey>	 ack
[11:21:32] <elukey>	 aiko: so I tried the new events code with changeprop/ml-staging and events generated fail to validate, but I noticed a weird thing and I opened the above code review to (hopefully) have better logging.
[11:21:41] <elukey>	 lemme know when you have a moment if you like the idea :)
[11:21:48] <elukey>	 going afk for lunch, ttl!
[11:26:58] <wikibugs>	 (03CR) 10AikoChou: [C: 03+1] "LGTM!" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/902689 (https://phabricator.wikimedia.org/T328576) (owner: 10Elukey)
[13:07:50] <chrisalbon>	 Good morning all
[14:07:17] <wikibugs>	 (03CR) 10Klausman: [C: 03+1] events.py: prioritize the excp handling of ClientResponseError [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/902689 (https://phabricator.wikimedia.org/T328576) (owner: 10Elukey)
[14:15:49] <wikibugs>	 (03CR) 10Elukey: [C: 03+2] events.py: prioritize the excp handling of ClientResponseError [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/902689 (https://phabricator.wikimedia.org/T328576) (owner: 10Elukey)
[14:23:11] <wikibugs>	 10Machine-Learning-Team, 10artificial-intelligence, 10Bad-Words-Detection-System, 10revscoring, 10Thai-Sites: Add language support for Thai (th) - https://phabricator.wikimedia.org/T304045 (10PatsagornY)
[14:32:36] <wikibugs>	 (03Merged) 10jenkins-bot: events.py: prioritize the excp handling of ClientResponseError [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/902689 (https://phabricator.wikimedia.org/T328576) (owner: 10Elukey)
[14:33:37] <wikibugs>	 10Lift-Wing, 10Machine-Learning-Team: Move Revert-risk language agnostic model from staging to production - https://phabricator.wikimedia.org/T332998 (10achou)
[15:11:59] <elukey>	 aiko: your fix works! I wasn't testing the right pod :)
[15:12:14] <elukey>	 but I found a possible good fix for the logging issue, sooo I am happy anyway
[15:12:48] <elukey>	 I sent some page_change events to the special staging kafka topic that we use with changeprop staging, and I see events in the revscoring-test topic now!
[15:17:59] <elukey>	 looks working very well
[15:20:42] <elukey>	 I sent thousands of events and everything went fine
[15:21:51] <wikibugs>	 10Machine-Learning-Team, 10Platform Team Workboards (Platform Engineering Reliability): Implement new mediawiki.revision-score streams with Lift Wing - https://phabricator.wikimedia.org/T328576 (10elukey) Finally we have something working, I've just tested ~20k events in changeprop staging, hitting the ml-stag...
[15:30:52] <wikibugs>	 10Machine-Learning-Team: Review and test the AMD GPU kubernetes plugin - https://phabricator.wikimedia.org/T333009 (10elukey)
[15:32:37] <wikibugs>	 10Machine-Learning-Team, 10Analytics-Radar, 10Data-Engineering-Icebox, 10Patch-For-Review: Upgrade ROCm to 4.5 - https://phabricator.wikimedia.org/T295661 (10elukey) Hi @fkaelin, we'll definitely try to upgrade during the next quarter to the latest ROCm release :)
[15:33:56] <elukey>	 ok I'll re-try to add the drafttopic stream
[15:33:59] <elukey>	 fingers crossed
[15:40:58] <elukey>	 wow something is working
[16:22:36] <wikibugs>	 10Machine-Learning-Team, 10Platform Team Workboards (Platform Engineering Reliability): Implement new mediawiki.revision-score streams with Lift Wing - https://phabricator.wikimedia.org/T328576 (10elukey) Stream deployed, I see traffic!!  https://grafana.wikimedia.org/d/zsdYRV7Vk/istio-sidecar?orgId=1&var-clus...
[16:30:53] <aiko>	 yayyy that's great!!!! \o/
[16:33:31] <elukey>	 aiko: for some weird reason changeprop is only pulling from the codfw page_change topic, not eqiad
[16:33:41] <elukey>	 I'll try to investigate on monday why, but so far all good!
[16:36:15] <aiko>	 elukey: good job Luca! congrats :D
[16:36:55] <elukey>	 aiko: you worked on it too! Milestone for the whole team :)
[16:37:03] <elukey>	 going afk folks! Have a good rest of the day and weekend :)
[16:37:53] <aiko>	 o/ have a nice weekend!
[16:51:50] <wikibugs>	 10Machine-Learning-Team, 10API Platform, 10API-Portal, 10Platform Team Initiatives (API Gateway Roadmap): Add documentation about LiftWing to the API Portal - https://phabricator.wikimedia.org/T325759 (10apaskulin) > For example, revscoring could point to https://github.com/wikimedia/revscoring, and outlin...
[23:41:24] <wikibugs>	 10Machine-Learning-Team, 10Add-Link, 10Growth-Team (Current Sprint), 10User-notice: Deploy "add a link" to 6th round of wikis - https://phabricator.wikimedia.org/T304550 (10Etonkovidova) Checked some wikis from the list - `ce.wp`, `it.wp`, `cy.wp`, `chy.wp`. Generally works as expected - no new issues; log...