[00:10:51] 10Machine-Learning-Team, 10ORES, 10Advanced-Search, 10All-and-every-Wikisource, and 67 others: Remove unnecessary targets definitions - https://phabricator.wikimedia.org/T328497 (10Jdlrobson) [00:52:50] 10Machine-Learning-Team, 10Add-Link, 10Growth-Team, 10Chinese-Sites, 10User-notice: Deploy "add a link" to 14th round of wikis - https://phabricator.wikimedia.org/T308139 (10kevinbazira) [00:52:55] 10Machine-Learning-Team, 10Add-Link, 10Growth-Team, 10Chinese-Sites, 10User-notice: Deploy "add a link" to 14th round of wikis - https://phabricator.wikimedia.org/T308139 (10kevinbazira) @kostajh, we published datasets for all 12/16 models that passed the evaluation in this round. [07:29:16] 10Machine-Learning-Team, 10DBA, 10Data Pipelines, 10Data-Engineering-Planning, and 10 others: eqiad row B switches upgrade - https://phabricator.wikimedia.org/T330165 (10Jelto) [07:33:36] hello folks [07:36:38] 10Machine-Learning-Team, 10DBA, 10Data Pipelines, 10Data-Engineering-Planning, and 10 others: eqiad row B switches upgrade - https://phabricator.wikimedia.org/T330165 (10ayounsi) 05Open→03Resolved a:03ayounsi Thanks again everybody! [07:56:25] 10Machine-Learning-Team, 10Observability-Logging: Logging spam from revscoring deploys - https://phabricator.wikimedia.org/T320468 (10elukey) 05Open→03Resolved Thanks a lot! [07:57:57] 10Machine-Learning-Team: Define SLI/SLO for Lift Wing - https://phabricator.wikimedia.org/T327620 (10elukey) From another task: https://wikitech.wikimedia.org/wiki/SLO/template_instructions https://wikitech.wikimedia.org/wiki/SLO/template [08:03:15] hii [08:05:13] o/ [09:40:06] 10Machine-Learning-Team, 10DBA, 10Data-Engineering, 10Infrastructure-Foundations, and 9 others: eqiad row C switches upgrade - https://phabricator.wikimedia.org/T331882 (10elukey) [09:44:09] 10Machine-Learning-Team, 10DBA, 10Data Pipelines, 10Data-Engineering-Planning, and 10 others: eqiad row B switches upgrade - https://phabricator.wikimedia.org/T330165 (10fgiunchedi) [09:45:26] 10Machine-Learning-Team, 10Data-Engineering, 10Data-Persistence, 10Discovery-Search, and 7 others: eqiad row D switches upgrade - https://phabricator.wikimedia.org/T333377 (10elukey) [10:09:46] 10Machine-Learning-Team, 10API Platform, 10API-Portal, 10Platform Team Initiatives (API Gateway Roadmap): Add documentation about LiftWing to the API Portal - https://phabricator.wikimedia.org/T325759 (10elukey) @apaskulin do you think that we are good for the initial version of the documentation, or shoul... [10:29:05] * elukey lunch! [11:09:54] * isaranto lunch as well [11:11:33] \o [11:12:03] elukey: https://github.com/Syllo/nvtop may prove useful in the future. Like top/htop, but for GPUs. Not nvidia-specific. [12:02:18] 10Machine-Learning-Team, 10Add-Link, 10Growth-Team, 10User-notice: Deploy "add a link" to 15th round of wikis - https://phabricator.wikimedia.org/T308141 (10kevinbazira) a:03kevinbazira [13:35:07] klausman: ah nice! We have `radeontop` for the moment that is not bad, nvtop seems really nice [13:35:26] I think nvtop is a bit better for seeing GPU use over time [14:35:50] 10Machine-Learning-Team, 10Epic: Lift Wing (metrics, documentation and user onboarding) - https://phabricator.wikimedia.org/T333453 (10calbon) [15:09:54] 10Machine-Learning-Team, 10Epic: Lift Wing improvements - https://phabricator.wikimedia.org/T333453 (10elukey) [15:10:44] 10Machine-Learning-Team: Define SLI/SLO for Lift Wing - https://phabricator.wikimedia.org/T327620 (10elukey) [15:10:46] 10Machine-Learning-Team, 10Epic: Lift Wing improvements - https://phabricator.wikimedia.org/T333453 (10elukey) [15:11:45] 10Machine-Learning-Team, 10API Platform, 10API-Portal, 10Platform Team Initiatives (API Gateway Roadmap): Add documentation about LiftWing to the API Portal - https://phabricator.wikimedia.org/T325759 (10elukey) [15:11:47] 10Machine-Learning-Team, 10Epic: Lift Wing improvements - https://phabricator.wikimedia.org/T333453 (10elukey) [15:11:56] 10Machine-Learning-Team, 10Epic: Lift Wing improvements - https://phabricator.wikimedia.org/T333453 (10elukey) [15:11:58] 10Lift-Wing, 10Machine-Learning-Team, 10Documentation: Create technical documentation for Lift Wing Infrastructure - https://phabricator.wikimedia.org/T276601 (10elukey) [15:12:14] 10Machine-Learning-Team: Add guidelines about how to write/use asyncio code in KServe - https://phabricator.wikimedia.org/T324313 (10elukey) [15:12:16] 10Machine-Learning-Team, 10Epic: Lift Wing improvements - https://phabricator.wikimedia.org/T333453 (10elukey) [15:12:32] 10Machine-Learning-Team, 10Epic: Lift Wing improvements - https://phabricator.wikimedia.org/T333453 (10elukey) [15:12:34] 10Lift-Wing, 10Machine-Learning-Team, 10Documentation: Improve Lift Wing documentation - https://phabricator.wikimedia.org/T316098 (10elukey) [15:12:42] 10Machine-Learning-Team: create checklist before adding models to api gateway/prod - https://phabricator.wikimedia.org/T332711 (10elukey) [15:12:45] 10Machine-Learning-Team, 10Epic: Lift Wing improvements - https://phabricator.wikimedia.org/T333453 (10elukey) [15:12:57] 10Machine-Learning-Team: Create documentation for a workflow for evaluating models submitted for deployment. - https://phabricator.wikimedia.org/T269171 (10elukey) [15:25:35] 10Machine-Learning-Team: Review ORES traffic to better understand Lift Wing's requirements - https://phabricator.wikimedia.org/T325763 (10elukey) 05Open→03Resolved [15:25:37] 10Machine-Learning-Team: Migrate ORES clients to LiftWing - https://phabricator.wikimedia.org/T312518 (10elukey) [15:33:11] 10Machine-Learning-Team, 10Epic: Lift Wing improvements to get out of MVP state - https://phabricator.wikimedia.org/T333453 (10Aklapper) [15:48:05] thanks for all the phab work elukey: ! [15:48:22] more to come! :D [15:49:01] Luca sure is fab (in the 60s English slang sense) [15:50:30] 10Machine-Learning-Team, 10Epic: Add GPUs to the Machine Learning infrastructure - https://phabricator.wikimedia.org/T333462 (10elukey) [15:51:01] I'll deal with phab tomorrow. I created a draft patch for the ores legacy endpoint and if u have some time Luca we could take a look together tomorrow [15:51:09] this is the one https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/904178 [15:51:42] 10Machine-Learning-Team: Investigate procuring and installing two GPUs on Lift Wing - https://phabricator.wikimedia.org/T327923 (10elukey) [15:51:44] 10Machine-Learning-Team, 10Epic: Add GPUs to the Machine Learning infrastructure - https://phabricator.wikimedia.org/T333462 (10elukey) [15:51:52] isaranto: sure! [15:51:58] 10Machine-Learning-Team, 10Analytics-Radar, 10Data-Engineering-Icebox, 10Patch-For-Review: Upgrade ROCm to 4.5 - https://phabricator.wikimedia.org/T295661 (10elukey) [15:52:02] 10Machine-Learning-Team, 10Epic: Add GPUs to the Machine Learning infrastructure - https://phabricator.wikimedia.org/T333462 (10elukey) [15:52:21] 10Machine-Learning-Team, 10Analytics-Radar, 10Data-Engineering-Icebox: Review ROCm deployment procedures and current packages - https://phabricator.wikimedia.org/T275896 (10elukey) [15:52:27] 10Machine-Learning-Team, 10Epic: Add GPUs to the Machine Learning infrastructure - https://phabricator.wikimedia.org/T333462 (10elukey) [15:52:31] 10Machine-Learning-Team: Review and test the AMD GPU kubernetes plugin - https://phabricator.wikimedia.org/T333009 (10elukey) [15:52:35] 10Machine-Learning-Team, 10Epic: Add GPUs to the Machine Learning infrastructure - https://phabricator.wikimedia.org/T333462 (10elukey) [16:02:30] loggin off, cu tomorrow! [16:03:42] o/ [16:06:06] 10Machine-Learning-Team, 10CirrusSearch, 10Discovery-Search (Current work): Add outlink topic model predictions to CirrusSearch indices - https://phabricator.wikimedia.org/T328276 (10dcausse) [16:07:07] going afk as well [16:14:42] 10Machine-Learning-Team, 10DBA, 10Data-Engineering, 10Infrastructure-Foundations, and 9 others: eqiad row C switches upgrade - https://phabricator.wikimedia.org/T331882 (10jcrespo) [17:59:31] 10Machine-Learning-Team, 10API Platform, 10API-Portal, 10Platform Team Initiatives (API Gateway Roadmap): Add documentation about LiftWing to the API Portal - https://phabricator.wikimedia.org/T325759 (10apaskulin) @elukey, I added a section for the [[ https://api.wikimedia.org/wiki/API_reference/Service/L...