[06:07:27] (03CR) 10Kevin Bazira: "thank you for working on this, Georgios. I've added our colleagues as reviewers so that they can share their thoughts; then we shall proce" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1110804 (https://phabricator.wikimedia.org/T383312) (owner: 10Gkyziridis) [07:56:10] 10Lift-Wing, 06Machine-Learning-Team, 13Patch-For-Review: Build and Publish ROCm-Compatible Python Packages - https://phabricator.wikimedia.org/T381859#10456877 (10kevinbazira) >>! In T381859#10448020, @MunizaA wrote: >>>! In T381859#10443645, @kevinbazira wrote: >> >> This process failed for vllm(P71890) w... [08:43:20] Hello folks o/ [08:54:43] georgekyz: o/ [08:55:21] when you want to update an existing change you can use git commit --amend so that all the changes will show in the same patch as different patchsets [08:57:57] isaranto: yeah I got that, Kevin explained it to me. [08:58:26] cool cool cool :D [08:58:50] nevermind me then [09:09:09] (03Abandoned) 10Gkyziridis: articletopic_outlink: update kserve to 0.14.1 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1110790 (https://phabricator.wikimedia.org/T383312) (owner: 10Gkyziridis) [09:09:27] (03Abandoned) 10Gkyziridis: articletopic_outlink: update kserve to 0.14.1 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1110789 (https://phabricator.wikimedia.org/T383312) (owner: 10Gkyziridis) [09:09:44] (03Abandoned) 10Gkyziridis: articletopic_outlink: update kserve to 0.14.1 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1110783 (https://phabricator.wikimedia.org/T383312) (owner: 10Gkyziridis) [09:10:10] (03Abandoned) 10Gkyziridis: articletopic_outlink: update kserve to 0.14.1 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1110745 (https://phabricator.wikimedia.org/T383312) (owner: 10Gkyziridis) [09:28:09] 10Lift-Wing, 06Machine-Learning-Team: [onboarding] Update revertrisk to kserve 0.14.1 - https://phabricator.wikimedia.org/T383119#10457020 (10gkyziridis) @MunizaA following the issue we are facing in T383119#10443851, please advise on whether we can loosen pandas considering [[ https://gitlab.wikimedia.org/rep... [11:16:07] * klausman lunch [13:10:01] hi folks o/ [13:10:16] If you have time please remember to review: https://gerrit.wikimedia.org/r/1110804 [13:55:38] ack! [13:57:11] georgekyz: I'll take a look before our meeting. Did you manage to build and run the service locally with docker? [14:17:59] 06Machine-Learning-Team, 10Observability-Metrics, 10SRE Observability (FY2024/2025-Q2): Gap in metrics rendered from Thanos Rules - https://phabricator.wikimedia.org/T352756#10457949 (10tappof) {F58194787} {F58194798} [14:20:17] 06Machine-Learning-Team, 10Observability-Metrics, 10SRE Observability (FY2024/2025-Q2): Gap in metrics rendered from Thanos Rules - https://phabricator.wikimedia.org/T352756#10457960 (10fgiunchedi) Update after some brainstorm: The gap in recent data for thanos-rule metrics is due to the fact that we query... [14:26:44] isaranto: yes I did [14:27:34] \o/ [14:40:46] (03CR) 10Ilias Sarantopoulos: [C:03+1] "LGTM, much simpler and more maintainable!" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1110804 (https://phabricator.wikimedia.org/T383312) (owner: 10Gkyziridis) [14:41:42] 06Machine-Learning-Team: Expose reference quality isvc on API gateway - https://phabricator.wikimedia.org/T378495#10458084 (10isarantopoulos) a:03isarantopoulos [14:48:22] 10Lift-Wing, 06Machine-Learning-Team: [draft] Update ROCm driver version on Lift Wing nodes - https://phabricator.wikimedia.org/T383230#10458110 (10isarantopoulos) [14:58:19] 10Lift-Wing, 06Machine-Learning-Team: Update ROCm driver version on Lift Wing nodes - https://phabricator.wikimedia.org/T383230#10458157 (10isarantopoulos) [15:14:18] (03PS2) 10Gkyziridis: articletopic_outlink: update kserve to 0.14.1 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1110804 (https://phabricator.wikimedia.org/T383312) [15:34:05] isaranto, klausman o/ - re: https://phabricator.wikimedia.org/T352756#10457960, it seems that there is a promising fix for the SLO metric gaps [15:34:39] there is still some work to do but if it gets fixed, we could start trying Pyrra and https://slo.wikimedia.org/ [15:35:09] (grizzly dashboards, what we used in the past, are not the tool of choice from Olly, Pyrra is preferred) [15:35:39] :+1: thanks for the heads up! [15:48:35] 06Machine-Learning-Team: Test the feasibility of deployment of Aya-expanse model in LiftWing - https://phabricator.wikimedia.org/T379052#10458495 (10isarantopoulos) [16:00:39] 06Machine-Learning-Team, 10Observability-Metrics, 13Patch-For-Review, 10SRE Observability (FY2024/2025-Q3): Gap in metrics rendered from Thanos Rules - https://phabricator.wikimedia.org/T352756#10458605 (10lmata) [18:56:35] 06Machine-Learning-Team, 10MediaWiki-extensions-ORES, 10Edit-Review-Improvements-RC-Page, 10MediaWiki-Recent-changes, 10Moderator-Tools-Team (Kanban): [SPIKE] How could we add topic filtering to Recent Changes? [8H] - https://phabricator.wikimedia.org/T381569#10459596 (10Kgraessle) a:03Kgraessle