[07:26:54] <wikibugs>	 10Lift-Wing, 06Machine-Learning-Team: Redeploy Liftwing pick up current mirror - https://phabricator.wikimedia.org/T429675#12039263 (10MoritzMuehlenhoff)
[07:31:42] <wikibugs>	 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26): Redeploy Liftwing pick up current mirror - https://phabricator.wikimedia.org/T429675#12039289 (10isarantopoulos)
[07:52:42] <wikibugs>	 (03PS1) 10Gkyziridis: qwen36-27b: copy build venv packages into production [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304729 (https://phabricator.wikimedia.org/T425680)
[08:08:21] <wikibugs>	 (03CR) 10Kevin Bazira: [C:03+1] qwen36-27b: copy build venv packages into production [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304729 (https://phabricator.wikimedia.org/T425680) (owner: 10Gkyziridis)
[08:21:29] <wikibugs>	 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26), 13Patch-For-Review: Deploy CoPE-B-A4B on LiftWing - https://phabricator.wikimedia.org/T427497#12039472 (10kevinbazira) >>! In T427497#12033330, @isarantopoulos wrote: > Awesome work Kevin! > It would be helpful to run load tests against the `v1/chat/comple...
[08:22:41] <wikibugs>	 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26), 13Patch-For-Review: Deploy CoPE-B-A4B on LiftWing - https://phabricator.wikimedia.org/T427497#12039473 (10kevinbazira) Following T427497#12033298, the cope-b-a4b isvc has been deployed in LiftWing production (eqiad). It can be accessed by tools that run wi...
[08:23:53] <wikibugs>	 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26), 13Patch-For-Review: Deploy CoPE-B-A4B on LiftWing - https://phabricator.wikimedia.org/T427497#12039478 (10isarantopoulos) Sounds great, thanks for clarifying!
[08:23:59] <wikibugs>	 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26), 13Patch-For-Review: Deploy CoPE-B-A4B on LiftWing - https://phabricator.wikimedia.org/T427497#12039479 (10kevinbazira) 05Open→03Resolved
[08:29:16] <wikibugs>	 (03PS1) 10Gkyziridis: outlink: bump bookworm base to 20260510 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304747 (https://phabricator.wikimedia.org/T429675)
[08:29:54] <wikibugs>	 (03CR) 10Gkyziridis: [C:03+2] qwen36-27b: copy build venv packages into production [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304729 (https://phabricator.wikimedia.org/T425680) (owner: 10Gkyziridis)
[08:32:12] <wikibugs>	 (03Merged) 10jenkins-bot: qwen36-27b: copy build venv packages into production [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304729 (https://phabricator.wikimedia.org/T425680) (owner: 10Gkyziridis)
[08:39:20] <wikibugs>	 06Machine-Learning-Team (Q4 FY2025-26), 07Essential-Work: Support GPU Partition specification in deployments - https://phabricator.wikimedia.org/T424923#12039535 (10BWojtowicz-WMF) **Current state. **  There are two patterns in use right now: 1. Default (untainted) GPU pool - used by e.g. `edit-check`, `revert...
[08:48:00] <wikibugs>	 06Machine-Learning-Team (Q4 FY2025-26): LLM observability - https://phabricator.wikimedia.org/T429597#12039584 (10DPogorzelski-WMF) amd-smi reports wrong memory numbers when using NPS4+CPX partitioning scheme. The lead partition of each physical GPU shows full VRAM amount of the physical device (192GB) instead o...
[08:48:03] <wikibugs>	 (03CR) 10Kevin Bazira: outlink: bump bookworm base to 20260510 (031 comment) [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304747 (https://phabricator.wikimedia.org/T429675) (owner: 10Gkyziridis)
[08:57:24] <wikibugs>	 (03PS2) 10Gkyziridis: outlink: bump bookworm base to 20260510 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304747 (https://phabricator.wikimedia.org/T429675)
[09:04:13] <wikibugs>	 (03CR) 10Kevin Bazira: [C:03+1] outlink: bump bookworm base to 20260510 (031 comment) [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304747 (https://phabricator.wikimedia.org/T429675) (owner: 10Gkyziridis)
[09:04:16] <wikibugs>	 (03PS3) 10Gkyziridis: outlink: bump bookworm base to 20260621 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304747 (https://phabricator.wikimedia.org/T429675)
[09:10:22] <wikibugs>	 06Machine-Learning-Team (Q4 FY2025-26), 07Essential-Work: Support GPU Partition specification in deployments - https://phabricator.wikimedia.org/T424923#12039736 (10BWojtowicz-WMF) First, let me give some context on options I'll share below - taints/tolerations and nodeAffinity do opposite jobs: a taint repels...
[09:44:25] <wikibugs>	 06Machine-Learning-Team (Q4 FY2025-26): Editing Suggestions - Increase/refresh LLM generated suggestions. - https://phabricator.wikimedia.org/T428882#12039887 (10OKarakaya-WMF) The first batch has processed 4735 articles. We have 11902 suggestions in total 9451 of them got 8 or higher overall score. 7863 of them...
[09:49:20] <wikibugs>	 06Machine-Learning-Team (Q4 FY2025-26): Editing Suggestions - Increase/refresh LLM generated suggestions. - https://phabricator.wikimedia.org/T428882#12039895 (10OKarakaya-WMF) I could not upload the csv with score into phab due to size. Therefore, I share it here: https://docs.google.com/spreadsheets/d/1ShvfIAW...
[10:04:46] <wikibugs>	 (03PS4) 10Gkyziridis: outlink: bump bookworm base to 20260621 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304747 (https://phabricator.wikimedia.org/T429675)
[10:09:16] <wikibugs>	 (03CR) 10Ozge: [C:03+1] outlink: bump bookworm base to 20260621 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304747 (https://phabricator.wikimedia.org/T429675) (owner: 10Gkyziridis)
[10:21:39] <wikibugs>	 (03CR) 10Gkyziridis: [C:03+2] outlink: bump bookworm base to 20260621 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304747 (https://phabricator.wikimedia.org/T429675) (owner: 10Gkyziridis)
[10:25:50] <wikibugs>	 (03Merged) 10jenkins-bot: outlink: bump bookworm base to 20260621 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304747 (https://phabricator.wikimedia.org/T429675) (owner: 10Gkyziridis)
[10:52:48] <wikibugs>	 (03PS3) 10AikoChou: revertrisk-wikidata: add predictions to events stream [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1259966 (https://phabricator.wikimedia.org/T420883) (owner: 10Ilias Sarantopoulos)
[10:53:15] <wikibugs>	 (03PS4) 10AikoChou: revertrisk-wikidata: add predictions to events stream [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1259966 (https://phabricator.wikimedia.org/T420883) (owner: 10Ilias Sarantopoulos)
[10:58:01] <wikibugs>	 (03CR) 10CI reject: [V:04-1] revertrisk-wikidata: add predictions to events stream [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1259966 (https://phabricator.wikimedia.org/T420883) (owner: 10Ilias Sarantopoulos)
[10:58:07] <wikibugs>	 (03PS5) 10AikoChou: revertrisk-wikidata: add predictions to events stream [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1259966 (https://phabricator.wikimedia.org/T420883) (owner: 10Ilias Sarantopoulos)
[11:01:54] <wikibugs>	 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26), 06ServiceOps new, 10ServiceOps-SharedInfra, 06Traffic: Host Qwen 3.6-27B as an inference service - https://phabricator.wikimedia.org/T425680#12040084 (10gkyziridis) === Update === After the vllm image is updated and tested in the policy-violation model...
[11:02:03] <wikibugs>	 (03CR) 10CI reject: [V:04-1] revertrisk-wikidata: add predictions to events stream [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1259966 (https://phabricator.wikimedia.org/T420883) (owner: 10Ilias Sarantopoulos)
[11:36:34] <wikibugs>	 (03CR) 10Bartosz Wójtowicz: [C:03+1] "LGTM! However, I see that there is a merge conflict so we need to rebase. I also left a small nit to the test file, but it's nothing very " [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304587 (https://phabricator.wikimedia.org/T421237) (owner: 10AikoChou)
[12:17:27] <wikibugs>	 06Machine-Learning-Team, 10EditCheck, 06Growth-Team, 10Revise-Tone-Structured-Task, and 3 others: Tone check: Improve handling of quoted content - https://phabricator.wikimedia.org/T426362#12040420 (10Esanders)
[13:43:40] <wikibugs>	 (03PS3) 10AikoChou: events: bump prediction_classification_change schema to 1.3.0 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304587 (https://phabricator.wikimedia.org/T421237)
[13:47:24] <wikibugs>	 (03CR) 10CI reject: [V:04-1] events: bump prediction_classification_change schema to 1.3.0 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304587 (https://phabricator.wikimedia.org/T421237) (owner: 10AikoChou)
[13:48:24] <wikibugs>	 (03PS4) 10AikoChou: events: bump prediction_classification_change schema to 1.3.0 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304587 (https://phabricator.wikimedia.org/T421237)
[13:57:16] <wikibugs>	 (03CR) 10AikoChou: [C:03+2] "Thanks for the review :)" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304587 (https://phabricator.wikimedia.org/T421237) (owner: 10AikoChou)
[14:01:22] <wikibugs>	 (03Merged) 10jenkins-bot: events: bump prediction_classification_change schema to 1.3.0 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304587 (https://phabricator.wikimedia.org/T421237) (owner: 10AikoChou)
[14:22:17] <wikibugs>	 (03CR) 10Ottomata: "TY!" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304587 (https://phabricator.wikimedia.org/T421237) (owner: 10AikoChou)
[14:23:08] <wikibugs>	 (03PS1) 10Gkyziridis: qwen36: fix aiter import crash by symlinking version-hip to version [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304823 (https://phabricator.wikimedia.org/T425680)
[14:33:41] <wikibugs>	 (03PS1) 10AikoChou: blubber: bump base images to drop mirrors.wikimedia.org [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304824 (https://phabricator.wikimedia.org/T429675)
[14:34:48] <wikibugs>	 (03PS2) 10Gkyziridis: qwen36: add hipconfig to fix aiter get_hip_version() import crash [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304823 (https://phabricator.wikimedia.org/T425680)
[14:35:46] <wikibugs>	 (03PS2) 10AikoChou: blubber: bump base images to drop mirrors.wikimedia.org [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304824 (https://phabricator.wikimedia.org/T429675)
[14:41:13] <wikibugs>	 (03PS1) 10Kevin Bazira: qwen36: add hipconfig shim to entrypoint.sh to unblock AITER import [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304827 (https://phabricator.wikimedia.org/T425680)
[14:46:34] <wikibugs>	 (03PS2) 10Kevin Bazira: qwen36: add hipconfig shim to entrypoint.sh to unblock AITER import [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304827 (https://phabricator.wikimedia.org/T425680)
[14:47:33] <wikibugs>	 (03PS3) 10Kevin Bazira: qwen36: add hipconfig shim to entrypoint.sh to unblock AITER import [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304827 (https://phabricator.wikimedia.org/T425680)
[14:55:51] <wikibugs>	 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26), 13Patch-For-Review: Redeploy Liftwing pick up current mirror - https://phabricator.wikimedia.org/T429675#12041104 (10achou) I filed a [[ https://gerrit.wikimedia.org/r/c/machinelearning/liftwing/inference-services/+/1304824 | this patch ]] to bump the base...
[14:57:40] <wikibugs>	 (03CR) 10Ozge: [C:03+1] blubber: bump base images to drop mirrors.wikimedia.org [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304824 (https://phabricator.wikimedia.org/T429675) (owner: 10AikoChou)
[15:06:29] <wikibugs>	 (03CR) 10Gkyziridis: [C:03+1] qwen36: add hipconfig shim to entrypoint.sh to unblock AITER import [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304827 (https://phabricator.wikimedia.org/T425680) (owner: 10Kevin Bazira)
[15:14:55] <wikibugs>	 (03CR) 10Kevin Bazira: [C:03+2] qwen36: add hipconfig shim to entrypoint.sh to unblock AITER import [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304827 (https://phabricator.wikimedia.org/T425680) (owner: 10Kevin Bazira)
[15:15:32] <wikibugs>	 (03Merged) 10jenkins-bot: qwen36: add hipconfig shim to entrypoint.sh to unblock AITER import [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304827 (https://phabricator.wikimedia.org/T425680) (owner: 10Kevin Bazira)
[20:51:38] <wikibugs>	 06Machine-Learning-Team, 10Add-Link-Structured-Task, 06Growth-Team: Reduce tendency of Add a Link to suggest overlinks - https://phabricator.wikimedia.org/T415622#12042668 (10Sdkb) Hmm, @OKarakaya-WMF, would there be any way to test out how such a filter would affect the suggestions?  My instinct is that tha...
[20:57:26] <wikibugs>	 06Machine-Learning-Team, 10Add-Link-Structured-Task, 10Community Feedback (Growth), 06Growth-Team: AI/ML model update request: Named Entity Recognition for Add-a-Link - https://phabricator.wikimedia.org/T405185#12042695 (10Sdkb) > Maybe it would be good to teach the check to present links to other Wikipedi...