[07:04:22] 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26), 07Essential-Work: Rebuild vLLM base images to use deb.debian.org in apt sources - https://phabricator.wikimedia.org/T429667 (10kevinbazira) 03NEW [10:37:01] 06Machine-Learning-Team (Q4 FY2025-26): Editing Suggestions - Increase/refresh LLM generated suggestions. - https://phabricator.wikimedia.org/T428882#12036681 (10OKarakaya-WMF) {F89485965} ` array([np.int64(27413), np.int64(22), np.int64(668), 'Demographics_of_Sri_Lanka', np.int64(1358095229), '2... [11:18:43] 06Machine-Learning-Team (Q4 FY2025-26), 10Liberica, 10Prod-Kubernetes, 06Traffic, and 2 others: Migrate ML k8s apiserver and services to IPIP - https://phabricator.wikimedia.org/T420438#12036722 (10isarantopoulos) [12:07:04] 06Machine-Learning-Team (Q4 FY2025-26): LLM observability - https://phabricator.wikimedia.org/T429597#12036783 (10isarantopoulos) [12:36:55] 06Machine-Learning-Team (Q4 FY2025-26), 10Wikidata, 06Wikidata-Omega: Threshold analysis for WMDE use cases of Wikidata RR model - https://phabricator.wikimedia.org/T429049#12036857 (10isarantopoulos) This is blocked by {T420883} which needs to be completed first. We need to assess if a backfill will be need... [12:47:14] 06Machine-Learning-Team (Q4 FY2025-26), 13Patch-For-Review: Add Wikidata RevertRisk predictions to mediawiki.page_revert_risk_prediction_change - https://phabricator.wikimedia.org/T420883#12036880 (10isarantopoulos) a:03achou [13:17:29] 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26), 06ServiceOps new, 10ServiceOps-SharedInfra, 06Traffic: Host Qwen 3.6-27B as an inference service - https://phabricator.wikimedia.org/T425680#12036973 (10gkyziridis) == CI Failure Root Cause == The postmerge of [[ https://gerrit.wikimedia.org/r/c/machin... [13:58:23] (03PS1) 10AikoChou: events: bump prediction_classification_change schema to 1.3.0 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304587 (https://phabricator.wikimedia.org/T421237) [13:59:50] (03CR) 10CI reject: [V:04-1] events: bump prediction_classification_change schema to 1.3.0 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304587 (https://phabricator.wikimedia.org/T421237) (owner: 10AikoChou) [14:00:45] 06Machine-Learning-Team (Q4 FY2025-26), 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10Event-Platform, 13Patch-For-Review: `mediawiki.page_change.v1`: two schema validation errors causing events to be silently dropped by EventGate - https://phabricator.wikimedia.org/T421237#12037112 (10achou) [14:21:49] 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26), 07Essential-Work: Rebuild vLLM base images to use deb.debian.org in apt sources - https://phabricator.wikimedia.org/T429667#12037207 (10kevinbazira) The vllm022 image has been rebuilt and now uses `deb.debian.org` in apt sources: ` $ docker run --network=h... [14:30:22] (03PS1) 10Kevin Bazira: policy-violation: update cope-b-a4b blubberfile to use the rebuilt vllm022 base image (T429667#12037207) [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304591 (https://phabricator.wikimedia.org/T429667) [14:41:30] (03PS2) 10AikoChou: events: bump prediction_classification_change schema to 1.3.0 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304587 (https://phabricator.wikimedia.org/T421237) [14:45:04] (03CR) 10Gkyziridis: [C:03+1] "Thnx for working on this!" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304591 (https://phabricator.wikimedia.org/T429667) (owner: 10Kevin Bazira) [14:47:25] (03CR) 10Kevin Bazira: [C:03+2] policy-violation: update cope-b-a4b blubberfile to use the rebuilt vllm022 base image (T429667#12037207) [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304591 (https://phabricator.wikimedia.org/T429667) (owner: 10Kevin Bazira) [14:48:44] (03Merged) 10jenkins-bot: policy-violation: update cope-b-a4b blubberfile to use the rebuilt vllm022 base image (T429667#12037207) [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1304591 (https://phabricator.wikimedia.org/T429667) (owner: 10Kevin Bazira) [15:27:09] 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26), 07Essential-Work, 13Patch-For-Review: Rebuild vLLM base images to use deb.debian.org in apt sources - https://phabricator.wikimedia.org/T429667#12037422 (10kevinbazira) Following T429667#12037207, we updated the cope-b-a4b model server to use the rebuilt...