[07:57:26] 06Machine-Learning-Team (Q1 FY2026-27), 05Goal, 07OKR-Work: Audio quality evaluation for TTS v1: ASR round-trip benchmark and resulting fixes - https://phabricator.wikimedia.org/T433923 (10kevinbazira) 03NEW [09:26:22] 06Machine-Learning-Team (Q1 FY2026-27): Benchmark the public API endpoint (from Toolforge) - https://phabricator.wikimedia.org/T432830#12182192 (10achou) 05Open→03Resolved a:03achou Public-endpoint baselines run from Toolforge for both models, results and overhead comparison posted above. Wrapper fix f... [09:26:36] 06Machine-Learning-Team (Q1 FY2026-27): Benchmark the public API endpoint (from Toolforge) - https://phabricator.wikimedia.org/T432830#12182197 (10achou) [09:30:39] 06Machine-Learning-Team (Q1 FY2026-27): Establish a benchmarking baseline for LiftWing LLM serving - https://phabricator.wikimedia.org/T431851#12182213 (10achou) 05Open→03Resolved a:03achou Baseline established and tooling landed (see [[ https://gerrit.wikimedia.org/r/plugins/gitiles/machinelearning/li... [09:31:09] 06Machine-Learning-Team (Q1 FY2026-27): Establish a benchmarking baseline for LiftWing LLM serving - https://phabricator.wikimedia.org/T431851#12182220 (10achou) [09:36:59] 06Machine-Learning-Team (Q1 FY2026-27), 05Goal, 07OKR-Work: Audio quality evaluation for TTS v1: ASR round-trip benchmark and resulting fixes - https://phabricator.wikimedia.org/T433923#12182276 (10kevinbazira) I have run the evaluation on 50 sections of the two pinned reference articles, generated under rul... [09:37:39] 06Machine-Learning-Team (Q1 FY2026-27), 05Goal, 07OKR-Work: Audio quality evaluation for TTS v1: ASR round-trip benchmark and resulting fixes - https://phabricator.wikimedia.org/T433923#12182278 (10kevinbazira) [10:52:00] (03PS1) 10Kevin Bazira: tts-section-generator: strip hidden-begin charts, pronunciation whitelist from ASR eval (ruleset 2026.08.04) [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1320813 (https://phabricator.wikimedia.org/T433923) [10:59:46] (03CR) 10CI reject: [V:04-1] tts-section-generator: strip hidden-begin charts, pronunciation whitelist from ASR eval (ruleset 2026.08.04) [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1320813 (https://phabricator.wikimedia.org/T433923) (owner: 10Kevin Bazira) [11:05:57] (03PS2) 10Kevin Bazira: tts-section-generator: strip hidden-begin charts, pronunciation whitelist from ASR eval (ruleset 2026.08.04) [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1320813 (https://phabricator.wikimedia.org/T433923) [11:19:16] (03CR) 10Kevin Bazira: [C:03+2] tts-section-generator: strip hidden-begin charts, pronunciation whitelist from ASR eval (ruleset 2026.08.04) [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1320813 (https://phabricator.wikimedia.org/T433923) (owner: 10Kevin Bazira) [11:42:54] 06Machine-Learning-Team (Q1 FY2026-27), 06Data-Platform-SRE (2026-07-03 - 2026-07-31): [draft] ml-serve: review documentation and apply AMD MI300X system-optimization settings to ml-serve1012-1015 - https://phabricator.wikimedia.org/T431553#12182836 (10ops-monitoring-bot) Icinga downtime and Alertmanager silen... [11:44:20] 06Machine-Learning-Team (Q1 FY2026-27), 06Data-Platform-SRE (2026-07-03 - 2026-07-31): [draft] ml-serve: review documentation and apply AMD MI300X system-optimization settings to ml-serve1012-1015 - https://phabricator.wikimedia.org/T431553#12182840 (10klausman) I will reboot into BIOS and take screenshots of... [11:59:17] 06Machine-Learning-Team (Q1 FY2026-27), 06Data-Platform-SRE (2026-07-03 - 2026-07-31): [draft] ml-serve: review documentation and apply AMD MI300X system-optimization settings to ml-serve1012-1015 - https://phabricator.wikimedia.org/T431553#12182871 (10klausman) Screenshots of the relevant BIOS screens: {F971... [12:27:05] 06Machine-Learning-Team (Q1 FY2026-27): Publish V0 LiftWing LLM documentation on Wikitech - https://phabricator.wikimedia.org/T431847#12182917 (10kevinbazira) As [requested on slack](https://wikimedia.slack.com/archives/G01A0FNPLG4/p1785837118275289), I had a look at the [[https://wikitech.wikimedia.org/w/index.... [13:02:36] 06Machine-Learning-Team, 13Patch-For-Review: Upgrade edit-check to kserve==0.17 - https://phabricator.wikimedia.org/T422812#12182996 (10isarantopoulos) [13:04:13] 06Machine-Learning-Team: Upgrade revise-tone-task-generator to kserve==0.17 - https://phabricator.wikimedia.org/T422797#12183001 (10isarantopoulos) [13:05:00] 06Machine-Learning-Team (Q1 FY2026-27): [draft] RR-Wikidata: Decide on Linked Artifact Cache for stream (WME consumption via Hoarde) - https://phabricator.wikimedia.org/T433699#12183009 (10isarantopoulos) [13:06:18] 10Lift-Wing, 06Machine-Learning-Team, 07Essential-Work: Update kserve Python package to 0.17 across all inference services - https://phabricator.wikimedia.org/T422591#12183014 (10isarantopoulos) [13:07:29] 06Machine-Learning-Team, 07Essential-Work: Update blubber version in inference services images - https://phabricator.wikimedia.org/T400446#12183020 (10isarantopoulos) [13:10:03] 06Machine-Learning-Team, 07Essential-Work: Override KServe default OpenAPI specs with the custom specs in docs - https://phabricator.wikimedia.org/T429305#12183030 (10isarantopoulos) [13:10:18] 06Machine-Learning-Team (Q1 FY2026-27), 07Essential-Work: ml-serve: assess and raise resource quotas to support rolling deployments in revertrisk (and other) namespaces - https://phabricator.wikimedia.org/T426947#12183032 (10isarantopoulos) [13:11:54] 06Machine-Learning-Team, 06Growth-Team, 10Revise-Tone-Structured-Task: Revise Tone: populate suggestions for new predicted topics - https://phabricator.wikimedia.org/T432765#12183036 (10isarantopoulos) [13:15:22] 06Machine-Learning-Team (Q1 FY2026-27): Publish V0 LiftWing LLM documentation on Wikitech - https://phabricator.wikimedia.org/T431847#12183084 (10kevinbazira) All changes suggested in T431847#12182917, have been done. The current documentation now shows: 1. [Available Models](https://wikitech.wikimedia.org/w/in... [13:20:45] 10Lift-Wing, 06Machine-Learning-Team, 06ServiceOps, 10ServiceOps-SharedInfra, 06MediaWiki-Core-Platform-Team (Radar): Investigate X-Wikimedia-Debug routing for api.wikimedia.org - https://phabricator.wikimedia.org/T428909#12183104 (10isarantopoulos) [13:21:03] 06Machine-Learning-Team, 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Event-Platform: `mediawiki.page_change.v1`: negative namespace_id schema validation errors - https://phabricator.wikimedia.org/T421237#12183117 (10isarantopoulos) [13:23:26] 06Machine-Learning-Team, 07Essential-Work: Improve logging on Liftwing - https://phabricator.wikimedia.org/T420327#12183147 (10isarantopoulos) [13:24:19] 06Machine-Learning-Team, 07Essential-Work: Fix logging on Revertrisk - https://phabricator.wikimedia.org/T409931#12183152 (10isarantopoulos) [13:26:36] 06Machine-Learning-Team: Optimize gpt-oss-safeguard-20b LiftWing deployment - https://phabricator.wikimedia.org/T418351#12183155 (10isarantopoulos) [13:30:21] 06Machine-Learning-Team (Q1 FY2026-27): [draft] Update kserve helm chart to include LLMInferenceService - https://phabricator.wikimedia.org/T433973 (10isarantopoulos) 03NEW [13:32:57] 06Machine-Learning-Team (Q1 FY2026-27): [draft] Q1 FY2026-27 LLM Platform Milestone 2 - https://phabricator.wikimedia.org/T433975 (10isarantopoulos) 03NEW [13:35:23] 06Machine-Learning-Team (Q1 FY2026-27): [draft] Update kserve helm chart to include LLMInferenceService - https://phabricator.wikimedia.org/T433973#12183244 (10isarantopoulos) [13:35:24] 06Machine-Learning-Team (Q1 FY2026-27): [draft] Q1 FY2026-27 LLM Platform Milestone 2 - https://phabricator.wikimedia.org/T433975#12183243 (10isarantopoulos) [13:35:25] 06Machine-Learning-Team (Q1 FY2026-27): qwen36: Support thinking mode on the OpenAI-compatible endpoints - https://phabricator.wikimedia.org/T433569#12183245 (10isarantopoulos) [13:35:27] 06Machine-Learning-Team (Q1 FY2026-27), 07Essential-Work, 13Patch-For-Review: Set ephemeral-storage requests/limits on ML predictor containers to prevent node disk eviction - https://phabricator.wikimedia.org/T431089#12183246 (10isarantopoulos) [13:36:46] 06Machine-Learning-Team (Q1 FY2026-27): [draft] explore vllm tool calling - https://phabricator.wikimedia.org/T433977 (10isarantopoulos) 03NEW [13:37:18] 06Machine-Learning-Team (Q1 FY2026-27): [draft] Q1 FY2026-27 LLM Platform Milestone 2 - https://phabricator.wikimedia.org/T433975#12183286 (10isarantopoulos) [13:37:19] 06Machine-Learning-Team (Q1 FY2026-27): [draft] explore vllm tool calling - https://phabricator.wikimedia.org/T433977#12183287 (10isarantopoulos) [13:39:39] 06Machine-Learning-Team (Q1 FY2026-27): [draft] Q1 FY2026-27 LLM Platform Milestone 2 - https://phabricator.wikimedia.org/T433975#12183311 (10isarantopoulos) [13:39:40] 06Machine-Learning-Team (Q1 FY2026-27), 06Data-Platform-SRE, 07Essential-Work: Support GPU Partition specification in deployments - https://phabricator.wikimedia.org/T424923#12183312 (10isarantopoulos) [13:48:43] 06Machine-Learning-Team (Q1 FY2026-27), 13Patch-For-Review: gpt-oss-safeguard-20b fails with 500 when output cannot be parsed as Harmony messages - https://phabricator.wikimedia.org/T432933#12183353 (10isarantopoulos) a:03BWojtowicz-WMF [14:26:53] 10Lift-Wing, 06Machine-Learning-Team (Q1 FY2026-27): Redeploy Liftwing pick up current mirror - https://phabricator.wikimedia.org/T429675#12183614 (10isarantopoulos) 05Open→03Resolved a:03isarantopoulos [15:42:37] 06Machine-Learning-Team (Q1 FY2026-27): Update Revise Tone pipelines to incorporate additional languages and update topic filters - https://phabricator.wikimedia.org/T433319#12184001 (10Sucheta-Salgaonkar-WMF) [17:26:48] 06Machine-Learning-Team (Q1 FY2026-27), 05Goal, 07OKR-Work: Audio quality evaluation for TTS v1: ASR round-trip benchmark and resulting fixes - https://phabricator.wikimedia.org/T433923#12184548 (10kevinbazira) We added fixes for mispronounciations detected in T433923#12182276. These included 13 pronunciatio... [17:28:09] 06Machine-Learning-Team, 10VisualEditor, 10VisualEditor Suggestion Mode, 06Editing-team (FY2026-27 Q1 sprint 3), 07OKR-Work: Announce availability of "experimental" suggestions - https://phabricator.wikimedia.org/T434006 (10ppelberg) 03NEW [18:26:04] (03PS1) 10Kevin Bazira: tts-section-generator: add the ASR round-trip evaluation instrument [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1321042 (https://phabricator.wikimedia.org/T433923) [18:41:22] (03CR) 10Kevin Bazira: [C:03+2] tts-section-generator: add the ASR round-trip evaluation instrument [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1321042 (https://phabricator.wikimedia.org/T433923) (owner: 10Kevin Bazira) [18:44:17] 06Machine-Learning-Team (Q1 FY2026-27), 05Goal, 07OKR-Work, 13Patch-For-Review: Audio quality evaluation for TTS v1: ASR round-trip benchmark and resulting fixes - https://phabricator.wikimedia.org/T433923#12184875 (10kevinbazira) [18:50:53] (03Merged) 10jenkins-bot: tts-section-generator: add the ASR round-trip evaluation instrument [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1321042 (https://phabricator.wikimedia.org/T433923) (owner: 10Kevin Bazira) [21:53:16] 06Machine-Learning-Team, 06Editing-team, 10VisualEditor, 10VisualEditor Suggestion Mode, 07Epic: [Epic] Develop a framework for invalidating edit suggestions - https://phabricator.wikimedia.org/T429426#12185405 (10ppelberg) [22:49:56] 06Machine-Learning-Team, 06Editing QA, 10VisualEditor, 10VisualEditor Suggestion Mode, and 3 others: Implement calls to action for experimental LLM-generated MoS suggestions - https://phabricator.wikimedia.org/T432666#12185525 (10ppelberg) [22:50:34] 06Machine-Learning-Team, 06Editing QA, 10VisualEditor, 10VisualEditor Suggestion Mode, and 3 others: Implement calls to action for experimental LLM-generated MoS suggestions - https://phabricator.wikimedia.org/T432666#12185526 (10ppelberg) I //think// everything is implemented and deployed and what's left... [22:57:24] 06Machine-Learning-Team, 10VisualEditor, 10VisualEditor Suggestion Mode, 06Editing-team (FY2026-27 Q1 sprint 3), and 2 others: LLM generated suggestions: attach the suggestion UUID to more events - https://phabricator.wikimedia.org/T430720#12185535 (10ppelberg) With @OKarakaya-WMF reviewing and, I think, a...