[08:51:03] 06Machine-Learning-Team (Q1 FY2026-27), 13Patch-For-Review: Serve jina-embeddings-v5-text-nano on Lift Wing - https://phabricator.wikimedia.org/T432717#12177889 (10OKarakaya-WMF) [08:51:41] 06Machine-Learning-Team (Q1 FY2026-27), 05Goal, 07OKR-Work: Fix text normalization edge cases before the TTS v1 batch run (ruleset 2026.08.03) - https://phabricator.wikimedia.org/T433820 (10kevinbazira) 03NEW [09:33:27] (03PS1) 10Kevin Bazira: tts-section-generator: fix misread symbols, abbreviations, and roman numerals (ruleset 2026.08.03) [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1320108 (https://phabricator.wikimedia.org/T433820) [09:43:15] (03CR) 10Kevin Bazira: [C:03+2] tts-section-generator: fix misread symbols, abbreviations, and roman numerals (ruleset 2026.08.03) [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1320108 (https://phabricator.wikimedia.org/T433820) (owner: 10Kevin Bazira) [09:53:29] (03Merged) 10jenkins-bot: tts-section-generator: fix misread symbols, abbreviations, and roman numerals (ruleset 2026.08.03) [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1320108 (https://phabricator.wikimedia.org/T433820) (owner: 10Kevin Bazira) [10:08:43] 06Machine-Learning-Team (Q1 FY2026-27), 13Patch-For-Review: Serve jina-embeddings-v5-text-nano on Lift Wing - https://phabricator.wikimedia.org/T432717#12178157 (10OKarakaya-WMF) works well in the grafana dashboard: https://grafana-rw.wikimedia.org/goto/dfu227cq03wn4e?orgId=default [10:24:19] 06Machine-Learning-Team (Q1 FY2026-27), 13Patch-For-Review: Serve jina-embeddings-v5-text-nano on Lift Wing - https://phabricator.wikimedia.org/T432717#12178192 (10OKarakaya-WMF) I'm closing this task as we have deployed Jina Nano embeddings to liftwing successfully. After some performance improvements, we ma... [10:31:22] 06Machine-Learning-Team (Q1 FY2026-27), 13Patch-For-Review: Serve jina-embeddings-v5-text-nano on Lift Wing - https://phabricator.wikimedia.org/T432717#12178209 (10OKarakaya-WMF) 05Open→03Resolved [11:57:41] 06Machine-Learning-Team (Q1 FY2026-27), 05Goal, 07OKR-Work: Fix text normalization edge cases before the TTS v1 batch run (ruleset 2026.08.03) - https://phabricator.wikimedia.org/T433820#12178461 (10kevinbazira) We tested the text normalization fixes we added to the tts-section-generator service. Below are r... [12:00:45] 06Machine-Learning-Team (Q1 FY2026-27), 05Goal, 07OKR-Work: Fix text normalization edge cases before the TTS v1 batch run (ruleset 2026.08.03) - https://phabricator.wikimedia.org/T433820#12178468 (10kevinbazira) Standalone extraction-only verification of the `{{Coord}}` hidden-duplicate strip. This tests `ex... [12:48:59] 06Machine-Learning-Team (Q1 FY2026-27), 05Goal, 07OKR-Work: Q1 FY2026-27 Deploy a batch of LLM-generated MoS suggestions in Suggestion Mode - https://phabricator.wikimedia.org/T433833 (10Sucheta-Salgaonkar-WMF) 03NEW [13:02:33] (03PS1) 10Kevin Bazira: tts-section-generator: corpus rescan under ruleset 2026.08.03 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1320155 (https://phabricator.wikimedia.org/T433820) [13:11:20] 06Machine-Learning-Team (Q1 FY2026-27), 05Goal, 07OKR-Work: Q1 FY2026-27 Deploy a batch of LLM-generated MoS suggestions in Suggestion Mode - https://phabricator.wikimedia.org/T433833#12178694 (10Sucheta-Salgaonkar-WMF) [13:14:04] (03CR) 10Kevin Bazira: [C:03+2] tts-section-generator: corpus rescan under ruleset 2026.08.03 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1320155 (https://phabricator.wikimedia.org/T433820) (owner: 10Kevin Bazira) [13:14:57] 06Machine-Learning-Team (Q1 FY2026-27), 05Goal, 07OKR-Work, 13Patch-For-Review: Fix text normalization edge cases before the TTS v1 batch run (ruleset 2026.08.03) - https://phabricator.wikimedia.org/T433820#12178725 (10kevinbazira) 05Open→03Resolved [13:16:12] (03Merged) 10jenkins-bot: tts-section-generator: corpus rescan under ruleset 2026.08.03 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1320155 (https://phabricator.wikimedia.org/T433820) (owner: 10Kevin Bazira) [15:01:35] 06Machine-Learning-Team (Q1 FY2026-27): Update Revise Tone pipelines to incorporate additional languages and update topic filters - https://phabricator.wikimedia.org/T433319#12179220 (10Sucheta-Salgaonkar-WMF) [15:02:14] 06Machine-Learning-Team (Q1 FY2026-27): Update Revise Tone pipelines to incorporate additional languages and update topic filters - https://phabricator.wikimedia.org/T433319#12179222 (10Sucheta-Salgaonkar-WMF) [16:02:19] 06Machine-Learning-Team (Q1 FY2026-27), 06SRE, 10SRE-swift-storage, 05Goal, 07OKR-Work: Provision and wire object storage for TTS v1 audio artifacts: S3 sink, Swift bucket, and credentials - https://phabricator.wikimedia.org/T432944#12179609 (10kevinbazira) @MatthewVernon following up on the storage buck... [17:03:49] FIRING: KubernetesDeploymentUnavailableReplicas: ... [17:03:49] Deployment revertrisk-multilingual-predictor-00001-deployment in revertrisk at eqiad has persistently unavailable replicas - https://wikitech.wikimedia.org/wiki/Kubernetes/Troubleshooting#Troubleshooting_a_deployment - ... [17:03:49] https://grafana.wikimedia.org/d/a260da06-259a-4ee4-9540-5cab01a246c8/kubernetes-deployment-details?var-site=eqiad&var-cluster=k8s-mlserve&var-namespace=revertrisk&var-deployment=revertrisk-multilingual-predictor-00001-deployment - https://alerts.wikimedia.org/?q=alertname%3DKubernetesDeploymentUnavailableReplicas [17:33:49] RESOLVED: KubernetesDeploymentUnavailableReplicas: ... [17:33:49] Deployment revertrisk-multilingual-predictor-00001-deployment in revertrisk at eqiad has persistently unavailable replicas - https://wikitech.wikimedia.org/wiki/Kubernetes/Troubleshooting#Troubleshooting_a_deployment - ... [17:33:49] https://grafana.wikimedia.org/d/a260da06-259a-4ee4-9540-5cab01a246c8/kubernetes-deployment-details?var-site=eqiad&var-cluster=k8s-mlserve&var-namespace=revertrisk&var-deployment=revertrisk-multilingual-predictor-00001-deployment - https://alerts.wikimedia.org/?q=alertname%3DKubernetesDeploymentUnavailableReplicas [17:40:20] 06Machine-Learning-Team, 10VisualEditor, 10VisualEditor Suggestion Mode, 06Editing-team (FY2026-27 Q1 sprint 2), and 2 others: Implement calls to action for experimental LLM-generated MoS suggestions - https://phabricator.wikimedia.org/T432666#12180262 (10DLynch) {F97042604} {F97042632} A fake suggestion... [19:38:33] 06Machine-Learning-Team, 06Editing QA, 10VisualEditor, 10VisualEditor Suggestion Mode, and 2 others: Implement calls to action for experimental LLM-generated MoS suggestions - https://phabricator.wikimedia.org/T432666#12180501 (10DLynch) The useful local testing trick is that after the line `.then( ( respo... [22:16:45] 06Machine-Learning-Team, 06Editing-team, 10VisualEditor, 10VisualEditor Suggestion Mode, 07Epic: [Epic] Develop a framework for invalidating edit suggestions - https://phabricator.wikimedia.org/T429426#12181096 (10ppelberg) >>! In T429426#12169147, @Isaac wrote: > To the open questions, I'd also add one... [22:17:23] 06Machine-Learning-Team, 06Editing-team, 10VisualEditor, 10VisualEditor Suggestion Mode, 07Epic: [Epic] Develop a framework for invalidating edit suggestions - https://phabricator.wikimedia.org/T429426#12181098 (10ppelberg) [23:48:36] 06Machine-Learning-Team, 06Growth-Team, 10Revise-Tone-Structured-Task: Revise Tone: batch create suggestions for "forgotten articles" - https://phabricator.wikimedia.org/T433912 (10KStoller-WMF) 03NEW