[09:30:44] FIRING: LiftWingServiceErrorRate: ... [09:30:44] LiftWing service has a high rate of non 2/3/400 error code responses - https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing/Alerts#LiftWingServiceErrorRate - https://grafana.wikimedia.org/d/G7yj84Vnk/istio?orgId=1&refresh=30s&var-cluster=eqiad%20prometheus/k8s-mlserve&var-namespace=revscoring-editquality-goodfaith&var-backend=ruwiki-goodfaith-predictor.%2A - https://alerts.wikimedia.org/?q=alertname%3DLiftWingServiceErrorRate [09:33:33] (03PS1) 10Bartosz Wójtowicz: outlink-topic-model: Retry transient MW API failures. [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1300060 (https://phabricator.wikimedia.org/T428127) [09:40:44] RESOLVED: LiftWingServiceErrorRate: ... [09:40:44] LiftWing service has a high rate of non 2/3/400 error code responses - https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing/Alerts#LiftWingServiceErrorRate - https://grafana.wikimedia.org/d/G7yj84Vnk/istio?orgId=1&refresh=30s&var-cluster=eqiad%20prometheus/k8s-mlserve&var-namespace=revscoring-editquality-goodfaith&var-backend=ruwiki-goodfaith-predictor.%2A - https://alerts.wikimedia.org/?q=alertname%3DLiftWingServiceErrorRate [10:09:37] (03Abandoned) 10Hashar: Bumps revscoring to 2.2.5 [research/ores/wheels] - 10https://gerrit.wikimedia.org/r/439983 (owner: 10Halfak) [10:09:40] (03Abandoned) 10Hashar: Bumps mwparserfromhell to 0.5.2 [research/ores/wheels] - 10https://gerrit.wikimedia.org/r/477310 (https://phabricator.wikimedia.org/T206654) (owner: 10Halfak) [10:25:53] 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26), 06ServiceOps new, 10ServiceOps-SharedInfra, 13Patch-For-Review: Expose LiftWing API for serving the openapi-specs through the /docs yaml files. - https://phabricator.wikimedia.org/T427902#12003987 (10Clement_Goubert) I've changed the policy so that we... [10:28:04] (03CR) 10Gkyziridis: [C:03+1] "It looks good with these tags. The grouping seems to me reasonable." [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1299635 (https://phabricator.wikimedia.org/T419455) (owner: 10Alex Paskulin) [10:28:27] (03CR) 10Gkyziridis: [C:03+2] docs: Add OpenAPI docs for Revscoring endpoints [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1293768 (https://phabricator.wikimedia.org/T419455) (owner: 10Alex Paskulin) [10:29:55] (03Merged) 10jenkins-bot: docs: Add OpenAPI docs for Revscoring endpoints [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1293768 (https://phabricator.wikimedia.org/T419455) (owner: 10Alex Paskulin) [10:30:56] (03CR) 10Gkyziridis: [C:03+2] docs: Organize endpoints into tags [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1299635 (https://phabricator.wikimedia.org/T419455) (owner: 10Alex Paskulin) [10:32:20] (03Merged) 10jenkins-bot: docs: Organize endpoints into tags [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1299635 (https://phabricator.wikimedia.org/T419455) (owner: 10Alex Paskulin) [11:12:58] (03PS1) 10Nik Gkountas: Lazy, request-scoped page-collection loading to cut OOM pressure [research/recommendation-api] - 10https://gerrit.wikimedia.org/r/1300097 [11:14:36] (03CR) 10CI reject: [V:04-1] Lazy, request-scoped page-collection loading to cut OOM pressure [research/recommendation-api] - 10https://gerrit.wikimedia.org/r/1300097 (owner: 10Nik Gkountas) [11:41:57] 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26), 06ServiceOps new, 10ServiceOps-SharedInfra, 13Patch-For-Review: Expose LiftWing API for serving the openapi-specs through the /docs yaml files. - https://phabricator.wikimedia.org/T427902#12004298 (10gkyziridis) The latest version of liftwing-openapi-s... [11:42:23] 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26), 06ServiceOps new, 10ServiceOps-SharedInfra, 13Patch-For-Review: Expose LiftWing API for serving the openapi-specs through the /docs yaml files. - https://phabricator.wikimedia.org/T427902#12004302 (10gkyziridis) [12:05:30] 06Machine-Learning-Team (Q4 FY2025-26): Editing Suggestions api changes. - https://phabricator.wikimedia.org/T428740 (10OKarakaya-WMF) 03NEW [12:10:04] (03PS2) 10Bartosz Wójtowicz: outlink-topic-model: Retry transient MW API failures. [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1300060 (https://phabricator.wikimedia.org/T428127) [12:10:17] (03PS1) 10Gkyziridis: docs(liftwing-openapi-server): Document architecture, local testing, and deployment [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1300108 (https://phabricator.wikimedia.org/T427902) [12:15:24] 06Machine-Learning-Team (Q4 FY2025-26): LLM Generated Editing Suggestions - API changes. - https://phabricator.wikimedia.org/T428740#12004469 (10OKarakaya-WMF) [12:19:27] (03CR) 10Kevin Bazira: [C:03+1] docs(liftwing-openapi-server): Document architecture, local testing, and deployment [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1300108 (https://phabricator.wikimedia.org/T427902) (owner: 10Gkyziridis) [12:43:00] (03CR) 10Gkyziridis: [C:03+2] docs(liftwing-openapi-server): Document architecture, local testing, and deployment [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1300108 (https://phabricator.wikimedia.org/T427902) (owner: 10Gkyziridis) [12:44:24] (03Merged) 10jenkins-bot: docs(liftwing-openapi-server): Document architecture, local testing, and deployment [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1300108 (https://phabricator.wikimedia.org/T427902) (owner: 10Gkyziridis) [12:56:35] (03PS2) 10Nik Gkountas: Lazy, request-scoped page-collection loading to cut OOM pressure [research/recommendation-api] - 10https://gerrit.wikimedia.org/r/1300097 [13:11:13] 06Machine-Learning-Team, 06Editing-team, 10VisualEditor Suggestion Mode, 13Patch-For-Review: Deploy LLM-generated MoS suggestions as experimental - https://phabricator.wikimedia.org/T428311#12004621 (10OKarakaya-WMF) Hey @ppelberg , thank you for creating this task. Could you elaborate more on how we targe... [13:44:27] 06Machine-Learning-Team (Q4 FY2025-26): LLM Generated Editing Suggestions - API changes. - https://phabricator.wikimedia.org/T428740#12004754 (10OKarakaya-WMF) Initial static descriptions. ` edit_types_dict = { "NPOV": { "static_description": "NPOV", "title": "NPOV", }, "simplify_lan... [14:42:47] 06Machine-Learning-Team, 06Editing-team, 10VisualEditor Suggestion Mode, 13Patch-For-Review: Deploy LLM-generated MoS suggestions as experimental - https://phabricator.wikimedia.org/T428311#12005070 (10OKarakaya-WMF) Hey @ppelberg , I have a question around how we plan to run this experiment and how the... [14:58:28] (03CR) 10Eamedina: [C:03+1] Lazy, request-scoped page-collection loading to cut OOM pressure [research/recommendation-api] - 10https://gerrit.wikimedia.org/r/1300097 (owner: 10Nik Gkountas) [15:01:04] 06Machine-Learning-Team (Q4 FY2025-26): LLM Generated Editing Suggestions - API changes. - https://phabricator.wikimedia.org/T428740#12005187 (10DLynch) `MOS:GEO` is an example of something where we should almost-certainly include a link. You'll be able to write wikitext in the descriptions and have it work, tho... [15:34:25] 06Machine-Learning-Team (Q4 FY2025-26): LLM Generated Editing Suggestions - API changes. - https://phabricator.wikimedia.org/T428740#12005402 (10OKarakaya-WMF) Indeed, it's en specific. On the other hand, the Manual of Style guides are generally wiki specific as well. Afaik there is no 1:1 matching on MOS betwe... [16:29:01] 06Machine-Learning-Team (Q4 FY2025-26): LLM Generated Editing Suggestions - API changes. - https://phabricator.wikimedia.org/T428740#12005661 (10DLynch) [16:29:06] 06Machine-Learning-Team, 10VisualEditor Suggestion Mode, 06Editing-team (Editing-current-Q4-8Jun-19Jun-2026), 13Patch-For-Review: Build PoC for integrating initial set of MoS LLM-generated suggestions into Suggestion Mode as `experimental` - https://phabricator.wikimedia.org/T428641#12005662 (10DLynch) [16:58:05] (03Abandoned) 10Nik Gkountas: Lazy recommender construction to fix OOM on collection cache load [research/recommendation-api] - 10https://gerrit.wikimedia.org/r/1297097 (owner: 10Nik Gkountas) [17:57:48] 06Machine-Learning-Team, 06Editing-team, 10VisualEditor Suggestion Mode, 13Patch-For-Review: Deploy LLM-generated MoS suggestions as experimental - https://phabricator.wikimedia.org/T428311#12006228 (10Wellverywell) So does option 1 is a part of Suggestion Mode and option 2 a part of something like newcome... [18:40:33] (03PS1) 10Alex Paskulin: docs: Add translation endpoint and fix tags [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1300229 (https://phabricator.wikimedia.org/T419455) [19:00:35] (03CR) 10Alex Paskulin: docs: Add translation endpoint and fix tags (032 comments) [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1300229 (https://phabricator.wikimedia.org/T419455) (owner: 10Alex Paskulin) [19:48:06] (03CR) 10Sbisson: [C:03+2] Lazy, request-scoped page-collection loading to cut OOM pressure [research/recommendation-api] - 10https://gerrit.wikimedia.org/r/1300097 (owner: 10Nik Gkountas) [19:50:21] (03Merged) 10jenkins-bot: Lazy, request-scoped page-collection loading to cut OOM pressure [research/recommendation-api] - 10https://gerrit.wikimedia.org/r/1300097 (owner: 10Nik Gkountas) [20:56:25] 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26), 06ServiceOps new, 10ServiceOps-SharedInfra, 13Patch-For-Review: Expose LiftWing API for serving the openapi-specs through the /docs yaml files. - https://phabricator.wikimedia.org/T427902#12007115 (10apaskulin) @Clement_Goubert We tried to deploy the n... [20:59:45] 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26), 06ServiceOps new, 10ServiceOps-SharedInfra, 13Patch-For-Review: Expose LiftWing API for serving the openapi-specs through the /docs yaml files. - https://phabricator.wikimedia.org/T427902#12007135 (10Catrope) (For completeness, the above error was at h...