[05:56:11] 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26), 06collaboration-services, 10ServiceOps-Mediawiki, and 2 others: Configure LiftWing's openAPI specs into mediawiki-config - https://phabricator.wikimedia.org/T426081#11979918 (10ABran-WMF) >>! In T426081#11973429, @Clement_Goubert wrote: > [...] > ###Alte... [07:02:11] 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26): Deploy CoPE-B-A4B on LiftWing - https://phabricator.wikimedia.org/T427497#11979981 (10kevinbazira) [07:02:12] 06Machine-Learning-Team (Q4 FY2025-26), 10WE4.12 Content policy model evaluation, 05Goal: Q2 FY2025-26 Goal: Host a content policy evaluation model on LiftWing - https://phabricator.wikimedia.org/T418267#11979982 (10kevinbazira) [07:09:30] (03PS1) 10Kevin Bazira: policy-violation: add cope-b-a4b model-server [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1296941 (https://phabricator.wikimedia.org/T427497) [07:11:32] (03PS1) 10Kevin Bazira: policy-violation: containerize cope-b-a4b model-server [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1296942 (https://phabricator.wikimedia.org/T427497) [07:25:37] 06Machine-Learning-Team, 06Infrastructure-Foundations: Move all Machine Learning Docker images under the /ml prefix in the Docker Registry - https://phabricator.wikimedia.org/T428022 (10elukey) 03NEW p:05Triage→03Medium [07:25:59] 06Machine-Learning-Team, 06Infrastructure-Foundations: Move all Machine Learning Docker images under the /ml prefix in the Docker Registry - https://phabricator.wikimedia.org/T428022#11980049 (10elukey) [07:28:04] 06Machine-Learning-Team (Q4 FY2025-26): ml-lab: Update pytorch-rocm to a current ROCm-compatible release - https://phabricator.wikimedia.org/T428023 (10isarantopoulos) 03NEW [07:34:16] 06Machine-Learning-Team (Q4 FY2025-26), 06Infrastructure-Foundations: Move all Machine Learning Docker images under the /ml prefix in the Docker Registry - https://phabricator.wikimedia.org/T428022#11980081 (10isarantopoulos) [07:36:40] (03CR) 10Bartosz Wójtowicz: "LGTM! ❤️ Leaving just 2 small comments" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1296941 (https://phabricator.wikimedia.org/T427497) (owner: 10Kevin Bazira) [07:40:51] (03CR) 10Bartosz Wójtowicz: [C:03+1] "Looks great! Let's merge after those 2 are in:" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1296942 (https://phabricator.wikimedia.org/T427497) (owner: 10Kevin Bazira) [08:26:38] (03CR) 10Hashar: "recheck after having deployed https://gerrit.wikimedia.org/r/c/integration/config/+/1296559" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1295879 (https://phabricator.wikimedia.org/T427794) (owner: 10Ozge) [08:30:20] (03CR) 10CI reject: [V:04-1] Add editing-suggestions KServe model server [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1295879 (https://phabricator.wikimedia.org/T427794) (owner: 10Ozge) [08:30:22] (03CR) 10Nikerabbit: [C:04-1] Fix unbounded concurrency in collection article fetching (031 comment) [research/recommendation-api] - 10https://gerrit.wikimedia.org/r/1296651 (owner: 10Nik Gkountas) [08:36:05] (03PS2) 10Kevin Bazira: policy-violation: add cope-b-a4b model-server [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1296941 (https://phabricator.wikimedia.org/T427497) [08:37:35] (03CR) 10Kevin Bazira: "thanks for the review. comments fixed. : )" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1296941 (https://phabricator.wikimedia.org/T427497) (owner: 10Kevin Bazira) [08:41:33] (03CR) 10AikoChou: "you can check the console output for the failed test build here:" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1295879 (https://phabricator.wikimedia.org/T427794) (owner: 10Ozge) [08:42:09] (03CR) 10Nikerabbit: [C:04-1] "Could this have an impact for https://phabricator.wikimedia.org/T405004?" [research/recommendation-api] - 10https://gerrit.wikimedia.org/r/1296651 (owner: 10Nik Gkountas) [08:43:00] (03CR) 10AikoChou: "unresolved it" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1295879 (https://phabricator.wikimedia.org/T427794) (owner: 10Ozge) [08:44:26] 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26), 10Semantic Search, 07Essential-Work: Transparent DNS Routing for LiftWing Services (eqiad vs Multi-DC) - https://phabricator.wikimedia.org/T422253#11980367 (10isarantopoulos) My assumption is that since we operate on an active-active mode in both DCs a r... [08:45:04] 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26), 10Semantic Search, 06Traffic, 07Essential-Work: Transparent DNS Routing for LiftWing Services (eqiad vs Multi-DC) - https://phabricator.wikimedia.org/T422253#11980368 (10isarantopoulos) [08:46:21] 06Machine-Learning-Team (Q4 FY2025-26), 07OKR-Work, 13Patch-For-Review: Integrate Article Topic model with the new caching service - https://phabricator.wikimedia.org/T418493#11980371 (10BWojtowicz-WMF) **Status update** We have pivoted from standalone gRPC <-> HTTP adapter to using native gRPC endpoint usi... [08:48:48] 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26), 10Semantic Search, 06Traffic, 07Essential-Work: Transparent DNS Routing for LiftWing Services (eqiad vs Multi-DC) - https://phabricator.wikimedia.org/T422253#11980377 (10DPogorzelski-WMF) but in that case we would see a good amount of 404 since some mo... [08:49:28] (03CR) 10Bartosz Wójtowicz: [C:03+1] "Thank you for working on this so swiftly! <3" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1296941 (https://phabricator.wikimedia.org/T427497) (owner: 10Kevin Bazira) [08:51:18] (03PS7) 10Ozge: Add editing-suggestions KServe model server [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1295879 (https://phabricator.wikimedia.org/T427794) [08:53:42] (03PS8) 10Ozge: Add editing-suggestions KServe model server [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1295879 (https://phabricator.wikimedia.org/T427794) [08:53:59] (03CR) 10Kevin Bazira: [C:03+2] policy-violation: add cope-b-a4b model-server (032 comments) [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1296941 (https://phabricator.wikimedia.org/T427497) (owner: 10Kevin Bazira) [08:54:39] (03CR) 10AikoChou: [C:03+1] "LGTM! we can merge it once we solve the test build issue :)" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1295879 (https://phabricator.wikimedia.org/T427794) (owner: 10Ozge) [08:55:27] (03Merged) 10jenkins-bot: policy-violation: add cope-b-a4b model-server [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1296941 (https://phabricator.wikimedia.org/T427497) (owner: 10Kevin Bazira) [08:59:49] 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26), 10Semantic Search, 06Traffic, 07Essential-Work: Transparent DNS Routing for LiftWing Services (eqiad vs Multi-DC) - https://phabricator.wikimedia.org/T422253#11980405 (10isarantopoulos) You are right, but given that we don't currently have production l... [09:02:31] (03CR) 10Ozge: "Done" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1295879 (https://phabricator.wikimedia.org/T427794) (owner: 10Ozge) [09:02:43] (03CR) 10Ozge: [C:03+2] Add editing-suggestions KServe model server [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1295879 (https://phabricator.wikimedia.org/T427794) (owner: 10Ozge) [09:08:02] (03PS1) 10Gkyziridis: feat(liftwing-openapi-server): Add static OpenAPI spec server with CORS support [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1297072 (https://phabricator.wikimedia.org/T427902) [09:10:54] (03Merged) 10jenkins-bot: Add editing-suggestions KServe model server [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1295879 (https://phabricator.wikimedia.org/T427794) (owner: 10Ozge) [09:12:11] (03CR) 10CI reject: [V:04-1] feat(liftwing-openapi-server): Add static OpenAPI spec server with CORS support [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1297072 (https://phabricator.wikimedia.org/T427902) (owner: 10Gkyziridis) [09:13:13] 06Machine-Learning-Team (Q4 FY2025-26), 06Infrastructure-Foundations: Move all Machine Learning Docker images under the /ml prefix in the Docker Registry - https://phabricator.wikimedia.org/T428022#11980477 (10elukey) Current ML images: ` amd-gpu-tester amd-pytorch-common amd-pytorch21 amd-pytorch22 amd-pytor... [09:14:18] (03PS2) 10Gkyziridis: feat(liftwing-openapi-server): Add static OpenAPI spec server with CORS support [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1297072 (https://phabricator.wikimedia.org/T427902) [09:14:26] (03CR) 10CI reject: [V:04-1] feat(liftwing-openapi-server): Add static OpenAPI spec server with CORS support [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1297072 (https://phabricator.wikimedia.org/T427902) (owner: 10Gkyziridis) [09:15:31] (03CR) 10Gkyziridis: "recheck" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1297072 (https://phabricator.wikimedia.org/T427902) (owner: 10Gkyziridis) [09:20:06] (03PS3) 10Gkyziridis: feat(liftwing-openapi-server): Add static OpenAPI spec server with CORS support [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1297072 (https://phabricator.wikimedia.org/T427902) [09:20:58] 06Machine-Learning-Team (Q4 FY2025-26), 06ServiceOps new, 06Traffic: k8s changes needed to allow article topic (and other future isvcs) to use the kserve v2 inference protocol (and gRPC) - https://phabricator.wikimedia.org/T424049#11980492 (10elukey) @isarantopoulos o/ before closing this task let's make sur... [09:24:29] 06Machine-Learning-Team (Q4 FY2025-26), 06Infrastructure-Foundations: Move all Machine Learning Docker images under the /ml prefix in the Docker Registry - https://phabricator.wikimedia.org/T428022#11980522 (10elukey) From this quick check it seems that amd-pytorch21 and 22 are not used: ` ~/Wikimedia/inferen... [09:29:05] 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26), 06ServiceOps new, 10ServiceOps-SharedInfra, 13Patch-For-Review: Expose LiftWing API for serving the openapi-specs through the /docs yaml files. - https://phabricator.wikimedia.org/T427902#11980535 (10gkyziridis) We decided to go with the option 1, havi... [09:54:03] (03CR) 10Hashar: "recheck after having having deployed I7ec8d1a83cb9e07b76eea3d3d83eb674a839934b *CI stage for cope-b-a4b model*." [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1296942 (https://phabricator.wikimedia.org/T427497) (owner: 10Kevin Bazira) [09:55:43] 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26), 06ServiceOps new, 10ServiceOps-SharedInfra, 13Patch-For-Review: Expose LiftWing API for serving the openapi-specs through the /docs yaml files. - https://phabricator.wikimedia.org/T427902#11980649 (10Clement_Goubert) You can probably use the [[ https:/... [09:56:47] (03CR) 10Hashar: "recheck after having deployed I0f93a5e9d3a03cfeb41e2ce89131f39e7cee42a1 *Add liftwing-openapi-server CI/CD pipelines*." [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1297072 (https://phabricator.wikimedia.org/T427902) (owner: 10Gkyziridis) [10:01:59] (03CR) 10CI reject: [V:04-1] policy-violation: containerize cope-b-a4b model-server [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1296942 (https://phabricator.wikimedia.org/T427497) (owner: 10Kevin Bazira) [10:06:03] (03CR) 10Gkyziridis: "recheck" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1297072 (https://phabricator.wikimedia.org/T427902) (owner: 10Gkyziridis) [10:06:05] (03CR) 10CI reject: [V:04-1] feat(liftwing-openapi-server): Add static OpenAPI spec server with CORS support [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1297072 (https://phabricator.wikimedia.org/T427902) (owner: 10Gkyziridis) [10:08:31] (03CR) 10Hashar: "I don't know what is the standard, but maybe you can use Apache2 instead of a custom Python based one 😊" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1297072 (https://phabricator.wikimedia.org/T427902) (owner: 10Gkyziridis) [10:08:38] (03PS4) 10Gkyziridis: feat(liftwing-openapi-server): Add static OpenAPI spec server with CORS support [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1297072 (https://phabricator.wikimedia.org/T427902) [10:11:45] (03CR) 10Kevin Bazira: "looks like a CI job for a `liftwing-openapi-server` was added in I0f93a5e9d3a03cfeb41e2ce89131f39e7cee42a1, yet server files and configs a" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1296942 (https://phabricator.wikimedia.org/T427497) (owner: 10Kevin Bazira) [10:16:15] (03CR) 10Clément Goubert: "Some pointers on a more standard way to expose an HTTP server" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1297072 (https://phabricator.wikimedia.org/T427902) (owner: 10Gkyziridis) [10:20:18] 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26), 06ServiceOps new, 10ServiceOps-SharedInfra, 13Patch-For-Review: Expose LiftWing API for serving the openapi-specs through the /docs yaml files. - https://phabricator.wikimedia.org/T427902#11980743 (10isarantopoulos) >>! In T427902#11980649, @Clement_Go... [10:20:24] 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26), 10Semantic Search, 06Traffic, 07Essential-Work: Transparent DNS Routing for LiftWing Services (eqiad vs Multi-DC) - https://phabricator.wikimedia.org/T422253#11980745 (10DPogorzelski-WMF) we are already in an active/active setup so it would be enough t... [10:25:45] 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26), 06ServiceOps new, 10ServiceOps-SharedInfra, 13Patch-For-Review: Expose LiftWing API for serving the openapi-specs through the /docs yaml files. - https://phabricator.wikimedia.org/T427902#11980757 (10Clement_Goubert) >>! In T427902#11980743, @isarantop... [10:27:40] 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26), 06ServiceOps new, 10ServiceOps-SharedInfra, 13Patch-For-Review: Expose LiftWing API for serving the openapi-specs through the /docs yaml files. - https://phabricator.wikimedia.org/T427902#11980768 (10isarantopoulos) This sounds even better (and simpler)! [10:39:42] (03PS1) 10Nik Gkountas: Lazy recommender construction to fix OOM on collection cache load [research/recommendation-api] - 10https://gerrit.wikimedia.org/r/1297097 [10:41:46] (03CR) 10CI reject: [V:04-1] Lazy recommender construction to fix OOM on collection cache load [research/recommendation-api] - 10https://gerrit.wikimedia.org/r/1297097 (owner: 10Nik Gkountas) [10:52:17] (03PS2) 10Kevin Bazira: policy-violation: containerize cope-b-a4b model-server [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1296942 (https://phabricator.wikimedia.org/T427497) [10:58:35] 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26), 06ServiceOps new, 10ServiceOps-SharedInfra, 13Patch-For-Review: Expose LiftWing API for serving the openapi-specs through the /docs yaml files. - https://phabricator.wikimedia.org/T427902#11980868 (10gkyziridis) Thank you both @isarantopoulos and @Clem... [11:00:03] (03PS3) 10Kevin Bazira: policy-violation: containerize cope-b-a4b model-server [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1296942 (https://phabricator.wikimedia.org/T427497) [11:08:23] (03PS4) 10Kevin Bazira: policy-violation: containerize cope-b-a4b model-server [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1296942 (https://phabricator.wikimedia.org/T427497) [11:08:28] (03CR) 10CI reject: [V:04-1] policy-violation: containerize cope-b-a4b model-server [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1296942 (https://phabricator.wikimedia.org/T427497) (owner: 10Kevin Bazira) [11:13:34] 06Machine-Learning-Team (Q4 FY2025-26), 05Goal, 07OKR-Work: Q1 FY2025-26 Goal: Make article topic data available at scale and within SLOs for Year in Review - https://phabricator.wikimedia.org/T392833#11980912 (10achou) Hi @Dbrant! I'd like to follow up on how Apps accesses article topics — specifically one... [11:17:00] (03CR) 10Kevin Bazira: "fixed in: https://gerrit.wikimedia.org/r/c/machinelearning/liftwing/inference-services/+/1296942/4" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1296942 (https://phabricator.wikimedia.org/T427497) (owner: 10Kevin Bazira) [11:19:44] (03CR) 10Bartosz Wójtowicz: [C:03+1] policy-violation: containerize cope-b-a4b model-server [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1296942 (https://phabricator.wikimedia.org/T427497) (owner: 10Kevin Bazira) [11:21:53] (03CR) 10Kevin Bazira: [C:03+2] policy-violation: containerize cope-b-a4b model-server [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1296942 (https://phabricator.wikimedia.org/T427497) (owner: 10Kevin Bazira) [11:29:44] (03Merged) 10jenkins-bot: policy-violation: containerize cope-b-a4b model-server [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1296942 (https://phabricator.wikimedia.org/T427497) (owner: 10Kevin Bazira) [11:40:37] 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26), 06ServiceOps new, 10ServiceOps-SharedInfra, 13Patch-For-Review: Expose LiftWing API for serving the openapi-specs through the /docs yaml files. - https://phabricator.wikimedia.org/T427902#11981010 (10Clement_Goubert) No, no need. The goal is to have th... [11:55:57] 06Machine-Learning-Team, 10MediaWiki-extensions-ORES, 10MediaWiki-Recent-changes, 06Moderator-Tools-Team, and 2 others: Roll out Revert Risk Filters to Wikis that don't have damaging/goodfaith Edit Models - https://phabricator.wikimedia.org/T408388#11981054 (10Samwalton9-WMF) [12:14:45] 06Machine-Learning-Team (Q4 FY2025-26), 13Patch-For-Review: Editing Suggestions - api - https://phabricator.wikimedia.org/T427794#11981120 (10OKarakaya-WMF) deployment patch: https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1297106 [12:38:21] 06Machine-Learning-Team, 10Liberica, 10Prod-Kubernetes, 06Traffic, and 2 others: Migrate ML k8s apiserver and services to IPIP - https://phabricator.wikimedia.org/T420438#11981196 (10elukey) @isarantopoulos not incredibly urgent but Service Ops is asking teams to complete it before the end of Q1 (earlier i... [12:41:28] 06Machine-Learning-Team (Q4 FY2025-26), 06ServiceOps new, 06Traffic: k8s changes needed to allow article topic (and other future isvcs) to use the kserve v2 inference protocol (and gRPC) - https://phabricator.wikimedia.org/T424049#11981211 (10isarantopoulos) ack, will make sure to do that! [12:43:14] (03PS5) 10Gkyziridis: feat(liftwing-openapi-server): Serve OpenAPI specs via Apache httpd [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1297072 (https://phabricator.wikimedia.org/T427902) [12:43:22] (03CR) 10CI reject: [V:04-1] feat(liftwing-openapi-server): Serve OpenAPI specs via Apache httpd [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1297072 (https://phabricator.wikimedia.org/T427902) (owner: 10Gkyziridis) [12:46:29] (03PS6) 10Gkyziridis: feat(liftwing-openapi-server): Serve OpenAPI specs via Apache httpd [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1297072 (https://phabricator.wikimedia.org/T427902) [12:49:15] (03PS7) 10Gkyziridis: feat(liftwing-openapi-server): Serve OpenAPI specs via Apache httpd [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1297072 (https://phabricator.wikimedia.org/T427902) [12:52:52] (03PS8) 10Gkyziridis: feat(liftwing-openapi-server): Serve OpenAPI specs via Apache httpd [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1297072 (https://phabricator.wikimedia.org/T427902) [12:57:17] (03CR) 10CI reject: [V:04-1] feat(liftwing-openapi-server): Serve OpenAPI specs via Apache httpd [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1297072 (https://phabricator.wikimedia.org/T427902) (owner: 10Gkyziridis) [12:57:47] 06Machine-Learning-Team, 10Liberica, 10Prod-Kubernetes, 06Traffic, and 2 others: Migrate ML k8s apiserver and services to IPIP - https://phabricator.wikimedia.org/T420438#11981252 (10isarantopoulos) @elukey Thanks for the ping! We'll make sure to do it and we'll reach out in case we have any questions. [12:59:07] (03CR) 10Gkyziridis: "recheck" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1297072 (https://phabricator.wikimedia.org/T427902) (owner: 10Gkyziridis) [12:59:36] (03CR) 10Sbisson: "This fix makes a lot of sense to me. The sudden spike in OOM is still mysterious but there is no doubt that loading all the collection dat" [research/recommendation-api] - 10https://gerrit.wikimedia.org/r/1297097 (owner: 10Nik Gkountas) [13:19:19] 06Machine-Learning-Team (Q4 FY2025-26), 05Goal, 07OKR-Work: Q1 FY2025-26 Goal: Make article topic data available at scale and within SLOs for Year in Review - https://phabricator.wikimedia.org/T392833#11981294 (10Dbrant) >>! In T392833#11980912, @achou wrote: > Hi @Dbrant! I'd like to follow up on how Apps a... [13:19:30] 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26), 10Semantic Search, 06Traffic, 07Essential-Work: Transparent DNS Routing for LiftWing Services (eqiad vs Multi-DC) - https://phabricator.wikimedia.org/T422253#11981295 (10elukey) @DPogorzelski-WMF the discovery records resolve to the IP of eqiad or codf... [14:13:19] (03PS9) 10Gkyziridis: feat(liftwing-openapi-server): Serve OpenAPI specs via Apache httpd [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1297072 (https://phabricator.wikimedia.org/T427902) [14:22:58] (03PS10) 10Gkyziridis: feat(liftwing-openapi-server): Serve OpenAPI specs via Apache httpd [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1297072 (https://phabricator.wikimedia.org/T427902) [14:47:28] (03PS1) 10Kevin Bazira: policy-violation: fix cope-b-a4b transformers version via PYTHONPATH order [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1297166 (https://phabricator.wikimedia.org/T427497) [15:00:15] (03CR) 10Bartosz Wójtowicz: [C:03+1] policy-violation: fix cope-b-a4b transformers version via PYTHONPATH order [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1297166 (https://phabricator.wikimedia.org/T427497) (owner: 10Kevin Bazira) [15:07:31] (03CR) 10Kevin Bazira: [C:03+2] policy-violation: fix cope-b-a4b transformers version via PYTHONPATH order [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1297166 (https://phabricator.wikimedia.org/T427497) (owner: 10Kevin Bazira) [15:09:09] (03Merged) 10jenkins-bot: policy-violation: fix cope-b-a4b transformers version via PYTHONPATH order [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1297166 (https://phabricator.wikimedia.org/T427497) (owner: 10Kevin Bazira)