[07:10:58] Iluvatar: thanks a lot for all the feedback! [07:19:47] I am going to investigate and create tasks [07:20:00] do you have a phabricator user? Do you want to get CCed? [07:23:40] 10Machine-Learning-Team, 10Epic: Index out of range in revert risk multi-lingual - https://phabricator.wikimedia.org/T340811 (10elukey) [07:26:25] 10Machine-Learning-Team, 10Epic: FeatureNotFound exception in revertrisk multi-lingual - https://phabricator.wikimedia.org/T340812 (10elukey) [07:26:53] 10Machine-Learning-Team, 10Epic: FeatureNotFound exception in revertrisk multi-lingual - https://phabricator.wikimedia.org/T340812 (10elukey) [07:28:30] 10Machine-Learning-Team: Index out of range in revert risk multi-lingual - https://phabricator.wikimedia.org/T340811 (10elukey) [07:28:39] 10Machine-Learning-Team: FeatureNotFound exception in revertrisk multi-lingual - https://phabricator.wikimedia.org/T340812 (10elukey) [07:29:49] elukey: I subscribed to these tasks, thanks! [07:31:30] 10Machine-Learning-Team: Pop index out of range exception in revertrisk multi lingual - https://phabricator.wikimedia.org/T340813 (10elukey) [07:32:49] Iluvatar: one qs - the 8% of 504s/timeouts/etc.. that you saw were for multi-lingual or agnostic? Or both : [07:32:52] :) [07:33:36] o/ [07:34:11] kalimera! [07:34:16] elukey: There is also the "pop index out of range" bug. shall I add it in the same task as the one above or create a new one? [07:34:21] elukey: see full log. Multilingual. [07:34:43] isaranto: already filed, see above : [07:34:45] :) [07:35:08] Iluvatar: out of curiosity, did you find the same rate of errors in agnostic? [07:35:12] if you tried it [07:35:13] More logs (today) see at Toolforge swviewer-service/www/us/server.log [07:35:23] aa nevermind! nice work [07:35:26] <3 [07:36:10] elukey: multilingual only. [07:36:13] isaranto: one thing that we may need to discuss is a way for people to get the list of supported wikis from the API [07:36:26] it is in the model's binary so easy for us [07:36:35] but I am wondering what's best [07:36:52] adding it to the :predict call may be ok, but probably counterintuitive? No idea [07:37:26] from the top of my mind my suggestion would be to add an extra post argument and then return it [07:37:37] but yes, it is counterintuitive [07:37:44] exactly I thought the same [07:38:02] but maybe we could add the list of supported wikis to a score, as extra [07:38:20] kserve is not great for these things [07:38:34] maybe having something like :explain would be ideal [07:40:24] need to run an errand, will open few more tasks later on [07:44:48] there is the convention that :explain for explainability. But yes it would be ideal to have sth like this [07:58:45] (03CR) 10Ilias Sarantopoulos: Add language identification service (031 comment) [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/932828 (https://phabricator.wikimedia.org/T340507) (owner: 10Santhosh) [08:35:28] \o Buon giorno! [08:36:03] elukey: does Kserve not allow for arbitrary endpoints? Does everything have to be :predict (or :explain)? [08:45:16] klausman: no idea, I have always seen :predict/:explain and nothing more [08:48:01] Hmmm. [08:49:46] I think a "defult GET endpoint" that points to the model card and lists the currently-available languages (and similar info) would be neat. Not quite the automatic REST docs we have on the o-legacy endpoint, but similar in spirit. [08:50:26] (is it dapper? dasher? I have name collisions with ancient projects I used to work on) [08:51:21] Swagger! [08:58:43] if we enable the docs, there will be a swagger UI that this info could exist but still this doesn't allow once to access these data in an app [08:59:22] iiuc they want to make a call to some endpoint extract the available languages and do their own stuff [09:00:02] yeah, I guess some kinda "capabilities" endpoint with machine-readable info would work. [09:01:02] I just wouldn't want to "pollute" the predict result schema with info that doesn't really belong there, even if it has to be explicitly requested. [09:01:52] we all agree on that one! [09:03:05] I haven't seen kserve supporting this out of the box but I would be curious to check if we can add an extra endpoint in the inference service. The issue is that if this works it would deploy an extra pod just for that info [09:03:42] the best thing would be if we could do it via code with kserve, since it is using fastapi [09:03:54] I admit I have little knowledge of the Kserve API. [09:04:20] But yeah, adding an additional handler to FastAPI should be doable, no? [09:08:26] I would say yes but it may need kserve tweaking so before I try it my response is :¯\_(ツ)_/¯ [09:08:37] 😄 [09:08:43] or, as the kids used to say: idk lol [09:08:43] I'll open a task, but https://github.com/kserve/kserve/blob/release-0.10/python/kserve/kserve/model.py#L322 may be a way [09:10:58] folks, one thing that we need to improve - when we discuss something and a problem arise, we need to open a task with all the details [09:11:12] I'll open one in this case, but in general let's remember to do it [09:14:24] https://github.com/kserve/kserve/blob/release-0.10/python/kserve/kserve/protocol/rest/server.py#L91 [09:14:31] this is where the FastAPI routes are set [09:16:23] ah ok so it is meant to explain a certain call [09:17:05] nice though, we may want to have our own little explainer in a python function [09:17:56] and by default kserve proxies the request to another pod, if it is set [09:26:27] https://github.com/kserve/kserve/blob/master/docs/predict-api/v2/rest_predict_v2.yaml#L66 is also very interesting [09:26:43] with the V2 API we have a specific way to emit the model's metadata [09:32:19] 10Machine-Learning-Team: Revert Risk multi-lingual model performance and reliability may need a review - https://phabricator.wikimedia.org/T340822 (10elukey) [09:51:20] 10Machine-Learning-Team: Provide an API endpoint that returns the model's metadata in Lift Wing - https://phabricator.wikimedia.org/T340824 (10elukey) [09:52:05] Iluvatar: any specific reason why you chose multi-lingual over to agnostic? [09:52:12] (The research team will surely ask) [09:55:49] elukey: you're right on the tasks thing. Will do now on! [10:00:10] elukey: "We recommend using this alternative model [multilingual] for the 47 languages it covers. For the remaining languages, it is advisable to use the original model [agnostic]" [10:00:13] :D [10:00:57] lol [10:01:04] sorry I wasn't aware of that, is it in the model card? [10:01:31] https://meta.wikimedia.org/wiki/Machine_learning_models/Proposed/Language-agnostic_revert_risk#Ethical_considerations,_caveats,_and_recommendations [10:02:24] super thanks, I'll follow up with the Research team [10:02:36] not sure if it is still valid and/or suggested [10:03:20] Iluvatar: I know this is a stretch and this is your free time, so please feel free to say no. If you have time to test agnostic as well it would be really awesome for us, your insights and reports have been great [10:08:27] elukey: I'm not testing, I'm already using it in a live large project. I use both models (I described how on github), all errors are logged in swviewer-service/www/js/server.log. [10:09:46] 10Machine-Learning-Team: Revert Risk multi-lingual model performance and reliability may need a review - https://phabricator.wikimedia.org/T340822 (10diego) >> ~8% of requests ends as { httpCode: 504, httpReason: 'upstream request timeout' } See full log (~1,5h; default settings of requests (python) / reques... [10:10:19] https://imgur.com/a/wxjyFqO [10:12:49] Iluvatar: okok thanks :) [10:13:48] I'll add more pods for multi-lingual [10:13:51] it may help [10:16:43] Iluvatar: I think that I don't have access to the swviewer logs on toolforge, my perms extends only up to the WMF prod infra (separated from the cloud one) [10:16:47] this is why I didn't check [10:19:35] Iluvatar: I am adding more pods to the multi-lingual config, this will mean more compute power, we have only one model server for it at the moment (and more for agnostic) [10:21:33] ah snap there is a new docker image to deploy, need to wait for Aiko's approval [10:22:19] it is the events.py stuff, should be good to be rolled out [10:25:09] ok moar pods for multi-lingual [10:26:08] 10Machine-Learning-Team, 10Patch-For-Review: Revert Risk multi-lingual model performance and reliability may need a review - https://phabricator.wikimedia.org/T340822 (10elukey) I have added more pods for multi-lingual, let's see if performances are better :) [10:27:13] 10Machine-Learning-Team, 10Patch-For-Review: Revert Risk multi-lingual model performance and reliability may need a review - https://phabricator.wikimedia.org/T340822 (10elukey) @diego we'll investigate on our side to see if we can repro, and after that we'll check logs and see if there is any culprit. I tried... [10:35:50] heading out for lunch, ttl! [11:00:13] same! [11:04:38] the future of querying! -> https://www.databricks.com/blog/introducing-english-new-programming-language-apache-spark? [11:08:56] I remain skeptical :D [11:12:09] that's good u should be :) [11:13:05] but lowering the bar for certain stuff is amazing [11:16:08] 10Machine-Learning-Team: Timeout for be-x-old and not support be-tarask in revertrisk language-agnostic - https://phabricator.wikimedia.org/T340830 (10achou) [11:26:47] 10Machine-Learning-Team: Timeout for be-x-old and not support be-tarask in revertrisk language-agnostic - https://phabricator.wikimedia.org/T340830 (10achou) I investigated all the Wikipedia projects listed in https://en.wikipedia.org/w/api.php?action=sitematrix&formatversion=2. This source was helpful for check... [11:36:51] 10Machine-Learning-Team: Timeout for be-x-old and not support be-tarask in revertrisk language-agnostic - https://phabricator.wikimedia.org/T340830 (10achou) Meanwhile, I have added the list of currently supported languages to the model card: https://meta.wikimedia.org/wiki/Machine_learning_models/Proposed/Langu... [11:44:44] 10Machine-Learning-Team, 10MinT, 10SRE, 10serviceops, and 2 others: New Service Deployment Request: NNLB-200 for machine translation - https://phabricator.wikimedia.org/T329971 (10Pginer-WMF) [11:48:12] agree to add a special call to return model's metadata, use :predict and special param should be simplest but a bit weird [11:51:47] I will investigate the errors in revertrisk multilingual today [11:52:13] thank elukey for opening tasks [12:05:05] 10Machine-Learning-Team: Provide an API endpoint that returns the model's metadata in Lift Wing - https://phabricator.wikimedia.org/T340824 (10isarantopoulos) I tested overriding the `explain` endpoint and it works fine. The downside is that we use a post method. Another option would be to manipulate the FastAPI... [12:15:22] Since 10:55 UTC 9 "time-out" (multilingual) errors in 1+ hour, but 88 “time-out” in -damaging model (ex-ORES). Total requests: ~4k. Log: https://pastebin.com/GmpUmEta [12:15:22] All working is very fast: https://imgur.com/a/5D5VK8B [12:15:22] Other errors: pop index, list index, Feature, two "connection failure" and one “unexpected content type for {rev-id}: {e}” (damaging; wikidata, 1926421323; 12 NS). [13:24:53] 10Machine-Learning-Team: Revert Risk multi-lingual model performance and reliability may need a review - https://phabricator.wikimedia.org/T340822 (10elukey) >>! In T340822#8980139, @diego wrote: > @elukey can we assume that different responses time for the same request are due //MediaWiki weather conditions/"?... [13:31:45] (03PS1) 10AikoChou: revertrisk: fix language be-x-old's host [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/934544 (https://phabricator.wikimedia.org/T340830) [13:40:24] Iluvatar: for damaging there was an error in one of our pods, I fixed it, lemme know if it improves [13:41:31] (03CR) 10AikoChou: "I'm not sure if this patch will fix the issue, since in the kserve logs for the error, I saw:" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/934544 (https://phabricator.wikimedia.org/T340830) (owner: 10AikoChou) [13:57:23] (03PS1) 10Elukey: revscoring: fix f strings in extractor_utils.py [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/934547 [13:57:27] this is an easy one [13:59:41] (03PS2) 10Elukey: revscoring: fix f strings in extractor_utils.py [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/934547 [14:06:58] (03CR) 10AikoChou: [C: 03+1] revscoring: fix f strings in extractor_utils.py [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/934547 (owner: 10Elukey) [14:12:55] (03CR) 10Elukey: [C: 03+2] revscoring: fix f strings in extractor_utils.py [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/934547 (owner: 10Elukey) [14:13:00] Thanks aiko_ :) [14:13:05] (03CR) 10Ilias Sarantopoulos: [C: 03+1] revscoring: fix f strings in extractor_utils.py [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/934547 (owner: 10Elukey) [14:13:12] and Ilias :) [14:15:51] I should have changed these while working on revscoring, we have some leftover stuff [14:16:13] elukey: what about the langid patch, shall I deploy a version of that in experimental? [14:18:25] +1 [14:19:47] (03Merged) 10jenkins-bot: revscoring: fix f strings in extractor_utils.py [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/934547 (owner: 10Elukey) [14:24:03] (03CR) 10Ilias Sarantopoulos: [C: 03+2] Add language identification service [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/932828 (https://phabricator.wikimedia.org/T340507) (owner: 10Santhosh) [14:24:15] (03PS18) 10Ilias Sarantopoulos: Add language identification service [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/932828 (https://phabricator.wikimedia.org/T340507) (owner: 10Santhosh) [14:26:37] I also remove the nsfw model (according to Hal's suggestion) [14:27:55] 10Machine-Learning-Team: The nsfw model hangs in predict() after moving to Kserve 0.10 - https://phabricator.wikimedia.org/T331416 (10isarantopoulos) I have removed the nsfw deployment from the experimental namespace of ml-staging-codfw. [14:33:29] (03CR) 10CI reject: [V: 04-1] Add language identification service [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/932828 (https://phabricator.wikimedia.org/T340507) (owner: 10Santhosh) [14:38:59] (03CR) 10Ilias Sarantopoulos: [V: 03+2] Add language identification service [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/932828 (https://phabricator.wikimedia.org/T340507) (owner: 10Santhosh) [14:39:51] hmm, llm test image failed again [14:43:01] (03CR) 10AikoChou: revertrisk: fix language be-x-old's host (031 comment) [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/934544 (https://phabricator.wikimedia.org/T340830) (owner: 10AikoChou) [14:44:25] (03PS2) 10AikoChou: revertrisk: fix language be-x-old's host [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/934544 (https://phabricator.wikimedia.org/T340830) [14:47:54] (03Merged) 10jenkins-bot: Add language identification service [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/932828 (https://phabricator.wikimedia.org/T340507) (owner: 10Santhosh) [14:49:01] (03CR) 10Elukey: [C: 03+1] revertrisk: fix language be-x-old's host [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/934544 (https://phabricator.wikimedia.org/T340830) (owner: 10AikoChou) [14:52:59] kevinbazira: Nice work on the rec api image! Sorry for the long comments, just wanted to explain my thinking :) [14:55:15] (03CR) 10Ilias Sarantopoulos: [C: 03+1] "LGTM! What worries me is that these if statements are getting bigger and bigger. If they continue to grow we can think of some alternative" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/934544 (https://phabricator.wikimedia.org/T340830) (owner: 10AikoChou) [14:56:39] 10Machine-Learning-Team: The nsfw model hangs in predict() after moving to Kserve 0.10 - https://phabricator.wikimedia.org/T331416 (10elukey) 05Open→03Resolved a:03elukey [14:58:03] (03PS3) 10AikoChou: revertrisk: fix language be-x-old's host [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/934544 (https://phabricator.wikimedia.org/T340830) [14:58:48] (03CR) 10AikoChou: [C: 03+2] "Thanks for the review :)" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/934544 (https://phabricator.wikimedia.org/T340830) (owner: 10AikoChou) [15:02:46] 10Machine-Learning-Team, 10Language-Team, 10Epic: Verify if the Python recommendation API can support the use cases of the nodejs one - https://phabricator.wikimedia.org/T340854 (10elukey) [15:04:27] (03Merged) 10jenkins-bot: revertrisk: fix language be-x-old's host [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/934544 (https://phabricator.wikimedia.org/T340830) (owner: 10AikoChou) [15:13:08] 10Machine-Learning-Team, 10Foundational Technology Requests: Content Translation Recommendations API - https://phabricator.wikimedia.org/T293648 (10elukey) >>! In T293648#8978054, @leila wrote: > @elukey the short answer is that we don't have an agreement, yet, and we should. : ) Nice :) > can you please pr... [15:20:22] aiko_: o/ https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/934560 [15:20:35] to update the revert risk images with the last two fixes [15:21:22] thanks!! [15:22:31] thank you! Deploying [15:30:19] 10Lift-Wing, 10Machine-Learning-Team, 10I18n, 10NewFunctionality-Worktype, 10Patch-For-Review: Create a language detection service in LiftWing - https://phabricator.wikimedia.org/T340507 (10isarantopoulos) The langid model server has been deployed in the experimental namespace in ml-staging-codfw. Sample... [15:32:27] (03PS1) 10Ilias Sarantopoulos: langid: get model name from env var [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/934572 (https://phabricator.wikimedia.org/T340507) [15:32:53] langid works fine! I also submitted another patch with sth I had missed (not urgent at all) [15:35:08] I think I fixed mediawiki extension also.. \o/ finally [15:35:15] wooww [15:35:40] ofc it was really stupid [15:37:30] niceeeee \o/ [15:37:33] (03PS25) 10Ilias Sarantopoulos: fix: corrent config variable access [extensions/ORES] - 10https://gerrit.wikimedia.org/r/932289 (https://phabricator.wikimedia.org/T319170) [15:39:00] 10Machine-Learning-Team, 10MediaWiki-extensions-ORES, 10MW-1.41-notes (1.41.0-wmf.15; 2023-06-27), 10Patch-For-Review: Move backend of ORES MediaWiki extension to Lift Wing - https://phabricator.wikimedia.org/T319170 (10PatchDemoBot) Test wiki **created** on [[ https://patchdemo.wmflabs.org | Patch demo ]]... [15:40:01] \o/ ores filters now appear -here is an example with patchdemo https://patchdemo.wmflabs.org/wikis/ad97984435/wiki/Special:RecentChanges?hidebots=1&limit=50&days=7&enhanced=1&urlversion=2 [15:40:29] \o/ \o/ nice work! [15:40:41] aiko_: green light to deploy readability! [15:40:41] (03PS26) 10Ilias Sarantopoulos: fix: corrent config variable access [extensions/ORES] - 10https://gerrit.wikimedia.org/r/932289 (https://phabricator.wikimedia.org/T319170) [15:41:54] elukey: thanks Luca! [15:42:16] Iluvatar: we fixed a couple of bugs: "unexpected content type for {rev-id}: [15:42:23] (now it should correctly print the msg) [15:42:31] and be-x-old [15:42:57] Thanks! [15:42:58] aiko_: when you have a moment can you test if be-x-old works fine via api gateway? [15:43:10] Iluvatar: more to come next week! Thank you for the report! [15:43:23] (03PS27) 10Ilias Sarantopoulos: fix: corrent config variable access [extensions/ORES] - 10https://gerrit.wikimedia.org/r/932289 (https://phabricator.wikimedia.org/T319170) [15:44:22] elukey: just tested. it works fine :) [15:44:29] super :) nice fix [15:44:50] 10Machine-Learning-Team: Timeout for be-x-old and not support be-tarask in revertrisk language-agnostic - https://phabricator.wikimedia.org/T340830 (10achou) Timeout issue for be-x-old has been fixed :) ` aikochou@wmf3132 ~ % curl https://api.wikimedia.org/service/lw/inference/v1/models/revertrisk-language-agno... [15:44:53] (03CR) 10Ilias Sarantopoulos: "@Amir The ORES filters appear again. You can check in this patchdemo deloyment -> https://patchdemo.wmflabs.org/wikis/ad97984435/wiki/Spec" [extensions/ORES] - 10https://gerrit.wikimedia.org/r/932289 (https://phabricator.wikimedia.org/T319170) (owner: 10Ilias Sarantopoulos) [15:45:03] going afk for the weekend folks! [15:45:13] have a nice rest of the day and weekend :) [15:45:48] have a nice weekend Luca! [15:46:14] (03CR) 10CI reject: [V: 04-1] fix: corrent config variable access [extensions/ORES] - 10https://gerrit.wikimedia.org/r/932289 (https://phabricator.wikimedia.org/T319170) (owner: 10Ilias Sarantopoulos) [15:46:52] (03PS28) 10Ilias Sarantopoulos: fix: corrent config variable access [extensions/ORES] - 10https://gerrit.wikimedia.org/r/932289 (https://phabricator.wikimedia.org/T319170) [15:47:40] I'm logging off as well, enjoy your weekend folks! [15:49:26] bye Ilias, have a nice weekend :) [15:53:11] heading out now as well, \o [15:53:20] aiko_: have a nice weekend :) [15:57:46] klausman: bye Tobias :) [16:26:31] 10Machine-Learning-Team, 10Data Engineering and Event Platform Team, 10Data-Engineering, 10Event-Platform: Implement PoC Event-Driven Data Pipeline for Revert Risk Model Scores using Event Platform Capabilities - https://phabricator.wikimedia.org/T338792 (10JArguello-WMF) [16:29:27] 10Machine-Learning-Team, 10Data Engineering and Event Platform Team, 10Data-Engineering, 10Event-Platform: Create new mediawiki.page_links_change stream based on fragment/mediawiki/state/change/page - https://phabricator.wikimedia.org/T331399 (10JArguello-WMF) [16:29:53] 10Machine-Learning-Team, 10Data Engineering and Event Platform Team, 10Data-Engineering, 10Edit-Review-Improvements-Integrated-Filters, and 3 others: Integration of Revert Risk Scores to Recent Changes as a filter - https://phabricator.wikimedia.org/T329071 (10JArguello-WMF) [16:30:01] 10Machine-Learning-Team, 10Data Engineering and Event Platform Team, 10Data-Engineering, 10Event-Platform: Add a new outlink topic stream for EventGate main - https://phabricator.wikimedia.org/T328899 (10JArguello-WMF) [16:30:23] 10Machine-Learning-Team, 10Data Engineering and Event Platform Team, 10Data-Engineering, 10Research, 10Event-Platform: Proposal: Create a stream end point for Revision Risk Model - https://phabricator.wikimedia.org/T326179 (10JArguello-WMF) [17:44:44] 10Machine-Learning-Team, 10API Platform, 10Anti-Harassment, 10Cloud-Services, and 18 others: Migrate PipelineLib repos to GitLab - https://phabricator.wikimedia.org/T332953 (10JArguello-WMF) [19:26:36] 10Machine-Learning-Team, 10CirrusSearch, 10Discovery-Search (Current work): Add outlink topic model predictions to CirrusSearch indices - https://phabricator.wikimedia.org/T328276 (10CodeReviewBot) ebernhardson updated https://gitlab.wikimedia.org/repos/search-platform/discolytics/-/merge_requests/28 ores:... [19:39:45] 10Machine-Learning-Team, 10CirrusSearch, 10Discovery-Search (Current work), 10Patch-For-Review: Add outlink topic model predictions to CirrusSearch indices - https://phabricator.wikimedia.org/T328276 (10CodeReviewBot) ebernhardson opened https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/me... [19:39:58] 10Machine-Learning-Team, 10CirrusSearch, 10Discovery-Search (Current work), 10Patch-For-Review: Add outlink topic model predictions to CirrusSearch indices - https://phabricator.wikimedia.org/T328276 (10CodeReviewBot)