[09:02:15] unmeeting happening in https://meet.google.com/hvn-zxxd-xrb [09:02:23] anyone is welcomed to join [09:52:28] lunch [10:05:13] I was just talking with pfischer about the work on the search update pipeline. It looks like we might be able to split his current task so that pfischer focuses on finishing the event bus stuff in mw-core and maybe ebernhardson can jump in to already start consuming those events on the flink side. [10:05:26] dcausse, ebernhardson: what do you think? [10:06:13] schema definitions can be hosted anywhere so we could get started right away. [11:13:37] Good morning. I have updated some information on Wikitech about the airflow instance for this team, an-airflow1005, now that an-airflow1001 has gone. [11:13:56] Please could someone verify that it is correct and update if necessary? https://wikitech.wikimedia.org/wiki/Data_Engineering/Systems/Airflow/Instances#search [11:20:57] Also, if I could get a review on this I'd be grateful: https://gerrit.wikimedia.org/r/c/analytics/refinery/scap/+/919036 - Does anyone know whether or not refinery will be needed on an-airflow1005? [11:37:38] infos on wikitech looks good to me, but I'll let ebernhardson confirm [11:38:36] we might have dependencies on refinery for the rdf analysis. pfischer would you know? Otherwise, ebernhardson again? [11:49:12] gehel, pfischer: we talked yesterday that Erik might start looking into weighted_tags [12:39:58] There is a thread on wikitech-l@ about word embeddings / vector search (https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/thread/PJGRN2H2CUSACZH4QSI2VOV3WBDZ3J6O/) and we've been called out (which makes sense). We should probably respond on list. [12:40:20] I'll get a doc started, but I'll need your input and review before pushing it out (unless someone else wants to take this). [13:07:45] o/ [14:03:11] btullis: refinery is needed on an-airflow1005 (it invokes the data retention scripts), but it should separately be cloned via the profile::analytics::refinery include. tbh i'm not sure what the environment targets there do [14:03:48] btullis: i suppose they get it updated slightly earlier? iirc puppet already does a git pull on the repos defined in it each run [15:02:20] inflatador, mpham: https://meet.google.com/eki-rafx-cxi retrospective! [15:49:12] dinner [16:01:06] inflatador: would you have time to contiue the deploy of admin_ng, the gerrit maintainance seems over? [16:01:35] dcausse Y was just about to ask ;) [16:01:42] cool :) [16:02:04] I'm in https://meet.google.com/iqe-wcuz-mpn?authuser=0 (puppet deploy window) [17:22:39] dinner [17:48:24] lunch, back in ~45 [18:30:19] back [18:51:16] ebernhardson: Thanks I'll double check if there's any need to have it in the scap target as well, or whether the puppet clone is all that's needed. [19:49:10] incident report here: https://wikitech.wikimedia.org/wiki/Incidents/2023-05-05_wdqs_not_updating_in_codfw will keep fillin it in based on David's timeline