[08:36:57] inflatador: we have to run 1 for the transition from elastic to opensearch but the plan is to upgrade to 2 at some point [10:14:56] dcausse: short meeting this afternoon to talk about T382295. Do you know more about that new stream? Will it use weighted tags as I suspect? And if that's the case, I'd expect most of the work to happen outside of our team, is that correct? [10:14:58] T382295: Create event stream for article-country model-server hosted on LiftWing - https://phabricator.wikimedia.org/T382295 [10:14:58] gehel: if you have a sec I have quick (scala) review https://gerrit.wikimedia.org/r/c/wikidata/query/rdf/+/1113204 [10:15:30] gehel: looking [10:15:58] gmodena: if you're interested in weighted tags, feel free to join as well. But this is probably going to be super high level manager stuff (dcausse: I'm not sure you're needed in that meeting, so feel free to skip as well) [10:16:06] dcausse: looking [10:17:28] dcausse: +2 [10:17:42] gehel: I think they used the article topic approach where it's us that consumed from a dedicated stream they created [10:18:32] any reason to have them take ownership of ingestion, via weighted tags? I thought that was the way forward? [10:18:43] but I understand that they create this stream solely for the search index so it might make sense for them to publish to the weighted tags stream directly instead of this "liftwing.test-article-country-events" [10:19:02] gehel: absolutely [10:19:55] I think they just followed what they did previously [10:21:39] I'm sure I saw tickets mentionning article country with Erik talking about the weighted tags stream [10:22:25] I'll check with them this afternoon. There's probably no need for you to join at this point. We can schedule an engineer to engineer meeting as needed. [10:23:09] yes it's here https://phabricator.wikimedia.org/T301671#10064971 [10:25:45] sadly this was lost... so ideally yes they should rework a bit what they've done and produce something to cirrussearch.page_weighted_tags_change.rc0 (we should probably remove that rc0 suffix now) [10:38:52] gehel: I'd suggest to invite someone from ML since they're going to do the plumbing work [10:41:08] I think that's where the info got lost, I have the impressions that some discussions between Search <-> Research got lost when ML started to work on this [10:49:22] errand+lunch [12:26:02] ah [12:26:52] dcausse: do they have any interest in exposing the predictions in eventstreams? is enterprise interested in article predictions? (or outlink topics even?) [12:26:52] Would it be useful to have those predictions in hive? [13:24:40] ottomata: indeed, good point reading the tickets I feel there're mainly interested in pushing this into search [13:25:45] but I don't see a problem if they produce both to our generic search weighted tag ingestion stream and a more generic stream [13:26:12] from our end we'd definitely prefer them to push to our ingestion stream [13:31:28] ottomata: unrelated but something curious I've seen, the eqiad.mediawiki.page_change.v1 topic (which should be empty since we run the master in codfw) is seeing few events from wikidatawiki, testwikidatawiki and commonswiki (wondering if it's wikibase allowing edits on non-POST requests) [14:01:46] o/ [14:59:38] dcausse: hm, re eqiad topic interesting [15:00:17] i could see maybe someone internal somehow getting an edit to happen on one of these wikis. But yeah could be some routing configs for GET that works for wikidata differently? [15:00:18] hm [15:00:36] dcausse: re prediction stream, yeah...we should ask [15:00:40] i'd really like people to think about data reuse [15:00:51] not just the final UI product (in this case mw search) [15:01:02] I like your ingestion pipeline a lot [15:01:08] i suppose they could just produce both! [15:01:11] probably easist [15:01:12] we have a meeting tomorrow want me to invite you? [15:50:48] dcausse: okay yes please! [15:51:42] done [16:03:39] David and I came up with a rough plan for the graph split migration, feel free to add/edit/ask questions https://etherpad.wikimedia.org/p/wdqs-split-graph-rollout-plan [16:03:45] ryankemper ^^ [16:35:57] ryankemper we're in SRE standup if you wanna join [17:17:41] inflatador: sorry taking another half sick day [17:17:47] Will look over the rollout plan tho [17:49:51] ack, hope you feel better