[14:06:15] <inflatador_>	 <o/
[14:14:07] <inflatador_>	 IRC issues w my user, working on it now
[15:31:55] <inflatador>	 .o/
[16:00:12] <ebernhardson>	 \o
[16:00:58] <inflatador>	 <o/
[16:44:11] * ebernhardson wonders if there is some way to have reindexing events, and a kafka source that is reconfigured based on those reindexing events
[16:51:10] <topranks>	 folks moves in rack a4 are done if you want to unban those elastic hosts 
[16:51:48] <ebernhardson>	 hmm, was almost another option with elasticsearch writing to multiple indexes from one request, but that was added in 7.14
[16:52:14] <ebernhardson>	 (have an alias that points to the live index and the one being reindexed into, instead of backfilling)
[16:53:47] <inflatador>	 topranks ACK, thanks for the update
[16:54:30] <ebernhardson>	 maybe could do that at the flink level i guess, but seems like hairy state to manage
[17:14:22] <inflatador>	 workout, back in ~40
[17:14:57] <ebernhardson>	 hmm, latest release of flink kafka connector includes a dynamic kafka source. It's intended for multi-kafka cluster stuff but maybe can repurpose...
[17:17:14] <ebernhardson>	 but that connector looks somehow broken...running our test suite fails with a NoSuchMethodError between the connector and flink base.
[17:35:43] * ebernhardson supposes it probably works with 1.18 (although it ships a 1.17 jar...that doesn't work). But our elasticsearch connector hasn't released for 1.18 yet :(
[17:41:35] * ebernhardson isn't sure what they compiled against,  connectors 3.0.2-1.17 and 3.1.0-1.17 fail against 1.17.0 and 1.17.1
[18:02:21] <inflatador>	 back
[18:41:01] <dr0ptp4kt>	 ebernhardson: okay if i merge https://gitlab.wikimedia.org/repos/search-platform/cirrus-streaming-updater/-/merge_requests/105 or you have any final touch up?
[18:42:13] <inflatador>	 lunch, back in ~50
[19:05:50] <ebernhardson>	 dr0ptp4kt: should be fine, if it needs changes we can always do more
[19:08:57] <inflatador>	 back
[19:09:21] <dr0ptp4kt>	 ebernhardson: done
[19:10:04] <ebernhardson>	 re: that comment, the info property adds the page['rev_id'] value that's used to detect bad revisions
[19:10:16] <ebernhardson>	 it turns out we have things like pages with no revisions, that fail when requested :)
[19:28:00] <ryankemper>	 inflatador: running 5-10 mins late to pairing
[19:28:16] <inflatador>	 ryankemper ACK, np
[20:30:47] <inflatador>	 If anyone has time to look at https://gerrit.wikimedia.org/r/c/operations/puppet/+/999088 LMK, this will be the 4th of 6 hosts that are migrating
[21:04:26] <inflatador>	 https://gerrit.wikimedia.org/r/c/operations/puppet/+/1003100 another CR, this one to add more migrated hosts as masters
[21:23:52] * ebernhardson realizes the kafka dynamic source implementation is missing a critical part...the bit that actually communicates with the taskmanagers is an interface you have to implement
[21:24:31] <ebernhardson>	 there is just an interface that the kafka code on taskmanager's query to figure out what they should subscribe to, and there is no provided implementation :P
[21:25:46] <inflatador>	 "This implementation is a stub..." ;P
[21:56:45] * ebernhardson is having terrible luck with internet today...mtr constantly reporting that my isp is junk-tier :P
[22:25:16] * ebernhardson is surprised to find flink broadcast state is nothing like spark broadcast state. In spark you mutate a thing and it's shipped out. In flink you need to be deterministic, every instance processes the inputs to the broadcast and need to give the same results
[22:31:24] <inflatador>	 hmm, I'm only seeing a few gigs of data that are older than a month...or (entirely possible) my logic is bad
[22:31:41] <inflatador>	 https://gitlab.wikimedia.org/repos/search-platform/sre/cleanup-flink-object-storage/-/blob/main/cleanup-flink-object-storage.py?ref_type=heads#L59
[22:38:00] <ebernhardson>	 inflatador: you might be backwards?  I think you want 'last_modified < thirty_days_ago'
[22:38:18] <ebernhardson>	 err, no you have it right i'm just bad at reading :P
[22:40:21] <inflatador>	 yeah, that was throwing me too...I **think** the newer date should have a smaller value? But if it's epoch time, the newer date should be bigger, right?
[22:41:40] <inflatador>	 so if it's epoch time, then I think you're right...the newer date should be bigger. `if last_modified < thirty_days_ago`
[22:42:48] <inflatador>	 yup, you were right, need a less than sign
[22:44:48] <inflatador>	 but that sounds wrong in human language "If last modified is less than 30 days ago"
[22:52:25] * inflatador wonders if I need to do anything to the gitlab repo to get it to post on phab tasks
[22:59:22] <ebernhardson>	 inflatador: usually i just do it in a repl with example values and see which gives the result i want :)
[22:59:35] <ebernhardson>	 inflatador: to get the phab auto-messages you need a Bug: line in the MR 
[22:59:58] <ebernhardson>	 i don't know if editing it in works or if it has to be there when creating the MR
[23:00:19] <inflatador>	 ebernhardson that's what I ended up doing re: repl . Re: Bug line, I've been pushing directly to main w/no MR so I guess that's why