[11:26:21] <gmodena>	 lunch
[11:34:01] <dcausse>	 meh can't pass a timestamp from values file to a flink app config file... it gets wrapped with double quotes causing '\"2025-01-21T09:25:00Z\"' could not be parsed at index 0
[11:36:01] <dcausse>	 well we use a toYaml function to create a properties file... that might explain...
[11:38:45] <dcausse>	 lunch
[12:28:03] <gmodena>	 dcausse :(
[13:16:43] <gmodena>	 today my plan was to scale up xgboost experiments... but so far it's been a lot fiddling with spark 
[13:17:26] <gmodena>	 oof
[13:17:53] <gmodena>	 i'm almost tempted to throw polars and a tone of ram/cores to the data prep stage
[13:59:54] <dcausse>	 gmodena: you mean polars to replace pandas?
[14:00:47] <gmodena>	 dcausse i mean polars to replace spark (in some cases)
[14:01:02] <dcausse>	 oh ok
[14:01:56] <dcausse>	 always worried to use those because I'm not sure what I'd do if at some point the data grows too big
[14:02:19] <dcausse>	 used pandas on small dataset but I know Erik is a big fan of pandas and uses it on pretty big things
[14:02:59] <dcausse>	 never used polars but reading about it it seems much more efficient mem wize
[14:06:22] <dcausse>	 gmodena: if you have a sec, I have a quick&dirty workaround for properties file format issue at https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1113139
[14:07:36] <inflatador>	 <o/
[14:07:53] <gmodena>	 dcausse i won't rewrite all the things, promise :D. I think I managed to fix some query imbalance issues I had with spark. Annoyingly, I think the data would have fit in memory to being with.
[14:08:07] <gmodena>	 dcausse checking (while said query runs in the background)
[14:08:09] <gmodena>	 inflatador o/
[14:08:54] <dcausse>	 gmodena: sure, please do whatever you think work best! :)
[14:09:01] <dcausse>	 o/
[14:18:48] <gmodena>	 dcausse left you a q on the CR. But feel free to merge at will if not relevant.
[14:19:00] <dcausse>	 gmodena: thanks, looking!
[19:33:16] <dcausse>	 finally got the wdqs flink job running in staging with event utilities APIs...
[19:33:30] <dcausse>	 will let it run for a while before moving to prod tho...
[19:33:33] <dcausse>	 dinner
[19:34:50] <gmodena>	 dcausse nice!
[19:35:30] <gmodena>	 and ack on CR. I'll take a look tomorrow am
[20:33:25] <inflatador>	 quick break, back in ~20
[21:51:40] <inflatador>	 did we ever reach a consensus on what Opensearch version we were gonna use (1 or 2) ? I thought we were going to use 1, but LMK if I'm misremembering. Goona try and dig up e-bernhardson's notes