[09:17:43] <dcausse>	 wdqs1015 is completely stuck, load rising to insane values (300+)
[09:35:23] <dcausse>	 wondering if I can trust spark DataFrame.cache() to not rebuild the underlying rdd from scratch in all cases, context is trying to extract a subset of the cirrus index without the content fields
[09:36:30] <dcausse>	 with cache: elasticsearch -> rdd -> df -> cache -> [write to two hive tables: cirrus_full & cirrus_without_content]
[09:37:49] <dcausse>	 or elasticsearch -> rdd -> df -> write to hive cirrus_full (keep the pipeline as is), and after read from hive cirrus_full -> remove content fields -> write to hive cirrus_without_content
[09:39:45] <dcausse>	 thinking about it I'm not convinced that writing to the spark cache is going to be more efficient than writing to hive so it's probably not worth trying...
[10:08:28] <dcausse>	 lunch
[11:42:54] <dcausse>	 errand
[12:01:39] <pfischer>	 Lunch
[12:56:33] <inflatador>	 <o/
[12:58:17] <inflatador>	 looks like wdqs1015 just came back up, LMK if you need me to investigate
[13:05:52] <inflatador>	 nm, it's still alerting
[13:06:11] <inflatador>	 just rebooted it via DRAC
[13:11:07] <inflatador>	 depooled wdqs2015, will check its lag
[13:12:23] <inflatador>	 oops, that should be 1015
[13:13:39] <inflatador>	 to be clear, I rebooted/depooled 1015, just wrote it wrong in chat
[13:14:20] <dcausse>	 thanks :)
[13:54:49] <inflatador>	 dcausse LMK if you have anything for pairing today, otherwise we can skip
[13:56:39] <dcausse>	 inflatador: we can skip, I'll try to get an image of the wdqs branch with flink 1.16.1 support ready to deploy to production for tomorrow 
[13:57:19] <inflatador>	 ACK, sounds good
[15:55:38] <dcausse>	 going offline
[17:49:27] <inflatador>	 lunch, back in ~45
[18:50:31] <inflatador>	 Data transfer cookbook is bombing out again...probably have to see if there's some python libraries instead of shelling out
[18:56:18] <inflatador>	 back