[10:51:47] <gehel>	 Lunch 
[11:13:51] <dcausse>	 going out for lunch, might be back a bit late around 3pm
[12:40:12] <pfischer>	 lunch
[14:07:34] <inflatador>	 <o/
[14:33:28] <dcausse>	 o/
[16:01:31] <ebernhardson>	 \o
[16:06:16] <dcausse>	 o/
[16:09:09] <ebernhardson>	 meh, just realized while re-reading my sharding of the cirrus dump (on the snapshots hosts, not hadoop), that this now re-dumps metastore for each invocation of the script...i guess thats fine, its tiny, but a little messy
[16:10:19] <dcausse>	 does not seem to be a big deal as long as it does not fail to overwrite existing file?
[16:11:17] <dcausse>	 but perhaps better to make it separate process (or simply drop it, do we need this dump)?
[16:11:23] <ebernhardson>	 hmm, i guess should update the temp file names used to make sure multiple invocations don't use the same temp file
[16:11:32] <ebernhardson>	 it writes to a temp file and then moves in place
[16:12:02] <ebernhardson>	 not sure if we need the metastore dump, i've never really used it (except indirectly, with saneitizer)
[16:12:21] <ebernhardson>	 i know it records the analysis chain versioning/git hashes, but we mostly track that manually
[16:12:45] <dcausse>	 yes, and we don't update the version numbers there so it's not really useful
[16:14:48] <dcausse>	 I feel that this index is not important enough that we spend time on it, so if it's a problem that it's dumped multiple times I'd be for stopping to dump it
[16:15:56] <ebernhardson>	 yea, i think it makes sense to simply drop it from this. Making it use unique temp files and move into place would make it correct, but still useless
[16:17:24] <ebernhardson>	 last question is what to do with the 'current' symlink ... in theory that should run after all the dumps have completed, but now they are all separate systemd timers :P   Maybe ignore that bit
[16:55:46] <dcausse>	 oh current is a folder :/
[16:55:58] <ebernhardson>	 hmm, it should be a symlink
[16:56:10] <ebernhardson>	 the code does `rm -f current; ln -s $today current`
[16:56:18] <Trey314159>	 ebernhardson: check on the quickview discussion on slack when you have sec.
[16:56:20] <Trey314159>	 I'm off to the dentist!
[16:56:29] <ebernhardson>	 i guess there is no -r, so it wouldn't delete if it became a folder at some point. looking
[16:56:53] <dcausse>	 so it'll only work if the system timer runs on the same date
[16:57:27] <dcausse>	 otherwize one day we'll see group 0 wikis and another one group 1 ones
[16:57:48] <ebernhardson>	 i think they should, they all use the same timer definition and the today variable gets captured before doing anything. But indeed it's not the strongest guarantee
[16:58:10] <ebernhardson>	 something like airflow could better schedule this kind of dependency :P
[16:58:11] <dcausse>	 or expose the group as a folder: dumps/group0/current 
[16:58:17] <dcausse>	 indeed!
[16:58:49] <ebernhardson>	 i suppose i don't really want to re-engineer, but i think we could generate the dumps from the hadoop side and simply copy them into hdfs
[16:59:07] <ebernhardson>	 err, copy into nfs
[16:59:26] <dcausse>	 ultimately yes
[16:59:46] <dcausse>	 so yes perhaps let's try the minimal effort approach and see how far we get?
[17:01:04] <inflatador>	 quick break, back in ~40
[17:01:05] <ebernhardson>	 yea i think will leave it with the less-than-perfect current symlink and see how it goes
[17:01:08] <dcausse>	 if Ariel is OK redoing the same symlink multiple times I'm all for it
[17:32:20] * ebernhardson realizes while filling out the ITC that it's supposed to be about a specific quarter but i usually write about whats been happening recently :P So my Q4 ITC talks about the elastic 6->7 upgrade, but technically thats the Q1 ITC that is due now
[17:33:57] <dcausse>	 it's confusing as hell, I try to speak about the proper Q but when discussing with discussing with I realize that we do not talk about the same period :)
[17:34:23] <dcausse>	 s/when discussing with discussing/when discussing with Guillaume/
[17:36:03] <ebernhardson>	 lol, yea sounds about right
[18:41:53] <inflatador>	 back, sorry it took so long
[19:04:47] <dcausse>	 dinner
[19:31:48] <gehel>	 ryankemper: pairing session: https://meet.google.com/eki-rafx-cxi
[19:41:30] <Trey314159>	 b?r[yi]an (aka inflatador and ryankemper ): is today or tomorrow an okay time to do some reindexing?
[21:20:43] <inflatador>	 Trey314159 I think it would be OK unless ebernhardson has any specific objections. We need to do a cluster restart at some point, but it's not urgent
[21:21:25] <Trey314159>	 It's one really small wiki and one medium, so it shouldn't take too long!
[21:29:49] <inflatador>	 Trey314159 Fire away then, but do hit us up here when you're done
[21:29:59] <Trey314159>	 Will do!
[21:33:34] <ryankemper>	 inflatador: gehel: bleh sorry for missing pairing, phone charger got unseated overnight somehow
[21:38:45] <inflatador>	 no worries, you did all the work last time ;)