[10:32:25] <dcausse>	 lunch
[13:04:51] <inflatador>	 <o/
[13:07:01] <dcausse>	 o/
[14:49:54] <inflatador>	 I know y'all would probably rather forget WDQS, but does anyone remember a time when blazegraph used a ton of space in `/tmp`? More context in T437298, but I'm mainly wondering if there's an automatic cleanup process that isn't working anymore
[14:49:54] <stashbot>	 T437298: Investigate high /tmp usage in wdqs-main eqiad - https://phabricator.wikimedia.org/T437298
[14:51:06] <inflatador>	 my best guess is that BG loses track of its tmpfiles when we restart, so it can't manage them
[14:59:08] <dcausse>	 inflatador: that does not ring a bell, wondering if it could be due to aggressive kills made because of some remediation like jvmquake or the one that kills based on high thread count? 
[14:59:49] <dcausse>	 looking over tmp I see jetty tmp files which are mainly the webapp and all blazegraph jars, so this can eat up a lot of space quickly
[15:00:37] <dcausse>	 I would test a graceful restart to see if they're properly cleaned up, if yes it's possibly because of more aggressive kills
[15:06:06] <inflatador>	 dcausse yeah, we have that auto-remediation script that kills based on time and thread count.
[15:06:42] <inflatador>	 I'll try the restart, but if it doesn't work we can probably use `systemd-tmpfiles` to clean it up
[15:09:49] <dcausse>	 inflatador: sure
[16:42:12] <dcausse>	 errand