[22:51:36] <simon04>	 Hi all. I've been trying to migrate one of my tools from gridengine to kubernetes. I've spent several hours, but I'm still failing. The tool in question is bldrwnsch. The main part is a regular job (used to be: hourly cron job) consisting of 1. SQL query to generate TSV file (using `sql` command), 2. converting the TSV to JSON, GEOJSON, GPX, KML,
[22:51:37] <simon04>	 KMZ, (3. running `tippecanoe` to generate protobuf tiles). For part 1 and 2, I've used a simple Makefile and ogr2ogr from GDAL. Neither of both is supported on kubernetes. Thus I've rewritten everything in Python/PyMySQL. Generating TSV via Python is a PITA (due to dealing with bytes, decimal.Decimal, bla bla types being returned from the database
[22:51:37] <simon04>	 query). Having solved this issue, too, now the `toolforge jobs` fails with a random `Killed                  ./updateBilderwuensche.py` error without giving any reason. I'm close to giving up. I'm close to throw in the sponge. I mean, how hard can it be to run an SQL query and generate a few text-based file-formats!?!?!?
[22:54:21] <wm-bb>	 <bd808> “Killed” sounds like you may be using up all of your job’s ram.
[22:56:55] <simon04>	 How are the limits? The TSV file is 42M. The GEOJSON file is 89M. Before writing the GPX file, `Killed` happens.
[23:02:28] <AntiComposite>	 512MiB is the default limit for a single pod. With toolforge-jobs, you can throw more memory at it with `--mem 1GiB`
[23:08:31] <AntiComposite>	 https://grafana.wmcloud.org/d/TJuKfnt4z/kubernetes-namespace?orgId=1&var-namespace=tool-bldrwnsch shows the memory usage, but I've found it doesn't always show large spikes that kill the pod
[23:11:16] <simon04>	 Cool. Using `--mem 1G` seems to fix the job. For now. Hopefully the data size doesn't grow above this limit. I mean, I could start low-level tuning the generation of JSON and XML files (by not using Python's built-in classes, but manually writing properly escaped string to the corresponding files.
[23:11:49] <simon04>	 All in all, this kubernetes migration has taken way more time than anticipated for a not overly complex use-case.
[23:27:40] <simon04>	 Done. Thanks. https://github.com/simon04/bldrwnsch/commit/5dec9f1c9f57c4578a61b503f3e978fe0ecdf0cf