I created a pipeline that strings multiple programs together, unfortunately these programs are creating a huge amount of temporary files in the /tmp
folder and when using large datasets my pipeline crashes because the /tmp
folder fills up.
How do I export temporary files so that they are created in my current working directory where the pipeline is being run and not in the /tmp
folder?
Currently I have tried to export the TMPDIR
env variable to an already created directory /work
in my current working directory, but the temporary files are still being created in the /tmp
folder:
export TMPDIR=$(mktemp -d --tmpdir=/work)
<script>
rm -rf $TMPDIR
The programs do not have the option to set different output folders for temporary files created.
Just change
/work
towork
if the directorywork
is in your current directory./work
means that you have a top-level directory named/work
. Without the forward slash, it will be a relative directory.I just tested this code on my computer. No files were written to
/tmp
that I noticed: