Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

niak clogs /tmp and kills the OS #391

Open
bpinsard opened this issue Aug 29, 2019 · 4 comments
Open

niak clogs /tmp and kills the OS #391

bpinsard opened this issue Aug 29, 2019 · 4 comments

Comments

@bpinsard
Copy link
Member

niak puts a lot of stuff in /tmp and never clear it up...
From looking into the code, the place for tmp is set by TMPDIR env variable. This should be documented to help people to place temp files in an appropriate location (eg. /scratch/).

@pbellec
Copy link
Member

pbellec commented Aug 29, 2019

A niak run that completes should leave no trace in tmp.

If there is a crash, or many concurrent instances, it is very possible that this creates lots of left-over.
It is better practice to use scratch as a tmp indeed, or whatever dedicated space there is for there.
This is a standard configuration step on a HPC, and there is generally a recommended folder for it that needs to be specified.
This indeed highly needs to be better documented, and has been a major source of issues for people who tried to install niak.

Typically, that aspect of the configuration is done once and for all on a given niak install through the psom_gb_vars_local.m file, and specifically the value of psom_gb_tmp. psom_gb_vars_local.m is a copy of the following file, which is edited by the user to set her own local configurations (whatever psom_gb_tmp_local is found first in the path of a given octave session will superseed all other configuration files)
https://github.com/SIMEXP/psom/blob/master/psom_gb_vars.m#L70
ideally this parameter would be exposed to script users through opt.psom.tmp, but this is not currently implemented.

It should indeed be documented in http://psom.simexp-lab.org/psom_configuration.html and http://niak.simexp-lab.org/niak_installation.html
Another way to deal with this is to mount whatever tmp location should be used on the /tmp of the docker container.
This should be done in any case, because otherwise the container /tmp is an in-memory virtual folder, which gets clogged really fast and creates chaos for windows and mac users...

FYI if you feel inspired to try and start a PR, location for the niak install doc is:
https://github.com/SIMEXP/niak/blob/gh-pages/niak_installation.md
and PSOM config doc is there:
https://github.com/SIMEXP/psom/blob/gh-pages/psom_configuration.md

@pbellec
Copy link
Member

pbellec commented Aug 29, 2019

Another note: it may also be that the current niak install does leak files in the tmp.
I have tested periodically it's not the case, but it's not part of our automated testing.
If you can confirm such a leak, it would be appreciated.
I'll create a separate issue to add this in the niak tests.

@HanadS
Copy link
Member

HanadS commented Aug 30, 2019

found some documentation on this
http://psom.simexp-lab.org/how_to_use_psom.html#misc-temporary-files

@pbellec
Copy link
Member

pbellec commented Aug 31, 2019

Thanks for pointing out @HanadS! It's not the best place for it, but at least it exists

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants