Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

About no space on /tmp #163

Open
wants to merge 6 commits into
base: gh-pages
Choose a base branch
from
Open

About no space on /tmp #163

wants to merge 6 commits into from

Conversation

manabuishii
Copy link
Contributor

No description provided.

This kind of error is happens mainly two cases.

1. tool using huge amount under `/tmp` .
2. You run on Mac.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain more and give and example below on how to fix if on a Mac?

@stain
Copy link
Member

stain commented Sep 11, 2019

Thanks for the addition, @manabuishii !

To be honest I have never understood the difference between --tmp-outdir-prefix and --tmpdir-prefix - should we try to explain that?

I am not sure how this would help sort which would instead need a different TMPDIR shell environment variable within the VM - e.g. to use /var/tmp

TMPDIR could be sent through with TMPDIR=/var/tmp cwltool --preserve-environment TMPDIR foo.cwl although it would be a different folder if the tool is run inside VM/Docker.

If sort etc are run within an Docker image, then /tmp could either be part of the differential file system (fine) or (if the Dockerfile or docker command so declares) a volume (also fine). I am not sure how running cwltool on the OSX-side would ever use /tmp within the Docker VM as it would be as part of a docker run command that mounts volumes from the host for inputs and outputs, and which uses whatever docker image is provided in DockerRequirement?

You could override TMPDIR for tmp-hungry tools within the cwl file in a EnvVarRequirement - e.g.:

requirements:
  EnvVarRequirement:
    envDef:
      TMPDIR: $(runtime.outdir)

This should in effect use --tmp-outdir-prefix as that is what cwltool has mapped in for outputs - but this would also be more portable across engines and deployments. You are however at mercy of the tool not overwriting the actual outputs and should not have over-inclusive glob patterns.

@swzCuroverse
Copy link
Contributor

@kinow do we want to rework this into the new user guide? I am not sure of best process - but it would nice to clean out these merge requests that are for the old format of the user guide. Let me know what you think is best here.

@kinow
Copy link
Member

kinow commented Oct 15, 2022

Hi @swzCuroverse I had a read and the advice from @manabuishii in this PR looks useful. Not sure if anything has changed in cwltool, but if it's still correct, then maybe @manabuishii or somebody else could take a look if it can/needs be updated after @stain 's comments, and then review/merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants