Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Memory resource constraints are not passed to IQ-TREE (via augur tree) #949

Open
AdrianLimaG opened this issue May 17, 2022 · 3 comments · May be fixed by #953
Open

Memory resource constraints are not passed to IQ-TREE (via augur tree) #949

AdrianLimaG opened this issue May 17, 2022 · 3 comments · May be fixed by #953
Labels
bug Something isn't working

Comments

@AdrianLimaG
Copy link

Current Behavior
When runing nextstrain build with the following command

nextstrain build . --configfile build.yaml --cores all --resources mem_mb=5000

Everything works up until the iqtree2 command is run. Here my OS kills the operation due to its memory limitations.
My current set up has 15gb of ram and when I check the "dmesg" I see the iqtree was using around 19gb of virtual memory which is why the OS killed it.

I am just confused as to why iqtrees is trying to use this much memory when i defined in the resource for snakemake to use only 5gb. It seems as if snakemake is not passing this down.

Have i defined the resources properly? Are there any other steps i can take to prevent this from happening?

image001
@AdrianLimaG AdrianLimaG added the bug Something isn't working label May 17, 2022
@tsibley
Copy link
Member

tsibley commented May 24, 2022

@AdrianLimaG Snakemake's resource settings help Snakemake's scheduler decide what rules can run simultaneously, but they don't automatically enforce limits on what the rule actually uses. It's left to the rule in question to pass through resource constraints to the commands it's running and for the commands to respect those. In this case, I don't think the rule running IQ-TREE is doing that, so IQ-TREE doesn't get the memo and defaults to using all available RAM (which seems insufficient for the size of the tree/alignment).

@tsibley tsibley changed the title Snakemake not passing down resource constrains Memory resource constraints are not passed to IQ-TREE (via augur tree) May 25, 2022
@tsibley
Copy link
Member

tsibley commented May 25, 2022 •
edited

While I originally didn't think there was a way to ask IQ-TREE to limit its memory usage, looking again just now to confirm I see this in iqtree --help:

--mem NUM[G|M|%]     Maximal RAM usage in GB | MB | %

So this is something we could update our Snakemake rule tree to do.

tsibley added a commit that referenced this issue May 25, 2022
@tsibley
Pass the tree rule's mem_mb through to IQ-TREE
acb198e 
While most of our mem_mb definitions are only heuristics for Snakemake's
scheduler and the commands themselves aren't limited or aware of the
mem_mb defined, IQ-TREE *does* support memory limits.

Resolves <#949>.
@tsibley tsibley linked a pull request May 25, 2022 that will close this issue
Open
3 tasks
@tsibley
Copy link
Member

tsibley commented May 25, 2022

Possible fix in #953.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
No open projects
Nextstrain planning (archived)
Status: In Review
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Memory resource definitions nextstrain/ncov
2 participants
@tsibley @AdrianLimaG