Replies: 1 comment 2 replies
-
I see what you are saying, but at the moment I still prefer to keep all the chains in a single target. Advantages:
Perhaps this is an issue to raise with the |
Beta Was this translation helpful? Give feedback.
2 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
As I understand it, the following runs 4 chains on a single machine, each with 4 threads - 16vcpu machine in total, or 32vcpu if not using hyperthreading.
Occasionally this will leave 24vcpus of a 32vcpu machine sitting idle for a few hours while a single slow chain finishes.
When using transient workers via
future.batchtools
it'd be better to release these unnecessary vcpus.Ideally it'd be submitted as an array job, but it looks like future.batchtools doesn't support this?
A nice-to-have workaround would be for
tar_stan_mcmc
to have an option which would internally parallelize the mcmc target to run single chains across 4 targets then combine them again before running the summary, draws & diagnostics jobs.Can you suggest any workaround within stantargets?
Thanks
Beta Was this translation helpful? Give feedback.
All reactions