request for enhancement of operational readiness

[cherusk]

I'm intending to include deap as a backend for the godon project.

I've only found references to scoop for operationally parallelizing the framework.

Hence the question, are there near or mid term plans on your end to extend the set of supported orchestration engines? Examples could be docker swarm, kubernetes or dask.

If not, could you give me pointers of what are be stumbling blocks to do so?

[cherusk]

Citing https://github.com/markcoletti:

"I see that you are interested in a distributed EA framework, which is a functionality that DEAP currently does not have. (And which is a frequently requested feature for DEAP.) You might be interested in Library for Evolutionary Algorithms in Python (LEAP) as it natively supports Dask for distributed evaluations. It has been successfully deployed at scale on the Summit supercomputer. Full disclosure: I am one of the LEAP developers. You can find LEAP here: https://github.com/AureumChaos/LEAP/tree/develop"

[cherusk]

LEAP looks promising.

As a terrific reference, the optuna framework dev team does offer a great option for dask parallelization integration, even part of their latest pre-release by now:

https://github.com/optuna/optuna-examples/blob/main/dask/dask_simple.py