Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Move Policy execution into remote process/thread #455

Open
CoolRunning opened this issue Dec 3, 2020 · 0 comments
Open

[FEATURE] Move Policy execution into remote process/thread #455

CoolRunning opened this issue Dec 3, 2020 · 0 comments

Comments

@CoolRunning
Copy link
Collaborator

Is your feature request related to a problem? Please describe.
Currently, archipelago evolution has a bottleneck since selection and migration policies are executed in the main thread. This is barely noticeable for the simple default selection and replacement policies, as they are very fast. More sophisticated strategies with a non-negligible overhead deployed to larger archipelagos (>100 islands) constitute a scalability problem though.

Describe the solution you'd like
To improve scalability, execution of selection and replacement policies should be moved inside the remote process/thread, i.e. handled on island level and not on archipelago level.

Describe alternatives you've considered
Depending on the selection and replacement logic, some of it could be moved as a workaround inside the evolve-method of the UDA, which is parallelized (i.e. flagging certain individuals during evolution so that the selection just needs to pick them up). However, this is not a very clean solution (breaks the semantics of evolve) and might not work in all cases. Using parallelization (like multi-processing) in user defined policies could maybe work as a speed up in some cases, but would not change the fact that multiple policy exeuctions are still executed back-to-back, producing a potentially large overhead for spawning new threads/processes every time. It would be more efficient and clean to use the already established parallelization structure of the archipelago for this task.

Additional context
This feature request came into existence while discussing if there was an "easy" way to re-evaluate migrants upon entering a new population? Current pagmo behavior copies the fitness vector of an individual over, which is fine as long as the udps of source and target island do not differ. Migrating between islands with "inconsistent" udps however require a reevaluation of individuals, which is doable by a user-defined replacement function, as long as it has access to its islands fitness function. More sophisticated migration strategies (i.e. replacement by "merging" individuals and migrants) might be another use-case where a fitness evaluation within a policy could be desirable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant