Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chunked particle data evaulation #115

Open
skuschel opened this issue Nov 14, 2017 · 0 comments
Open

Chunked particle data evaulation #115

skuschel opened this issue Nov 14, 2017 · 0 comments

Comments

@skuschel
Copy link
Owner

skuschel commented Nov 14, 2017

Particle Data may be carried out in chunks of a few million particles. This may have several advantages:

  • If supported by the reader the data can be read chunk by chunk and therefore I would allow to process more particle than can fit into the computers memory. Requires heavy testing to ensure its working properly.
  • Probably speeding up particle processing, as one chunk can be processed while another is read.
  • The proposed change also provides the prerequisite to parallelize the particle data processing over several machines -- for example using ipyparallel.
  • As this involves a possible restructuring of the project it has to be discussed before v1.0
@skuschel skuschel added Field core routines affected low priority labels Nov 14, 2017
@skuschel skuschel added this to the v1.0 milestone Nov 15, 2017
@skuschel skuschel modified the milestones: v1.0, v0.4 Nov 22, 2017
@skuschel skuschel self-assigned this Jan 11, 2018
@skuschel skuschel removed the Field core routines affected label Jan 30, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

No branches or pull requests

1 participant