Skip to content
This repository has been archived by the owner on Dec 22, 2021. It is now read-only.

Parallel tesseroid computation doesn't work with generators #295

Open
leouieda opened this issue Jul 14, 2016 · 1 comment
Open

Parallel tesseroid computation doesn't work with generators #295

leouieda opened this issue Jul 14, 2016 · 1 comment
Labels
Milestone

Comments

@leouieda
Copy link
Member

Parallel execution of fatiando.gravmag.tesseroid functions doesn't work on generators. For example:

def modelgen(lat, lon, topo, step):
    for i in range(topo.size):
        t = Tesseroid(w=lon[i] - step/2,
                      e=lon[i] + step/2,
                      s=lat[i] - step/2,
                      n=lat[i] + step/2,
                      top=topo[i],
                      bottom=0,
                      props={'density': 2670})
        yield t

model = modelgen(lat, lon, topo, step)
gz = tesseroid.gz(lat, lon, height, model, njobs=8)

will fail because multiprocessing can't pickle the generator object.

A possible way around this is to use a better multiprocessing alternative (toolz, joblib, dask).

@leouieda leouieda added the bug label Jul 14, 2016
@leouieda leouieda added this to the 0.6 milestone Jul 14, 2016
@eusoubrasileiro
Copy link
Contributor

just following your effort on using multiprocessing instead of cython

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

2 participants