Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parallel processing blocked #86

Open
MattWellie opened this issue Apr 12, 2022 · 1 comment
Open

Parallel processing blocked #86

MattWellie opened this issue Apr 12, 2022 · 1 comment

Comments

@MattWellie
Copy link

I've run into an issue trying to parallelize code using Peddy. I'm parsing a PED file and VCF, then splitting all variants into groups to process. This splitting makes it a strong candidate for parallelization, but I can't pickle the Ped() object, so multiprocessing is blocked.

_pickle.PicklingError: Can't pickle <class 'peddy.peddy.UNKNOWN'>: it's not the same object as peddy.peddy.UNKNOWN

Probably relating to the handling of unknown members in the Pedigree:
https://github.com/brentp/peddy/blob/master/peddy/peddy.py#L102-L104

This is completely non-urgent, and I'll see if I can work out a fix which can be ported upstream

@brentp
Copy link
Owner

brentp commented Apr 13, 2022

It might be because of this: https://github.com/brentp/peddy/blob/master/peddy/peddy.py#L133

Instead of working on peddy, you might try somalier it's faster and scales to more samples.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants