Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rq-py3.4 not compatible with rq-py2.7 due to pickle protocol version #598

Closed
glaslos opened this issue Nov 3, 2015 · 4 comments
Closed

Comments

@glaslos
Copy link
Contributor

glaslos commented Nov 3, 2015

I am playing around with workers with different Python versions. One of the pitfalls is the pickle protocol version used. For python3.4 rq uses the highest available protocol version (4) but python2.7 only understand up to 2. This can be fixed by using a custom Job class and forcing python3.4 to use protocol version 2 but it's rather awkward.
I managed to use a custom Job class (overwriting the dumps method) to overcome this issue but exposing this as a flag might be more convenient.

@ldath
Copy link

ldath commented Nov 5, 2015

👍

@selwin
Copy link
Collaborator

selwin commented Nov 6, 2015

There's an issue for implementing a customizable args/kwargs serializer. I think that's the better approach for this particular problem :)

@selwin selwin closed this as completed Nov 6, 2015
@selwin
Copy link
Collaborator

selwin commented Nov 8, 2015

Here's a link to the issue I mentioned: #369 . I forgot to attach it when I closed this issue, sorry.

@bivald
Copy link

bivald commented Nov 27, 2015

As a side note, while not particularly pretty, this appears to actually work:

import pickle
pickle.HIGHEST_PROTOCOL = 2
from rq import Queue 
# from here on RQ will use pickle protocol 2

At least when using python 3.4 to queue the job, and 2.7 to read it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants