BTrees 4.4.1 manylinux wheels broken for Python 2.7 - no C extension

[jamadden]

Installing the BTrees 4.4.1 manylinux wheels for Python 2.7 doesn't get you the C extension.

This shows up as a build failure by anything that needs the C extension. Notably, because of zopefoundation/zope.security#20, security checks on iter(tree.items()) fail.

These wheels should probably be deleted from PyPI.

(Unfortunately they can't be re-uploaded again.)

[jamadden]

Hmm, so it looks like the problem may lie in persistent's wheels:

$ pip install --only-binary :all: BTrees
Collecting BTrees
  Downloading BTrees-4.4.1-cp27-cp27mu-manylinux1_x86_64.whl (1.3MB)
    100% |████████████████████████████████| 1.3MB 428kB/s
Collecting zope.interface (from BTrees)
  Downloading zope.interface-4.4.0-cp27-cp27mu-manylinux1_x86_64.whl (171kB)
    100% |████████████████████████████████| 174kB 516kB/s
Collecting persistent (from BTrees)
  Downloading persistent-4.2.4.1-cp27-cp27mu-manylinux1_x86_64.whl (180kB)
    100% |████████████████████████████████| 184kB 612kB/s
Requirement already satisfied: setuptools in ./lib/python2.7/site-packages (from zope.interface->BTrees)
Installing collected packages: zope.interface, persistent, BTrees
Successfully installed BTrees-4.4.1 persistent-4.2.4.1 zope.interface-4.4.0
(tmp-349cf9efe9ff94dd)adminuser@adminuser-VirtualBox:~/.virtualenvs/tmp-349cf9efe9ff94dd$ python
Python 2.7.12 (default, Nov 19 2016, 06:48:10)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import BTrees._OOBTree
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: /home/adminuser/.virtualenvs/tmp-349cf9efe9ff94dd/local/lib/python2.7/site-packages/persistent/cPersistence.so: undefined symbol: PyUnicodeUCS2_AsEncodedString

I wonder if the build and dist directories aren't being cleaned out between builds in the docker image? That causes the wrong binaries to get included, since the build directories don't have the wideness flags. I ran into that in gevent. If that's the case then all the recent wheels are probably bad for Python 2.7 and should be removed.

cc @fgregg @tseaver

[fgregg]

Argh! That sounds like likely cause. Do you find a fix for gevent?

[fgregg]

@jamadden I think you are right that the problem is only in persistent

This successful build used the manylinux BTree, but built persistent and zope.interface from source
https://travis-ci.org/dedupeio/dedupe/jobs/224459476

[jamadden]

gevent's manylinux script is not sophisticated (I don't run it on travis, just locally). I fixed this issue by having it simply delete those directories.

[fgregg]

I built the wheels locally using the script amended by zopefoundation/persistent#64 and can confirm a fix.

[jamadden]

The PRs all look good to me. I don't have PyPI rights to any of the projects to do a release or remove the broken persistent wheel, though.

[mgedmin]

Just FYI I don't have PyPI rights to persistent or BTrees either. Ping @tseaver or @jimfulton.

[jimfulton]

@jamadden @mgedmin pls remind me what your pypi ids are and what projects you need rights to.

[jamadden]

@jimfulton It's jamadden and mgedmin. Right now the projects that need releases are persistent, BTrees, and zope.interface. (zope.index doesn't yet have a manylinux release, but it will need one soon.)

[jimfulton]

OK, you're both owners of all 4. (Marius was already an owner of the zope packages.

…

On Sun, Apr 23, 2017 at 12:41 PM, Jason Madden ***@***.***> wrote: @jimfulton <https://github.com/jimfulton> It's jamadden and mgedmin. Right now the projects that need releases are persistent, BTrees, and zope.interface. (zope.index doesn't yet have a manylinux release, but it will need one soon.)

[jamadden]

Thanks. I've removed the broken persistent 2.7 wheels from the existing release, and just released 4.2.4.5 so that new wheels get built with the fix.

[jamadden]

This issue seems to be addressed with the new wheels so I'll close.