Missing Linux AArch64arm64 wheels on PyPi

[odidev]

Expected behavior and actual behavior.

Missing Linux AArch64 wheels on PyPi. The command: pip install fiona should be able to download the wheel from PyPi and install it.
Currently pip tries to build the wheel from source code and fails.

Steps to reproduce the problem.

On Linux AArch64, run: pip install fiona

Operating system

Linux

Fiona and GDAL version and provenance

Not applicable

[odidev]

Can you please let me know which CI/CD you are using to build and deploy the wheels. I found a repository but it look to be outdated now. Can you please point out the current wheel builder?

Or, you are building and uploading the wheels manually?

[odidev]

@sgillies, can you please let me know your interest and point out the wheel builder?

Building and uploading wheel on PyPi can be useful as -

1. The Arm architecture has been available on AWS (Graviton2) for about 2 years now and has seen a lot of adoption by various organizations. The install base of Arm users is growing at a rapid pace.
2. Oracle cloud will be releasing Arm based Compute Shapes in the future. Please see here
3. Generally, there is a lot of momentum behind various products switching to Arm based CPUs. For example, the Microsoft surface Pro X is an Arm based device.

[odidev]

@sgillies, can you please let me know your thoughts on releasing Linux AArch64 wheels? Thanks

[thomasaarholt]

I second this desire. Currently one cannot install fiona (and hence e.g. geopandas) on arm linux or M1 macs without using anaconda. I would very much like to see wheels for these two archs.

[sgillies]

Fiona 1.8.21 wheels for x86_64 manylinux2014 and macos (10.10) only were built on GitHub actions using https://github.com/sgillies/fiona-wheels.

There are no current plans to build arm64 wheels for Linux or macOS. While I understand how great it would be to get software for free, that you don't even have to compile (better-than-free!), that you can run on, for example, fancy new AWS instances (not free), I believe arm64 wheels yield diminishing returns for the Fiona project. In other words, I don't see that this project would get much back in the way of bug fixes or other contributions in return for work in getting GDAL and all its dependencies to compile on what looks to me to be a production-only platform.

In the M1 (also not free!) case, we're mainly blocked by the lack of free CI with native M1 workers. Cross-compilation is, in theory, possible, and I've done it for the GEOS library, but there are a lot of unknowns for the deep stack of libraries that GDAL depends on.

For the time being: use packages from conda-forge. The conda-forge project's mission is to build packages. You can even sponsor the project with money to do it. The Fiona project is not currently on a mission to build packages for every available platform. This is a software code project, not a packaging or distribution project.

[ghost]

Support it, a lot of people use the arm64 architecture, at least China is currently the case.

[sgillies]

The comment this person left at OSGeo/gdal#7301 violates Fiona's code of conduct https://github.com/Toblerity/Fiona/blob/master/CODE_OF_CONDUCT.md and I have blocked them.

[rbebb]

@sgillies Is this something that's still being considered now that arm64 wheels are provided for macOS? Thank you for your work!

[sgillies]

@rbebb I found that Linux aarch64 wheels are not trivial. You can see the flailings and debris here: sgillies/fiona-wheels#35. I think you'll have to continue to work around, this project doesn't have the resources to nail down builds for every single platform.

[rbebb]

If it helps at all, Fiona does build on ARM Linux if you have gdal-bin, libgdal-dev, and g++ installed.

[rbebb]

It looks like the Linux ARM CI build can't find python3. I assume python3 needs to be symlinked to the Python executable like it is for the macOS ARM CI build. I believe the correct command for Python 3.8 would be ln -s python3 /opt/python/cp38-cp38/bin/python (and so on for the other Python versions).

[jorisvandenbossche]

I got wheel building for manylinux aarch64 working for pyogrio (an alternative GDAL interface, https://github.com/geopandas/pyogrio/). If this turns out to actually produce correct wheels, the workflow could be ported to fiona as well (similarly as was done for windows).

Now, I don't actually have an aarch64 machine to test, so if someone could give the wheels a test drive on their machine, that would be useful!
The wheels can be downloaded from https://app.circleci.com/pipelines/github/geopandas/pyogrio/28/workflows/58f0e3b6-eaa5-421b-98e1-782e1222f303/jobs/30/artifacts (the PR working on this: geopandas/pyogrio#243, so feedback can be provided there)

[rbebb]

@jorisvandenbossche, I assume you confirmed that the new CI job produced the correct wheels for pyogrio? I just saw that the PR was merged!