Azure AD Auth fails on ARM64

[george-zubrienko]

Environment

Amazon Linux 2023, ARM64 arch host, container based on python3.11-slim-bookworm

Delta-rs version:

0.17.2

Binding:

python

Environment:

• Cloud provider: AWS
• OS: AL2023
• Other: Python3.11

---

Bug

What happened:

When trying to load the table using AZURE_CLIENT_ID etc credentials, getting this:

OSError: Generic MicrosoftAzure error: Error performing token request: Error after 10 retries in 2.050817107s, max_retries:10, retry_timeout:180s, source:error sending request for url (https://login.microsoftonline.com/.../oauth2/v2.0/token): error trying to connect: failed to get random bytes

Stack trace:

What you expected to happen:

Table reads as before on 0.8.1 version on the same host/container

How to reproduce it:

Run table read on ARM64/AL2023 vm with Azure auth against az://... table path

More details:
N/A

[ion-elgreco]

@george-zubrienko are you able to test the connection from non-arm environments?

[george-zubrienko]

@ion-elgreco yes if I change machine type to amd64 it works fine. Fun part, people on Mac M2's do not have this issue

[george-zubrienko]

I'm building a new image rn with 0.17.3, fyi this was the wheel used by ARM64 build: deltalake-0.17.2-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl

Same result with 0.17.3. I assume the error is coming from Rust code, not python as it is reported as OSError

[ion-elgreco]

I'm building a new image rn with 0.17.3, fyi this was the wheel used by ARM64 build: deltalake-0.17.2-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl

Same result with 0.17.3. I assume the error is coming from Rust code, not python as it is reported as OSError

Could you do a bisect, to see which release this problem started occurring for you?

[george-zubrienko]

I'm building a new image rn with 0.17.3, fyi this was the wheel used by ARM64 build: deltalake-0.17.2-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Same result with 0.17.3. I assume the error is coming from Rust code, not python as it is reported as OSError

Could you do a bisect, to see which release this problem started occurring for you?

We just upgraded from 0.8.1 to 0.17.* and I got the error. I can try to narrow it down a little. In case this adds anything, we are loading data like this: https://github.com/SneaksAndData/adapta/blob/main/adapta/storage/delta_lake/_functions.py#L39-L90

[george-zubrienko]

@ion-elgreco it works up to 0.16.1, 0.16.2 breaks it

[ion-elgreco]

@george-zubrienko hmm strange, nothing implies there is a change that could have caused this in that release.

Maybe its the rust version which it gets compiled with, are you able to compile with some older version and check that

[george-zubrienko]

@george-zubrienko hmm strange, nothing implies there is a change that could have caused this in that release.

Maybe its the rust version which it gets compiled with, are you able to compile with some older version and check that

Arent 0.16.2 and 0.16.1 compiled with the same version? I can try to force this package to compile from source rather than use the wheel

[george-zubrienko]

@ion-elgreco

So, using this compiler version:

# rustc --version
rustc 1.78.0 (9b00956e5 2024-04-29)

Running this:

# pip install --upgrade deltalake==0.17.2 --no-binary :all:
Collecting deltalake==0.17.2
  Using cached deltalake-0.17.2.tar.gz (4.8 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: pyarrow>=8 in /usr/local/lib/python3.11/site-packages (from deltalake==0.17.2) (16.0.0)
Requirement already satisfied: pyarrow-hotfix in /usr/local/lib/python3.11/site-packages (from deltalake==0.17.2) (0.6)
Requirement already satisfied: numpy>=1.16.6 in /usr/local/lib/python3.11/site-packages (from pyarrow>=8->deltalake==0.17.2) (1.26.4)
Building wheels for collected packages: deltalake
  Building wheel for deltalake (pyproject.toml) ... |yrtydone
  Created wheel for deltalake: filename=deltalake-0.17.2-cp38-abi3-linux_aarch64.whl size=27450351 sha256=1c47967d8c6cd74a7414607321671e8c5fb093b6767035bdffc5562c95014f33
  Stored in directory: /root/.cache/pip/wheels/cf/6f/ad/d4179379730a3e8649079d3050ccba5bdf1ccad191074b0027
Successfully built deltalake
Installing collected packages: deltalake
  Attempting uninstall: deltalake
    Found existing installation: deltalake 0.17.3
    Uninstalling deltalake-0.17.3:
      Successfully uninstalled deltalake-0.17.3
Successfully installed deltalake-0.17.2

Everything works. So it is a problem with the wheel?

Kernel: Linux 6.1.84-99.169.amzn2023.aarch64 #1 SMP Mon Apr 8 19:19:24 UTC 2024 aarch64 GNU/Linux
OS: Debian 12 (bookworm)

[ion-elgreco]

@george-zubrienko ok that's helpful!

I think it's because of the rust compiler.

• V0.16.1 was compiled with rust 1.76.0
• v0.16.2 until v0.17.3 is compiled with 1.77.x

You seem to get it working when it's compiled with 1.78.0

[george-zubrienko]

That is the case! Any chance we can get compiler bumped for 0.17.4? 🙏

[ion-elgreco]

That is the case! Any chance we can get compiler bumped for 0.17.4? 🙏

Probably next release. Rust 1.78 got released today and our release just missed that version

[gaurav7261]

I'm also facing same issue with aws sts, using latest deltalake version

[gaurav7261]

@ion-elgreco tag me here once you release the latest version, right now we are reverting back to amd

[rtyler]

I would be surprised if this was the compiler. It is more likely that a dependency for which we have a loose version range specifier. Either way, I figure the next release should take care of this for ya 😄

[gaurav7261]

@rtyler we are also facing same with aws as well