Multiarch builds

[martinfleis]

The container currently doesn't run on M1 Macs and cannot even be built for osx-arm64 architecture as it is. I have initiated conda-forge builds for both osx-arm64 (conda-forge/conda-forge-pinning-feedstock#2161) that is needed to install packages natively via conda and linux-aarch64 (conda-forge/conda-forge-pinning-feedstock#2167) that is used within arm-based container. Once that work is done (it may take a while as not all libraries can be compiled without issues) we may consider building the container for both architectures, similarly to what jupyter/minimal-notebook does. Here's a guide https://www.docker.com/blog/multi-arch-build-and-images-the-simple-way/.

Some of the libraries that do not come from conda-forge at the moment can be installed via pip, which compiles them on demand, but I am not sure if all would work (probably not).

[darribas]

Here's a current strategy that lets you run a container built for Intel on Apple silicon:

https://tongfamily.com/2021/12/21/docker-on-apple-m1-break-on-jupyter-notebooks-switch-to-podman/

@martinfleis has tried it and confirmed it worked.

It's not ideal but gets around for the time being.

[darribas]

Adding as we start readying for 8.0, the official Jupyter stack discussed move to cross-chip support at docker-stacks#1019 and currently supports builds for arm64 (e.g., M1) withing the minimal-notebook image the gds_env relies on.

Currently, the build command is:

https://github.com/jupyter/docker-stacks/blob/8be708bd9c072a67d3180a21235d4c44c8070e0b/Makefile#L100-L110

[darribas]

Quick update that, on a M2 Pro chip, the 9.0 container seems to (mostly) work under Rosetta. This needs to be turned on explicitly in Docker for Mac's advanced options.

In terms of performance, all Python libraries seem to work fine (e.g., the GDS Book completes fully), but some of the dev utilities in gds_dev raise issues.

For folks wanting to install the Python stack natively, there is now an explicit file at: https://github.com/darribas/gds_env/blob/master/gds_py/gds_py_explicit_macos-arm.txt

[martinfleis]

We could build an arm version of the container, no? I think that Jupyter containers have both versions.