Use cross-compilation for CI, build the Docker image from source.

[33KK]

• Removed outdated actions from actions-rs
• All the builds are now done natively using cross-compilation
• Dockerfile doesn't download binaries anymore
• Added nightly release, though that may need tweaking, since it just updates the old release on each run, and that doesn't update the release date
• Docker images also build for latest commit under the edge tag
• The Docker build isn't split into two matrix jobs, it takes about 40 minutes to run the two builds sequentially, not sure if it's worth adding all that image merging stuff back

Also includes the opentelemetry update from #57 since it's necessary for this to build. This should be easy enough to adapt for the standalone JMAP/IMAP/SMTP servers.

[CLAassistant]

All committers have signed the CLA.

[mdecimus]

Thanks for this, great work! Were you able to cross compile the musl targets as well as the Windows arm64 binaries?

[33KK]

I haven't tried, I'll look into it

[33KK]

aarch64 Windows doesn't build due to this, that will have to wait until ring 0.17: briansmith/ring#1167

[33KK]

aarch64-unknown-linux-musl built just fine on Arch Linux, fails on Ubuntu though 🤷

[33KK]

aarch64-unknown-linux-musl built just fine on Arch Linux, fails on Ubuntu though 🤷

x86_64-unknown-linux-musl seems to build just fine, couldn't get aarch64-unknown-linux-musl working though, not sure why, works on my machine ™️ Not sure what's different with Arch's musl packages, maybe they include arm64 builds or something, I tried installing musl:arm64 in github actions but it didn't work either

[33KK]

Actually no, it also fails with the same error on Arch, I was building the x86 musl target instead of aarch64

[mdecimus]

I'll merge your changes once I am ready to launch v0.4 in about 4 to 6 weeks. I need to test the cross-compiled binaries properly. Whenever you have a chance please remove Cargo.lock and crates/utils/Cargo.toml from the PR to resolve the conflicts. Thanks!

[williamdes]

💯

[williamdes]

I did some change on my version to allow using from source releases

FROM docker.io/debian:bookworm-slim AS source

WORKDIR /source

ARG VERSION="0.3.8"
ADD https://github.com/stalwartlabs/mail-server/archive/refs/tags/v$VERSION.tar.gz /tmp/v$VERSION.tar.gz
RUN tar --strip-components=1 -C /source -xzf /tmp/v$VERSION.tar.gz

And then

- COPY . .
+ COPY --from=source /source .

Maybe it's worth having one step only for the source

[33KK]

I did some change on my version to allow using from source releases

I think its better to just checkout the version tag in git

[33KK]

Not sure how to get musl cross-compile targets working, probably would need to build musl with musl-cross-make or use https://github.com/rust-cross/rust-musl-cross or something

[williamdes]

I did some change on my version to allow using from source releases

I think its better to just checkout the version tag in git

Yeah but you do not want to have the burden of git all of such stuff just to build an image. And having to clone. Etc..

Only from source ensures you can build the image from a production server without too much tooling
Just Docker, yay!

[nka11]

I did some change on my version to allow using from source releases

I think its better to just checkout the version tag in git

Yeah but you do not want to have the burden of git all of such stuff just to build an image. And having to clone. Etc..

Only from source ensures you can build the image from a production server without too much tooling Just Docker, yay!

But what if I want to build from the current working copy that is not a release ?

This contribution, as it is, makes it easy for me to contribute, just because I want to build a image and test it against a random commit, or even a dirty working copy, and not from a registered tag released on github.

Besides, I see an anti-pattern here :
A production server is not meant to build a docker image, unless it hosts a CI/CD agent which will build from the git repo and push to a registry.
The production server needs only to pull the image and run it.

EDIT: And the dockerfile is in the project git's repository, it then sounds very natural to git clone and checkout the desired tag to get the proper Dockerfile and related code version...

[williamdes]

But what if I want to build from the current working copy that is not a release ?

Then only use COPY . . and still keep an easy step for everybody

Look in this example how it was easy to apply patches on the source since I have it in a single step: https://github.com/wdes/mails.wdes.eu/blob/dfefb2ca9e70aa59409180b583e334c42fc60356/docker/Dockerfile#L4-L21

This contribution, as it is, makes it easy for me to contribute, just because I want to build a image and test it against a random commit, or even a dirty working copy, and not from a registered tag released on github.

As I said, all approaches can remain, but using one single layer for the source helps everyone build the final image

Besides, I see an anti-pattern here :
A production server is not meant to build a docker image

yes and no, sometimes you need to build from source. Hello images not cross arch ^^
I think this point is mostly theorical best practice.

[33KK]

I don't think the Dockerfile in the root of the repo should worry about downloading any code. Maybe a separate Dockerfile.release or something like that, or even outside of the repo. If the Dockerfile is straight in the root of the repo I'd only ever expect it to COPY . .

[33KK]

Well, the musl target seems to just work with cross, aarch64 windoze should also work but ring needs to be updated to 0.17, ldap3 apparently still depends on ring 0.16

[33KK]

That should be fixed when this is merged inejge/ldap3#116

[33KK]

aarch64 mac build seems to have just worked

[mdecimus]

Hi, thanks for all the work! Just so you know, I am currently working on version 0.5.0 which should be released in 2 or 3 weeks. This version will include:

• Support for mySQL and PostgreSQL backends. Soon after that, support for Cassandra, ScyllaDB and maybe also SurrealDB will be added.
• It will be possible to compile Stalwart with support for all databases. Currently it is either SQLite or FoundationDB.
• For single-node deployments, support for RocksDB will be re-introduced (as it is much faster than SQLite) which might make cross-compilation tricky.

So please do not spend too much time fixing the CI jobs as they will probably need to be tweaked for 0.5.0. Thanks!

[33KK]

Ok I give up on aarch64-windoze for now, hickory-resolver didn't yet publish a version that doesn't depend on ring 0.16, they did bump it in the repo though.

So please do not spend too much time fixing the CI jobs as they will probably need to be tweaked for 0.5.0. Thanks!

As long as the database drivers aren't using native libraries, it should work fine /shrug. RocksDB is vendored in the crate, only FoundationDB seems to be particularly painful to deal with.

[33KK]

https://github.com/33KK/mail-server/actions/runs/6955297536/job/18923787212 not sure whats going on here, whatever, will deal with this later i guess

[mdecimus]

https://github.com/33KK/mail-server/actions/runs/6955297536/job/18923787212 not sure whats going on here, whatever, will deal with this later i guess

I've just fixed this on the development branch. It seems that one of those targets is a 32 bit platform and usize is 32 bit long rather than 64.

[mdecimus]

I did some tests on v0.5.0 before merging this PR but cross compiling RocksDB seems to be broken:

$ RUSTFLAGS="-C linker=aarch64-linux-gnu-gcc" cargo build --release --target aarch64-unknown-linux-gnu --no-default-features --features "sqlite postgres mysql rocks elastic s3 redis"
   Compiling librocksdb-sys v0.11.0+8.1.1
error: failed to run custom build command for `librocksdb-sys v0.11.0+8.1.1`

Caused by:
  process didn't exit successfully: `/home/a/mail-server/target/release/build/librocksdb-sys-104455b5f5c54342/build-script-build` (exit status: 101)
  --- stderr
  /usr/include/stdint.h:26:10: fatal error: 'bits/libc-header-start.h' file not found
  thread 'main' panicked at /home/a/.cargo/registry/src/index.crates.io-6f17d22bba15001f/librocksdb-sys-0.11.0+8.1.1/build.rs:40:10:
  unable to generate rocksdb bindings: ClangDiagnostic("/usr/include/stdint.h:26:10: fatal error: 'bits/libc-header-start.h' file not found\n")
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...

There is an open issue at the Rust-RocksDB repository regarding this issue but it seems that the devs behind Fuel seem to have found a workaround to be able to cross compile it.

There's no rush to fix this, we'll continue using QEMU to compile aarch64 in the meantime.

Edit: I was using cargo instead of cross. After switching tocross it looks like clang needs to be updated:

error: failed to run custom build command for `librocksdb-sys v0.11.0+8.1.1`

Caused by:
  process didn't exit successfully: `/target/release/build/librocksdb-sys-104455b5f5c54342/build-script-build` (exit status: 101)
  --- stderr
  rocksdb/include/rocksdb/c.h:45:9: warning: #pragma once in main file [-Wpragma-once-outside-header]
  rocksdb/include/rocksdb/c.h:65:10: fatal error: 'stdarg.h' file not found
  thread 'main' panicked at /cargo/registry/src/index.crates.io-6f17d22bba15001f/clang-sys-1.6.1/src/lib.rs:1735:1:

  A `libclang` function was called that is not supported by the loaded `libclang` instance.

      called function = `clang_getTranslationUnitTargetInfo`
      loaded `libclang` instance = 3.8.x

  This crate only supports `libclang` 3.5 and later.
  The minimum `libclang` requirement for this particular function can be found here:
  https://docs.rs/clang-sys/latest/clang_sys/clang_getTranslationUnitTargetInfo/index.html

  Instructions for installing `libclang` can be found here:
  https://rust-lang.github.io/rust-bindgen/requirements.html

  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...

[33KK]

RocksDB fails to link now, no idea why. May be relevant https://gitlab.com/famedly/conduit/-/merge_requests/261/diffs

[mdecimus]

Hi @33KK ,

I finally "merged" your CI job and Dockerfile. I say "merged" because there were multiple conflicts after the 0.7.0 release so I manually copied, adapted your changes and credited you in the files that changed.

Overall the CI jobs great, only the arm targets fail to compile but this is not urgent. Version 0.7.1 has been released using the new job.

Thank you!