Move ARM build to Github Actions

[gaborznagy]

As we focus more on GitHub Actions, we could use a github-action for testing builds on ARM instead of Travis CI.
The github-action: https://github.com/uraimo/run-on-arch-action
It is using Docker + QEMU on Linux and it can build on armv7 and ppc64le too.

This PR migrates the current Travis config to the run-on-arch github-action.

Notes:

• contrary to Travis CI it's not a native ARM runner though, it uses QEMU,
• build time is slow: even after putting dependency installation (including locally-built ones),
  syslog-ng build is still around 27 minutes (Github runners have 2-core CPUs, but it was slow on my 4-core i7 CPU too)
• it requires a docker image to be built first, in order to run the commands itself.
  We can cache the docker images in the Github Packages (similarly to dbld images).
  image name is: run-on-arch-${GITHUB_REPOSITORY}-${GITHUB_WORKFLOW}-${arch}-${distro}
  The image is used as a cache for later runs, so any change in the image (e.g. new dependency), will trigger the docker build.

Alternative github-action:

• https://github.com/pguyot/arm-runner-action
  Similarly it can run ARM binaries with QEMU.
  Not docker-based, it uses a chroot environment and an image of the given distribution.
  It also creates an image that can be used on Raspberry PI.
  Currently it is limited to Raspberry PI.

Some example runs: https://github.com/gaborznagy/syslog-ng/actions/workflows/run-on-arm.yml

TODO:

• fix pip install (warnings about install path)

This is just an interesting idea, unfortunately the runtime is long. The PR is open to discussions. :)

[kira-syslogng]

Build SUCCESS

[kira-syslogng]

Build FAILURE

[Kokan]

lgtm
some additiona notes:

• it seems to me that caching docker image does not work: https://github.com/syslog-ng/syslog-ng/runs/4326195497?check_suite_focus=true#step:3:4037 also do we have information how to cache is invalidated ?
• the arm job is considered as optional, it would be nice if in case of failure it does not block merge.
  or maybe we could consider changing our view, and state that it is not optional.

[Kokan]

optional: that script from dbld could be used here directly

[gaborznagy]

I see the benefit of it: logic is not copied, if new Criterion is released it would available instantly, though it would require some changes to dbld: the scripts are expected to run inside the container: e.g. this is in dbld/builddeps:

. /dbld/functions.sh

[Kokan]

optional: we do not really required to do install, you can just skip that (some time can be spared there)

[gaborznagy]

true, I'll check it.

[kira-syslogng]

Build FAILURE

[MrAnno]

I think it should not be optional. ARM is a famous architecture, which is worth supporting/testing in the CI. We have also found general hard-to-catch bugs using an ARM32/64 environment. (Also, Travis was not configured with allow_failures either, we took that job seriously as far as I know.)

[gaborznagy]

lgtm some additiona notes:

• it seems to me that caching docker image does not work: https://github.com/syslog-ng/syslog-ng/runs/4326195497?check_suite_focus=true#step:3:4037 also do we have information how to cache is invalidated ?
• the arm job is considered as optional, it would be nice if in case of failure it does not block merge.
  or maybe we could consider changing our view, and state that it is not optional.

For security reasons pull requests don't get write-access to the base repository.

With the exception of GITHUB_TOKEN, secrets are not passed to the runner when a workflow is triggered from a forked repository. The permissions for the GITHUB_TOKEN in forked repositories is read-only.

https://docs.github.com/en/actions/learn-github-actions/events-that-trigger-workflows#pull_request

You can check my fork to see how caching works, here is a workflow that uses the cached image:
https://github.com/gaborznagy/syslog-ng/runs/4347340237?check_suite_focus=true
In case the run: option is changed, the image is not rebuilt:
https://github.com/gaborznagy/syslog-ng/runs/4351319125?check_suite_focus=true
Here is a test for image rebuild:
https://github.com/gaborznagy/syslog-ng/runs/4301298441?check_suite_focus=true

Image is rebuilt if the install option is changed (hence the docker image is changed too). In general, docker image rebuild is decided by docker build --cache-from=<image>.
Dockerfiles in run-on-arch github action consists of 3 lines, here is the Dockerfile for arch:aarch64, distro:bullseye:

FROM arm64v8/debian:bullseye    
    
COPY ./run-on-arch-install.sh /root/run-on-arch-install.sh    
RUN chmod +x /root/run-on-arch-install.sh && /root/run-on-arch-install.sh    

[gaborznagy]

I think docker logout is missing from run-on-arch:
https://github.com/gaborznagy/syslog-ng/runs/4301298441?check_suite_focus=true#step:3:79

See code:
https://github.com/uraimo/run-on-arch-action/blob/master/src/run-on-arch.sh#L65-L77

[gaborznagy]

@kira-syslogng retest this please;

[kira-syslogng]

Build SUCCESS

[alltilla]

There were some improvements around ARM runners by GitHub: https://resources.github.com/devops/actions/arm-virtual-hardware/

Haven't read it thoroughly, but we could take a look.

[gaborznagy]

There were some improvements around ARM runners by GitHub: https://resources.github.com/devops/actions/arm-virtual-hardware/

Haven't read it thoroughly, but we could take a look.

Thanks for the info. It still looks like it requires AWS.
I'll check it out during the weekend in more detail. And the alternatives too, just for fun (like Cirrus CI). :)

[alltilla]

One thing to consider, is that we already use AWS for Kira, maybe we can utilize that.

[n-rodriguez]

Hi there! Any news?

[bazsi]

Fyi: the folks at Axoflow started to build an alpine based container for arm, here: https://github.com/axoflow/syslog-ng-docker You can find the images in the "packages" section of GitHub, they are hosted on ghcr.io Those builds are automatically built as a new syslog-ng release is made, nightly is also available. Disclaimer: I am also affiliated with axoflow.

…

On Fri, Apr 7, 2023, 04:07 Nicolas Rodriguez ***@***.***> wrote: Hi there! Any news? — Reply to this email directly, view it on GitHub <#3847 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFOK5RWEMNWELORYI3ZC3LW75SEDANCNFSM5IUHPG6A> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[alltilla]

@bazsi @MrAnno In the light of the AxoSyslog arm images, do we want to keep this PR open? I know that we don't have a per PR arm build, but it seems good enough to find errors there in the nightly builds.

[MrAnno]

We need something that runs per-PR on ARM.
I don't have a strong opinion about the details of this anymore, anything running under 25 minutes is okay for me.

We can close this PR and open an issue instead, if that's convenient for you.

[czanik]

Testing ARM should be a lot faster now, as instead of emulation, it runs now on Ampere servers. At least my Twitter feed was loud about it a few weeks ago.

[MrAnno]

Also, a bit out of topic, but still "aarch64":

https://github.blog/2023-10-02-introducing-the-new-apple-silicon-powered-m1-macos-larger-runner-for-github-actions/

[HofiOne]

Also, a bit out of topic, but still "aarch64":

https://github.blog/2023-10-02-introducing-the-new-apple-silicon-powered-m1-macos-larger-runner-for-github-actions/

i could sware that i checked this multiple times at that time and did not yet work for none large or xlarge targets, thanks for pointing this out again!

#4833

[MrAnno]

I'll revisit this topic and add an ARM-based workflow for running our tests, so let's close this one for now.