UnixPb: Install x64 alpine bootjdks where available #2586

Haroon-Khel · 2022-05-30T12:10:42Z

commit message has one of the standard prefixes
faq.md updated if appropriate
other documentation is changed or added (if applicable)
playbook changes run through VPC or QPC (if you have access)
for inventory.yml changes, bastillion/nagios/jenkins updated accordingly

As far as I can see, the alpine jdk build jobs run the build on our alpine docker images, so I have updated what I think are the appropriate files. The alpine pr check will be the judge :)

I've kept some of the oddities, such as JDK7_BOOT_DIR="/usr/lib/jvm/zulu8 despite being a jdk7 bootjdk I've kept it as a jdk8 binary.

karianna

I think we're starting to support Alpine on Aarch64? Does this invalidate that?

sxa · 2022-05-30T12:50:34Z

I think we're starting to support Alpine on Aarch64? Does this invalidate that?

It would depend if we're happy with a nightly going in as we haven't released a GA version on that platform yet for any version.

sxa · 2022-05-30T12:52:00Z

@Haroon-Khel I think for now I still want to install the Zulu JDK8/11 (as a minimum) on aarch64 but I think this PR might be removing that.

sxa · 2022-05-30T17:09:02Z

@Haroon-Khel FYI #2568

sxa · 2022-05-31T17:26:23Z

@Haroon-Khel will this not override JDK7_BOOT_DIR etc. on Alpine/aarch64 which will still have zulu boot JDK and won't have anything /usr/lib/jvm/jdk8 etc. for now?

Haroon-Khel · 2022-06-01T15:33:05Z

So now it should install temurin jdks 8 11 and 17 for both alpine x64 and aarch64 and leave the rest as zulu
To install the aarch64 binaries, I have placed 'static' jenkins nightly links which I do believe the alpine images are able to download. However I do not know if these links are the best option as I do not know if they expire. Open to review

Haroon-Khel · 2022-06-01T15:43:01Z

The code for installing the aarch64 is a bit messy as it uses a nightly link and not a release link. This is temporary until we have a ga aarch64 alpine linux binary

Haroon-Khel · 2022-06-01T15:57:48Z

Does the alpine3 docker image get pushed to dockerhub after the pr gets merged or after the github workflow passes? @sxa @karianna

sxa · 2022-06-01T16:45:38Z

I hope it's after the merge takes place 👀

My only concern just now is that I'm not convinced that the Temurin 11 alpine/aarch64 JDK is stable at the moment (If you look at https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11u/job/jdk11u-alpine-linux-aarch64-temurin/ you can see that things are getting 'stuck' until they hit the 10 hours timeout) so I'm tempted to suggest leaving that one as zulu for now (or potentially installing both for 11)

sxa · 2022-06-01T16:48:58Z

However I do not know if these links are the best option as I do not know if they expire.

Yeah those links from jenkins will expire after a couple of builds as it takes up too much space on the jenkins server to retain them - even though it says lastSuccessfulBuild the artifacts on those jobs are datestamped so the old names will disappear within a few days. You could pull them directly from the github repo e.g. https://github.com/adoptium/temurin17-binaries/releases/download/jdk17u-2022-05-27-19-32-beta/OpenJDK17U-jdk_aarch64_alpine-linux_hotspot_2022-05-27-17-01.tar.gz which are always retained.

Haroon-Khel · 2022-06-06T16:29:38Z

I've changed the links for the aarch64 alpine jdk 8 11 and 17 binaries to links from our github/temurin*-binaries repos, so these shouldn't expire

My only concern just now is that I'm not convinced that the Temurin 11 alpine/aarch64 JDK is stable at the moment (If you look at https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11u/job/jdk11u-alpine-linux-aarch64-temurin/ you can see that things are getting 'stuck' until they hit the 10 hours timeout) so I'm tempted to suggest leaving that one as zulu for now (or potentially installing both for 11)

The binary links that I've added for the aarch64 binaries do not get overridden do they? ie, the links in https://github.com/adoptium/temurin8-binaries/releases/tag/jdk8u-2022-05-28-05-51-beta do not get overridden by newer builds? If so then I do not see how the failing build job will impact the installation of the binary. Unless you mean that the failing build job causes you to lose confidence in the binary itself?

Haroon-Khel · 2022-06-06T16:30:29Z

I hope it's after the merge takes place 👀

This may be the case. I've pulled the latest adoptopenjdk/alpine3_build_image and the changes are not in place

sxa · 2022-06-07T17:06:45Z

The binary links that I've added for the aarch64 binaries do not get overridden do they? ie, the links in https://github.com/adoptium/temurin8-binaries/releases/tag/jdk8u-2022-05-28-05-51-beta do not get overridden by newer builds?

Correct

If so then I do not see how the failing build job will impact the installation of the binary. Unless you mean that the failing build job causes you to lose confidence in the binary itself?

At the moment the Temurin 11 builds on Alpine/aarch64 build and publish but do not work correctly, so I don't want to rely on those builds as the boot JDK for building the nightlies of JDK11. At present we're using a non-Temurin build for the boot JDK, and if we install the Temurin one we're risking making it more unstable as it's known to have problems.

(It probably needs to be tested to be honest - check whether running a build with zulu or Temurin as the boot JDK makes any difference to the build)

Haroon-Khel · 2022-06-09T11:36:53Z

(It probably needs to be tested to be honest - check whether running a build with zulu or Temurin as the boot JDK makes any difference to the build)

I'll give this a try

sxa · 2022-06-09T11:38:39Z

(It probably needs to be tested to be honest - check whether running a build with zulu or Temurin as the boot JDK makes any difference to the build)

I'll give this a try

We have quite a few options now for bootstrapping JDK11 - There's the Alpine-supplied openjdk 11 package, a zulu one, or a Temurin nightly :-)

Haroon-Khel · 2022-06-09T16:01:09Z

@sxa After two attempts (first attempt hung), I got jdk11 to build on dockerhost-equinix-ubuntu2004-armv8-1, currently the only machine that builds jdk11 aarch64 alpine linux.

I also got it to build on https://ci.adoptopenjdk.net/computer/docker-equinix-ubuntu2004-armv8-1/, and have now added the alpine-linux label to it so that it may get picked up by some of the alpine jdk build jobs

To summarise, I think the instability of the jdk11 aarch64 alpine linux build job is due to infra related issues on dockerhost-equinix-ubuntu2004-armv8-1. The timed out test jobs in adoptium/temurin-build#2961 also ran on that machine. Either way, I think using a temurin nightly as a bootjdk for the alpine images is fine

sxa · 2022-06-09T16:41:30Z

Interesting ... Do you have Grinder runs from it running on the other machine?

Haroon-Khel · 2022-06-09T19:36:22Z

Good point.
I've set up https://ci.adoptopenjdk.net/computer/test-docker-alpine311-aarch64-1/ and https://ci.adoptopenjdk.net/computer/test-docker-alpine312-aarch64-1/ on https://ci.adoptopenjdk.net/computer/docker-equinix-ubuntu2004-armv8-1/. It seems the only aarch64 test alpine containers were hosted on dockerhost-equinix-ubuntu2004-armv8-1.

I've kicked off some grinders on the alp311 and 312 containers

Sanity openjdk https://ci.adoptopenjdk.net/job/Grinder/4970/console
Extended functional https://ci.adoptopenjdk.net/job/Grinder/4971/console
Sanity functional https://ci.adoptopenjdk.net/job/Grinder/4972/console
Sanity system https://ci.adoptopenjdk.net/job/Grinder/4973/console
Sanity perf https://ci.adoptopenjdk.net/job/Grinder/4974/console
Extended system https://ci.adoptopenjdk.net/job/Grinder/4975/console

Hoping they dont time out

Haroon-Khel · 2022-06-13T12:25:31Z

The tests look unstable even on the second machine. So I've returned jdk11 back to being installed from the alpine repos, but I have kept its link as /usr/lib/jvm/jdk-11 instead of zulu11 so that the links can be consistent on x64 and aarch64

Haroon-Khel · 2022-06-13T12:25:52Z

ping @sxa for review

Haroon-Khel · 2022-06-13T17:21:06Z

I'm not quite sure what the linter is complaining about

Error: schema [{'os_img': 'aix/ppc64/jdk'}, {'heap': 'normal'}, {'impl': 'hotspot'}, {'new_baseurl': 'https://api.adoptium.net/v3/binary/latest'}, {'new_vendor': 'eclipse'}, {'baseurl': 'https://api.adoptopenjdk.net/v3/binary/latest'}, {'vendor': 'adoptopenjdk'}, {'project': '{{ heap }}/{{ vendor }}?project=jdk'}] is not of type 'object'
Error: schema None is not of type 'boolean'
Warning: ignore-errors Use failed_when and specify error conditions instead of using ignore_errors.
Error: schema 1 is not of type 'string'
Error: schema [{'ansible_python_interpreter': '/usr/local/python2/bin/python2.7'}] is not of type 'object'
Warning: var-spacing Jinja2 variables and filters should have spaces before and after.
You can skip specific rules or tags by adding them to your configuration file:
# .config/ansible-lint.yml
warn_list:  # or 'skip_list' to silence them completely
  - experimental  # all rules tagged as experimental
  - var-spacing  # Jinja2 variables and filters should have spaces before and after.
Finished with 1 failure(s), 5 warning(s) on 414 files.
Error: Process completed with exit code 2.

sxa · 2022-06-13T18:14:31Z

I'm not quite sure what the linter is complaining about

Seems to be happening on other PRs too, so unlikely to be related to anything here.

sxa

I think I'm ok with this now. Will aim to merge tomorrow (Don't want to do it before the nightly builds...

install x64 alpine bootjdks where available

d19fde8

Haroon-Khel requested review from sxa and gdams May 30, 2022 12:10

github-actions bot added ansible docker arch:x64 labels May 30, 2022

Haroon-Khel changed the title ~~install x64 alpine bootjdks where available~~ UnixPb: Install x64 alpine bootjdks where available May 30, 2022

trailing spaces

cb89bfe

karianna requested changes May 30, 2022

View reviewed changes

Haroon-Khel added 3 commits June 1, 2022 16:16

add jdk 8 11 and 17 alpine linux temurin aarch64 binaries

81e72ce

remove alpine aarch64 condition from bootjdk roles 8 11 and 17

34e1009

trailing spaces

ca985f7

Haroon-Khel requested a review from karianna June 1, 2022 15:33

change aarch64 binary links

0ed78c0

karianna approved these changes Jun 6, 2022

View reviewed changes

switch jdk11 back to zulu

deccd67

sxa approved these changes Jun 13, 2022

View reviewed changes

sxa merged commit 4e1cf66 into adoptium:master Jun 14, 2022

sxa mentioned this pull request Jun 14, 2022

unixPB: Fix install of Alpine JDK8 boot JDK #2603

Merged

5 tasks

Haroon-Khel deleted the alpine.bootjdk branch June 23, 2022 12:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

UnixPb: Install x64 alpine bootjdks where available #2586

UnixPb: Install x64 alpine bootjdks where available #2586

Haroon-Khel commented May 30, 2022 •

edited

karianna left a comment

sxa commented May 30, 2022

sxa commented May 30, 2022

sxa commented May 30, 2022

sxa commented May 31, 2022

Haroon-Khel commented Jun 1, 2022

Haroon-Khel commented Jun 1, 2022

Haroon-Khel commented Jun 1, 2022

sxa commented Jun 1, 2022

sxa commented Jun 1, 2022

Haroon-Khel commented Jun 6, 2022

Haroon-Khel commented Jun 6, 2022

sxa commented Jun 7, 2022

Haroon-Khel commented Jun 9, 2022

sxa commented Jun 9, 2022

Haroon-Khel commented Jun 9, 2022

sxa commented Jun 9, 2022 •

edited

Haroon-Khel commented Jun 9, 2022 •

edited

Haroon-Khel commented Jun 13, 2022

Haroon-Khel commented Jun 13, 2022

Haroon-Khel commented Jun 13, 2022

sxa commented Jun 13, 2022

sxa left a comment

UnixPb: Install x64 alpine bootjdks where available #2586

UnixPb: Install x64 alpine bootjdks where available #2586

Conversation

Haroon-Khel commented May 30, 2022 • edited

karianna left a comment

Choose a reason for hiding this comment

sxa commented May 30, 2022

sxa commented May 30, 2022

sxa commented May 30, 2022

sxa commented May 31, 2022

Haroon-Khel commented Jun 1, 2022

Haroon-Khel commented Jun 1, 2022

Haroon-Khel commented Jun 1, 2022

sxa commented Jun 1, 2022

sxa commented Jun 1, 2022

Haroon-Khel commented Jun 6, 2022

Haroon-Khel commented Jun 6, 2022

sxa commented Jun 7, 2022

Haroon-Khel commented Jun 9, 2022

sxa commented Jun 9, 2022

Haroon-Khel commented Jun 9, 2022

sxa commented Jun 9, 2022 • edited

Haroon-Khel commented Jun 9, 2022 • edited

Haroon-Khel commented Jun 13, 2022

Haroon-Khel commented Jun 13, 2022

Haroon-Khel commented Jun 13, 2022

sxa commented Jun 13, 2022

sxa left a comment

Choose a reason for hiding this comment

Haroon-Khel commented May 30, 2022 •

edited

sxa commented Jun 9, 2022 •

edited

Haroon-Khel commented Jun 9, 2022 •

edited