unixPb: GPG signature verification added to jdk downloads #2999

Haroon-Khel · 2023-03-17T14:38:53Z

commit message has one of the standard prefixes
faq.md updated if appropriate
other documentation is changed or added (if applicable)
playbook changes run through VPC or QPC (if you have access)
VPC/QPC not applicable for this PR
for inventory.yml changes, bastillion/nagios/jenkins updated accordingly

Added signature verification to the linux, alpine linux, solaris and mac jdk downloads in the unix playbook.
Refactored the linux and alpine linux tasks into one.
Can't get a signature file for jdk 10, api will not return release information for jdk 10, so verification is skipped for jdk 10.
Refactored tasks into blocks for easier management.
Removed duplicate code at the bottom.

github-actions

A block has been put on this Pull Request as this repository is temporarily under a code freeze due to an ongoing release cycle.

If this pull request needs to be merged during the release cycle then please comment /merge and a PMC member will be able to remove the block.

If the code freeze is over you can remove this block by commenting /thaw.

Haroon-Khel · 2023-03-17T14:57:10Z

vpc job https://ci.adoptium.net/job/VagrantPlaybookCheck/1640/console

Haroon-Khel · 2023-03-17T15:20:23Z

The api does not return a signature_link for jdk16 x64 mac it seems, causing it to fail. Using https://api.adoptium.net/q/swagger-ui/#/Assets/getLatestAssets

Request url: https://api.adoptium.net/v3/assets/latest/16/hotspot?architecture=x64&image_type=jdk&os=mac&vendor=eclipse

sxa · 2023-03-17T15:45:36Z

The api does not return a signature_link for jdk16 x64 mac it seems, causing it to fail.

GPG signatures were added to all current release lines last year so I would expect it will have excluded JDK16 on all platforms.

However, I believe (please verify this for all platforms - I'm confident linux is ok though) that all of the JDK17 builds are now bootstrapped with JDK17, so JDK16 is no longer required on the machines, in which case it can be removed from the playbooks. (Having said that, presumably we can't check them on JDK10 for bootstrapping 11 either)

Haroon-Khel · 2023-03-20T12:31:12Z

However, I believe (please verify this for all platforms - I'm confident linux is ok though) that all of the JDK17 builds are now bootstrapped with JDK17, so JDK16 is no longer required on the machines, in which case it can be removed from the playbooks.

@sxa You're right, i just checked. I've removed jdk16 from the playbook

Haroon-Khel · 2023-03-20T12:31:43Z

Latest vpc https://ci.adoptium.net/job/VagrantPlaybookCheck/1645/console

sxa · 2023-03-20T13:54:59Z

Can you adjust the first commit message so it has the correct unixPB: prefix please?

Haroon-Khel · 2023-03-20T17:35:34Z

Jdk8 s390x also does not have a signature file, so I've excluded the check for that platform

Haroon-Khel · 2023-03-20T17:37:38Z

Fails on Centos 6 in the vpc, but passes on Centos 6 in the github checks
https://ci.adoptium.net/job/VagrantPlaybookCheck/1645/

Rerunning on just Centos 6 https://ci.adoptium.net/job/VagrantPlaybookCheck/1651/console

Haroon-Khel · 2023-03-21T10:17:31Z

New vpc job centos6 only https://ci.adoptium.net/job/VagrantPlaybookCheck/1653/console

ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/Common/vars/CentOS.yml

Haroon-Khel · 2023-03-21T14:59:13Z

I dont know why the vars arent being inherited from the block part of the task

  vars:
    ansible_python_interpreter: /usr/local/python2/bin/python2.7

12:02:33 TASK [adoptopenjdk_install : Download latest JDK 8 release (CentOS6)] **********
12:02:45 FAILED - RETRYING: Download latest JDK 8 release (CentOS6) (3 retries left).
12:03:01 FAILED - RETRYING: Download latest JDK 8 release (CentOS6) (2 retries left).
12:03:17 FAILED - RETRYING: Download latest JDK 8 release (CentOS6) (1 retries left).
12:03:33 fatal: [127.0.0.1]: FAILED! => {"attempts": 3, "changed": false, "msg": "Aborting, target uses selinux but python bindings (libselinux-python) aren't installed!"}

Haroon-Khel · 2023-03-21T15:20:09Z

New vpc https://ci.adoptium.net/job/VagrantPlaybookCheck/1654/console
If this still fails, I'm replacing the get_url task with a wget command

Haroon-Khel · 2023-03-27T13:20:50Z

Latest vpc https://ci.adoptium.net/job/VagrantPlaybookCheck/1656/console

Haroon-Khel · 2023-03-27T15:33:23Z

VPC looks good. I've removed one of the commits from earlier where I install libselinux-python for centos 6 (I found that it isnt needed). This pr is ready to merge

sxa

LGTM but can you adjust the description to say you've run VPC on it - currently the checkbox is for "Doesn't need to be run" :-)

karianna

Might be worth commenting each usage of the older API and why it is called

ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/adoptopenjdk_install/tasks/main.yml

Haroon-Khel · 2023-03-29T10:44:32Z

New vpc https://ci.adoptium.net/job/VagrantPlaybookCheck/1657/console

Haroon-Khel · 2023-03-29T11:35:11Z

Jdk 10 binaries cannot be retrieved using api.adoptium.net, so ive kept them with api.adoptopenjdk.net

https://ci.adoptium.net/job/VagrantPlaybookCheck/1658/console

sxa · 2023-03-29T11:55:06Z

Jdk 10 binaries cannot be retrieved using api.adoptium.net, so ive kept them with api.adoptopenjdk.net

Yeah I'm not sure I like this extra complexity. I would recommend leaving as-is for the purposes of this PR and if there's a desire to use the different APIs then it can be done independently since this has nothing to do with enabling GPG verification.

In fact it may make more sense to consider having a separate adoptium_install role to make it cleaner since these variables covering both looks quite messy to me.

Haroon-Khel · 2023-03-29T14:37:25Z

Yeah I'm not sure I like this extra complexity. I would recommend leaving as-is for the purposes of this PR and if there's a desire to use the different APIs then it can be done independently since this has nothing to do with enabling GPG verification.

Ok, i'll leave the api as it is. Any update to it can happen in a separate pr

karianna · 2023-03-30T02:21:47Z

/thaw

Pull Request unblocked - code freeze is over.

karianna · 2023-03-30T08:31:42Z

Solaris Host is failing to connect

PLAY [Ansible Unix playbook] ***************************************************

TASK [Gathering Facts] *********************************************************
fatal: [127.0.0.1]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Warning: Permanently added '[127.0.0.1]:2222' (ED25519) to the list of known hosts.\r\nssh_dispatch_run_fatal: Connection to 127.0.0.1 port 2222: incorrect signature", "unreachable": true}

PLAY RECAP *********************************************************************
127.0.0.1 : ok=0 changed=0 unreachable=1 failed=0 skipped=0 rescued=0 ignored=0

Haroon-Khel · 2023-03-30T13:53:59Z

@karianna Yes that has been a failure for sometime now. I'll raise an issue. Meanwhile this pr has passed the solaris 10 vpc check https://ci.adoptium.net/job/VagrantPlaybookCheck/1645/OS=Solaris10,label=vagrant/console

add gpg verification to jdk downloads, and refactor

668f805

github-actions bot added the ansible label Mar 17, 2023

github-actions bot previously requested changes Mar 17, 2023

View reviewed changes

Haroon-Khel added 2 commits March 17, 2023 14:49

satisfy linter

a52c688

more linter

6e1aa8f

Haroon-Khel mentioned this pull request Mar 17, 2023

Add GPG signature verification to adoptium binary downloads #2908

Closed

3 tasks

Haroon-Khel added 2 commits March 17, 2023 15:15

relabel tasks

4ba7949

more relabelling

07f72ca

remove bootjdk 16

f992716

Haroon-Khel added the QEMU-playbook-check Runs build_qemu workflow label Mar 20, 2023

exclude gpg for s390x jdk8

cb80fdd

Haroon-Khel added QEMU-playbook-check Runs build_qemu workflow and removed QEMU-playbook-check Runs build_qemu workflow labels Mar 20, 2023

install libselinux-python for centos6

76d69fa

sxa reviewed Mar 21, 2023

View reviewed changes

ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/Common/vars/CentOS.yml Outdated Show resolved Hide resolved

Haroon-Khel changed the title ~~UnixPb: GPG signature verification added to jdk downloads~~ unixPb: GPG signature verification added to jdk downloads Mar 21, 2023

move vars into block tasks

986f3b4

change get_url to wget for centos6

bb858ab

remove centos 6 specific packages I added earlier

8801048

sxa approved these changes Mar 28, 2023

View reviewed changes

Haroon-Khel requested review from gdams and karianna March 28, 2023 13:28

karianna requested changes Mar 28, 2023

View reviewed changes

change to adoptium api

6dc1a0f

Haroon-Khel requested a review from karianna March 29, 2023 10:07

add eclipse parameters to adoptium api

a7e87ba

Haroon-Khel added QEMU-playbook-check Runs build_qemu workflow and removed QEMU-playbook-check Runs build_qemu workflow labels Mar 29, 2023

if jdk10 then adoptopenjdk api, else adoptium

99d7f34

revert back to adoptopenjdk api

76093ed

karianna approved these changes Mar 30, 2023

View reviewed changes

Haroon-Khel merged commit ff06288 into adoptium:master Mar 30, 2023
8 of 9 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

unixPb: GPG signature verification added to jdk downloads #2999

unixPb: GPG signature verification added to jdk downloads #2999

Haroon-Khel commented Mar 17, 2023 •

edited

github-actions bot left a comment

Haroon-Khel commented Mar 17, 2023

Haroon-Khel commented Mar 17, 2023 •

edited

sxa commented Mar 17, 2023

Haroon-Khel commented Mar 20, 2023

Haroon-Khel commented Mar 20, 2023

sxa commented Mar 20, 2023

Haroon-Khel commented Mar 20, 2023

Haroon-Khel commented Mar 20, 2023 •

edited

Haroon-Khel commented Mar 21, 2023 •

edited

Haroon-Khel commented Mar 21, 2023

Haroon-Khel commented Mar 21, 2023

Haroon-Khel commented Mar 27, 2023

Haroon-Khel commented Mar 27, 2023

sxa left a comment

karianna left a comment

Haroon-Khel commented Mar 29, 2023

Haroon-Khel commented Mar 29, 2023 •

edited

sxa commented Mar 29, 2023

Haroon-Khel commented Mar 29, 2023

karianna commented Mar 30, 2023

karianna commented Mar 30, 2023

Haroon-Khel commented Mar 30, 2023

unixPb: GPG signature verification added to jdk downloads #2999

unixPb: GPG signature verification added to jdk downloads #2999

Conversation

Haroon-Khel commented Mar 17, 2023 • edited

github-actions bot left a comment

Choose a reason for hiding this comment

Haroon-Khel commented Mar 17, 2023

Haroon-Khel commented Mar 17, 2023 • edited

sxa commented Mar 17, 2023

Haroon-Khel commented Mar 20, 2023

Haroon-Khel commented Mar 20, 2023

sxa commented Mar 20, 2023

Haroon-Khel commented Mar 20, 2023

Haroon-Khel commented Mar 20, 2023 • edited

Haroon-Khel commented Mar 21, 2023 • edited

Haroon-Khel commented Mar 21, 2023

Haroon-Khel commented Mar 21, 2023

Haroon-Khel commented Mar 27, 2023

Haroon-Khel commented Mar 27, 2023

sxa left a comment

Choose a reason for hiding this comment

karianna left a comment

Choose a reason for hiding this comment

Haroon-Khel commented Mar 29, 2023

Haroon-Khel commented Mar 29, 2023 • edited

sxa commented Mar 29, 2023

Haroon-Khel commented Mar 29, 2023

karianna commented Mar 30, 2023

karianna commented Mar 30, 2023

Haroon-Khel commented Mar 30, 2023

Haroon-Khel commented Mar 17, 2023 •

edited

Haroon-Khel commented Mar 17, 2023 •

edited

Haroon-Khel commented Mar 20, 2023 •

edited

Haroon-Khel commented Mar 21, 2023 •

edited

Haroon-Khel commented Mar 29, 2023 •

edited