Two jdk8 security tests consistently failed on test-osuosl-centos74-ppc64le-1 #2625

Using jinfo with either the -sysprops, -flags options, or no option, will cause the error. Meanwhile using the -flag option (the later tests in the linked file) will not cause the error

Haroon-Khel · 2022-07-12T12:09:13Z

Testing this in a command line environment on a failing machine, test-osuosl-centos74-ppc64le-1:

[jenkins@test-centos74-1 bin]$ ./jinfo -J-XX:+UsePerfData -flags +PrintGC 21443
Usage:
    jinfo <option> <pid>
       (to connect to a running process)

where <option> is one of:
    -flag <name>         to print the value of the named VM flag
    -flag [+|-]<name>    to enable or disable the named VM flag
    -flag <name>=<value> to set the named VM flag to the given value
    -h | -help           to print this help message
[jenkins@test-centos74-1 bin]$ ./jinfo -J-XX:+UsePerfData -sysprops 21443
Usage:
    jinfo <option> <pid>
       (to connect to a running process)

where <option> is one of:
    -flag <name>         to print the value of the named VM flag
    -flag [+|-]<name>    to enable or disable the named VM flag
    -flag <name>=<value> to set the named VM flag to the given value
    -h | -help           to print this help message
[jenkins@test-centos74-1 bin]$ ./jinfo -J-XX:+UsePerfData 21443
Usage:
    jinfo <option> <pid>
       (to connect to a running process)

where <option> is one of:
    -flag <name>         to print the value of the named VM flag
    -flag [+|-]<name>    to enable or disable the named VM flag
    -flag <name>=<value> to set the named VM flag to the given value
    -h | -help           to print this help message

While If I use one of the working options:

[jenkins@test-centos74-1 bin]$ ./jinfo -J-XX:+UsePerfData -flag -PrintGC 21443
[jenkins@test-centos74-1 bin]$ echo $?
0

Haroon-Khel · 2022-07-12T12:14:34Z

I think I've got it. These failing tests are not supposed to run on this machine. For example, on a passing machine, test-skytap-ubuntu2004-ppc64le-1, the if statement block which runs these failing tests does not run, https://github.com/adoptium/jdk8u/blob/a18e9043fa2a0a14098e1ec25d32577aaac6c023/jdk/test/sun/tools/jinfo/Basic.sh#L62, since the $runSA = true condition is false.

On test-skytap-ubuntu2004-ppc64le-1, the command ptrace_scope=/sbin/sysctl -n kernel.yama.ptrace_scope, from https://github.com/adoptium/jdk8u/blob/a18e9043fa2a0a14098e1ec25d32577aaac6c023/jdk/test/sun/tools/jinfo/Basic.sh#L54 returns 1 while on a failing machine, test-osuosl-centos74-ppc64le-1, it returns 0

Haroon-Khel · 2022-07-12T12:24:42Z

Setting kernel.yama.ptrace_scope to 1 has fixed this

[jenkins@test-centos74-1 bin]$ /sbin/sysctl -n kernel.yama.ptrace_scope
1
[jenkins@test-centos74-1 bin]$ ./java -jar ../../jtreg/lib/jtreg.jar ../../jdk8u/jdk/test/sun/tools/jinfo/Basic.sh
Test results: passed: 1
Report written to /home/jenkins/jdk8u332-b09/bin/JTreport/html/report.html
Results written to /home/jenkins/jdk8u332-b09/bin/JTwork

Haroon-Khel · 2022-07-12T12:28:18Z

Rerunning sun/tools/jinfo/Basic.sh

centos-1
https://ci.adoptopenjdk.net/view/Test_grinder/job/Grinder/5170/console
centos-2
https://ci.adoptopenjdk.net/view/Test_grinder/job/Grinder/5171/console

Haroon-Khel · 2022-07-18T12:06:35Z

Ref sun/security/pkcs11/fips/TestTLS12.java

On a passing machine, test-skytap-ubuntu2004-ppc64le-1, this test is skipped due to
Test skipped: TLS 1.2 mechanisms not supported by current SunPKCS11 back-end

Need to find why it isnt skipped on the failing machine, or whether it should be skipped

Haroon-Khel · 2022-07-18T12:16:05Z

On a passing machine, the value for sunPKCS11NSSProvider is null, causing the test to skip. While on our 2 failing machines, this value is SunPKCS11-NSSKeyStore version 1.8

Haroon-Khel · 2022-07-18T12:18:13Z

Of the passing machines in #2625 (comment), the test skips

Haroon-Khel · 2022-07-21T11:34:55Z

The test skips on passing machine because https://github.com/adoptium/jdk8u/blob/3dca446d440e55cbb7dc3555392f4520ec9ff3bc/jdk/test/sun/security/pkcs11/SecmodTest.java#L45 returns false due to not being able to find libsoftokn3.so, https://github.com/adoptium/jdk8u/blob/3dca446d440e55cbb7dc3555392f4520ec9ff3bc/jdk/test/sun/security/pkcs11/PKCS11Test.java#L95, while the failing centos machines have this library installed, and so the test will run on these machines and subsequently fail.
However, on test-skytap-ubuntu2004-ppc64le-1 I installed an nsssoftoken which did provide the missing library into /usr/lib, yet the test still could not find the library and skipped the test

sxa · 2022-10-05T14:56:12Z

Bumping this to October milestone as it's a dependency of #2662 which is in there too

steelhead31 · 2023-06-28T08:48:13Z

I believe the root cause of this is that the version of SunPKCS11 being used on Centos 7.4 does not support TLS1.2, and as such the test should be skipped as per Ubuntu, not sure what a fix might be, but I'll keep digging..

steelhead31 · 2023-06-28T14:30:10Z

In order to get the test to pass on Centos7.4, this parameter is required -Djdk.tls.ephemeralDHKeySize=2048

sxa · 2023-06-29T10:46:29Z

Is this specific to ppc64le? That would seem quite strange if true. Also does the same fix allow the test to pass on Ubuntu systems?

steelhead31 · 2023-06-29T10:53:26Z

The fix is specific to centos, on Ubuntu & Fedora, the test gets skipped...

I've had this feedback from the OpenJDK team..

I think the permanent fix should go on the RHEL 9 build of OpenJDK, where
we need to align with RH1974274 "java.txt file contains both security
properties and system properties", now that System properties are in
the /etc/crypto-policies/back-ends/javasystem.config file.

It seems Andrew Hughes is aware of this, by the conversations from the
original RH1883312 ticket and the fedora-crypto-policies MR #104.

However, this is likely to have low priority in a tight backlog. Also,
we need to re-think this in the context of the removal/reduction of
downstream RHEL patches in OpenJDK packages with the goal shipping
near-binary-identical builds to Temurin ones (OPENJDK-1199).

RH1974274: https://bugzilla.redhat.com/show_bug.cgi?id=1974274
RH1883312: https://bugzilla.redhat.com/show_bug.cgi?id=1883312
MR #104: https://gitlab.com/redhat-crypto/fedora-crypto-policies/-/merge_requests/104
OPENJDK-1199: https://issues.redhat.com/browse/OPENJDK-1199

sxa · 2023-06-29T11:05:35Z

The fix is specific to centos, on Ubuntu & Fedora, the test gets skipped...

~~Is that in aqa-tests or is it skipped in the openjdk codebase on those distros?~~ Scratch that - we can't restrict by distro yet in aqa-tests AFAIK. Still a bit unsure why we're only hitting it on ppc64le (I think we have comparable distributions on other architectures...)

steelhead31 · 2023-06-29T11:09:46Z

Yes, given that its skipped on Ubuntu & Fedora in aqa-tests, is excluding the test on Linux/ppc64le an appropriate solution ?

smlambert · 2023-06-29T12:01:53Z

I think it is appropriate to exclude it ProblemList_openjdk8.txt with reference to https://bugs.openjdk.org/browse/JDK-8029661 (or whichever bug notes that there is not an intention to fix in JDK8).

If only seen on ppc64le, is it because our CentOS nodes for x64 Linux are different version of CentOS or have different configuration?

steelhead31 · 2023-06-29T13:25:06Z

After further discussions with @smlambert , and applying the required option would apply it too liberally to a number of tests, the decision has been made to exclude this test, so as to have the minimum impact. as this is deemed a low priority to fix upstream.

sxa · 2023-06-29T13:33:23Z

If only seen on ppc64le, is it because our CentOS nodes for x64 Linux are different version of CentOS or have different configuration?

I've been wondering this too, although ppc64le is the only platform where we have active CentOS7 test systems as far as I can tell (https://ci.adoptium.net/manage/computer/test%2Daws%2Drhel76%2Darmv8%2D1/ exists but is currently not labelled for test - perhaps we could force a Grinder to it anyway to verify?)

We have a few CentOS/RHEL8 (e.g. https://ci.adoptium.net/manage/computer/test%2Ddocker%2Dcentos8%2Darmv8%2D1/) if we wanted to compare 7 vs 8 directly.

steelhead31 · 2023-06-29T14:15:06Z

Tested on Centos 8 ( x64 & Arm64 ) without error, am just looking into whether it skips.

sxa · 2023-06-29T14:33:32Z

Suggest doing a test on the AWS aarch64 RHEL7 one too if possible.

steelhead31 · 2023-06-29T15:02:44Z

@sxa It has run ok on the AWS rhel 7.6 machine, however I've managed to replicate the failure ( and the fix ) in Grinder on the AWS rhel 8 (x64) machine , Grinders 7316 & 7317...

steelhead31 · 2023-06-30T09:22:45Z

In lieu of the upstream bug, this test has now been excluded in aqa PR ( adoptium/aqa-tests#4652 )

Excluding this test from JDK8. ( see https://bugs.openjdk.org/browse/JDK-8029661 )

sophia-guo added the testFail label Jun 21, 2022

sophia-guo mentioned this issue Jun 21, 2022

Triage recent weekly Aqa-test with jdk8 temurin adoptium/aqa-tests#3775

Closed

github-actions bot added arch:ppc64le (pLinux) provider:osuosl labels Jun 21, 2022

Haroon-Khel mentioned this issue Jul 7, 2022

Problem machines for release #2662

Open

sxa added this to the 2022-10 (October) milestone Oct 5, 2022

sxa modified the milestones: 2022-10 (October), 2022-11 (November) Nov 7, 2022

sxa modified the milestones: 2022-11 (November), 2022-12 (December) Dec 1, 2022

sxa modified the milestones: 2022-12 (December), 2023-02 (February) Jan 27, 2023

sxa mentioned this issue Jan 30, 2023

sun/security/pkcs11 tests failing on RHEL8 systems #2900

Open

sxa added the critical label Jan 30, 2023

sxa modified the milestones: 2023-02 (February), 2023-04 (April) Apr 6, 2023

sophia-guo self-assigned this Apr 17, 2023

sophia-guo removed their assignment Apr 17, 2023

sxa modified the milestones: 2023-04 (April), 2023-06 (June) Jun 6, 2023

steelhead31 self-assigned this Jun 27, 2023

steelhead31 mentioned this issue Jun 29, 2023

Exclude sun/security/pkcs11/fips/TestTLS12.java from JDK8 adoptium/aqa-tests#4652

Merged

steelhead31 closed this as completed Jun 30, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Two jdk8 security tests consistently failed on test-osuosl-centos74-ppc64le-1 #2625

Two jdk8 security tests consistently failed on test-osuosl-centos74-ppc64le-1 #2625

sophia-guo commented Jun 21, 2022

Haroon-Khel commented Jul 6, 2022

Haroon-Khel commented Jul 6, 2022 •

edited

Haroon-Khel commented Jul 6, 2022

Haroon-Khel commented Jul 12, 2022 •

edited

Haroon-Khel commented Jul 12, 2022

Haroon-Khel commented Jul 12, 2022

Haroon-Khel commented Jul 12, 2022 •

edited

Haroon-Khel commented Jul 12, 2022

Haroon-Khel commented Jul 18, 2022

Haroon-Khel commented Jul 18, 2022

Haroon-Khel commented Jul 18, 2022

Haroon-Khel commented Jul 21, 2022 •

edited

sxa commented Oct 5, 2022

steelhead31 commented Jun 28, 2023

steelhead31 commented Jun 28, 2023

sxa commented Jun 29, 2023

steelhead31 commented Jun 29, 2023

sxa commented Jun 29, 2023 •

edited

steelhead31 commented Jun 29, 2023

smlambert commented Jun 29, 2023

steelhead31 commented Jun 29, 2023

sxa commented Jun 29, 2023

steelhead31 commented Jun 29, 2023

sxa commented Jun 29, 2023

steelhead31 commented Jun 29, 2023

steelhead31 commented Jun 30, 2023

Two jdk8 security tests consistently failed on test-osuosl-centos74-ppc64le-1 #2625

Two jdk8 security tests consistently failed on test-osuosl-centos74-ppc64le-1 #2625

Comments

sophia-guo commented Jun 21, 2022

Haroon-Khel commented Jul 6, 2022

Haroon-Khel commented Jul 6, 2022 • edited

Haroon-Khel commented Jul 6, 2022

Haroon-Khel commented Jul 12, 2022 • edited

Haroon-Khel commented Jul 12, 2022

Haroon-Khel commented Jul 12, 2022

Haroon-Khel commented Jul 12, 2022 • edited

Haroon-Khel commented Jul 12, 2022

Haroon-Khel commented Jul 18, 2022

Haroon-Khel commented Jul 18, 2022

Haroon-Khel commented Jul 18, 2022

Haroon-Khel commented Jul 21, 2022 • edited

sxa commented Oct 5, 2022

steelhead31 commented Jun 28, 2023

steelhead31 commented Jun 28, 2023

sxa commented Jun 29, 2023

steelhead31 commented Jun 29, 2023

sxa commented Jun 29, 2023 • edited

steelhead31 commented Jun 29, 2023

smlambert commented Jun 29, 2023

steelhead31 commented Jun 29, 2023

sxa commented Jun 29, 2023

steelhead31 commented Jun 29, 2023

sxa commented Jun 29, 2023

steelhead31 commented Jun 29, 2023

steelhead31 commented Jun 30, 2023

Haroon-Khel commented Jul 6, 2022 •

edited

Haroon-Khel commented Jul 12, 2022 •

edited

Haroon-Khel commented Jul 12, 2022 •

edited

Haroon-Khel commented Jul 21, 2022 •

edited

sxa commented Jun 29, 2023 •

edited