Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrading to 1.78 results in class not found errors when running in OSGI containers #1621

Open
bpaunescu opened this issue Apr 9, 2024 · 19 comments
Assignees

Comments

@bpaunescu
Copy link

We've recently upgraded from 1.77 to 1.78 and are observing the following error:

Caused by: java.lang.NoClassDefFoundError: org/bouncycastle/util/io/pem/PemWriter
at net.corda.crypto.impl.CipherSchemeMetadataProvider.objectToPem(CipherSchemeMetadataProvider.kt:239)
at net.corda.crypto.impl.CipherSchemeMetadataProvider.encodeAsString(CipherSchemeMetadataProvider.kt:210)
... 116 more
Caused by: java.lang.ClassNotFoundException: org.bouncycastle.util.io.pem.PemWriter not found by bcpkix-jdk18on [3]
... 118 more

Our platform uses OSGI. From what we can see, bcpkix-jdk18on library is missing a substantial amount of entries from the Import-Package section of its Manifest file. In 1.77 there are explicit import entries for bcprov packages which have now been removed and are causing these issues when using the libraries in OSGI environments.

tomaswolf added a commit to apache/mina-sshd that referenced this issue Apr 10, 2024
This reverts commit c237cdf.

Bouncy Castle 1.78 has broken OSGi metadata: the package imports in
MANIFEST.MF are missing. See also [1] and [2].

[1] bcgit/bc-java#1621
[2] eclipse-platform/eclipse.platform.releng.aggregator#1968 (comment)
eclipse-jgit-bot pushed a commit to eclipse-jgit/jgit that referenced this issue Apr 10, 2024
This reverts commit 718d121.

Bouncy Castle 1.78 has broken OSGi metadata: the package imports in
MANIFEST.MF are missing. See also [1] and [2].

[1] bcgit/bc-java#1621
[2] eclipse-platform/eclipse.platform.releng.aggregator#1968 (comment)

Change-Id: Iedec921215dabe0b63fb0dd7f6a4b28c18f854a0
Signed-off-by: Thomas Wolf <twolf@apache.org>
@tomaswolf
Copy link

These broken manifests make BC 1.78 unusable in OSGi environments. Would be nice if this could be fixed and a patch release be published.

@dghgit
Copy link
Contributor

dghgit commented Apr 12, 2024

Try what's on https://downloads.bouncycastle.org/betas/ if that works I'll put together a proper one upload it to Maven Central.

@bpaunescu
Copy link
Author

bpaunescu commented Apr 12, 2024

@tomaswolf have you been able to successfully use the 1.78.1 versions linked above? We've installed those in our local Maven repo and the project is now failing to compile altogether.

Looking at the bcpkix manifest file, it's still missing the required import-package entries.

@tomaswolf
Copy link

@tomaswolf have you been able to successfully use the 1.78.1 versions linked above?

No, but I didn't try. I did download them and looked at the manifests, and they still have the same problem.

I guess the reason (for bcpg) is this manifest setting. This is missing something for importing "org.bouncycastle.*", possibly with a version (range). Same for the other bundles.

@dghgit
Copy link
Contributor

dghgit commented Apr 12, 2024

Ah, thanks @tomaswolf - I seem to have fixed only one of 2 problems. I've uploaded a new set of 1.78.1 jars to https://downloads.bouncycastle.org/betas this one adds the missing import - they appear to activate in my local version of kharaf.

In case of caching, sha256 checksums for the latest:

6635bcdafb39922aef0245ecf291963db63e3d716e8176d7ebefb1d91609a4d0 core/build/libs/core-1.78.1.jar
a1e8f245595c0b9ef2220d8940cfabb46d398702a53aa4e488a74f60a33b0c99 jmail/build/libs/bcjmail-jdk18on-1.78.1.jar
356041517799fed719c5bb22a5d145af3b20d97d16df6fe818acb454a4c81bf7 mail/build/libs/bcmail-jdk18on-1.78.1.jar
ddfa5e85f74d335e241bc2bded9aebbf34ec36edb4c8f736f60d3c9723c513f2 mls/build/libs/bcmls-jdk18on-1.78.1.jar
1c699c555e548155d8e69800a3f3b6dd59c2409d0cd5cad2aa9bd52af1bb3ee2 pg/build/libs/bcpg-jdk18on-1.78.1.jar
fac0fd1d4dbb8ffa1eaab0282a5ae6e87d122021316995ccbf01964efb272da2 pkix/build/libs/bcpkix-jdk18on-1.78.1.jar
abaca1a1b71b9554c9d8afc0f7bcf39b3a767b622168173cdaa291c17d5b2770 prov/build/libs/bcprov-jdk18on-1.78.1.jar
272b6576d3742e53901e9221d5c968fb60d09e3086479f9e5694a9c1f2381e9d test/build/libs/test-1.78.1.jar
9fdd1bd605042d198757376bdb748ea2f878a535695734587d99d0a596926ae1 tls/build/libs/bctls-jdk18on-1.78.1.jar
388286e9823df9dd59606eb7183d861eae02bb432d453b285ae3dcfd2abf0ed5 util/build/libs/bcutil-jdk18on-1.78.1.jar

@dghgit dghgit self-assigned this Apr 12, 2024
@tomaswolf
Copy link

The manifests look better, but this doesn't work yet.

In Eclipse we use bcprov, bcpkix, bcpg, and bcutil. With these 1.78.1 JARs I get:

java.lang.NoClassDefFoundError: org/bouncycastle/asn1/gnu/GNUObjectIdentifiers
	at org.bouncycastle.openpgp.PGPPublicKey.init(PGPPublicKey.java:109)
	at org.bouncycastle.openpgp.PGPPublicKey.<init>(PGPPublicKey.java:227)
	at org.bouncycastle.openpgp.PGPPublicKeyRing.<init>(PGPPublicKeyRing.java:122)
	at org.bouncycastle.openpgp.PGPObjectFactory.nextObject(PGPObjectFactory.java:130)
	at org.bouncycastle.openpgp.PGPPublicKeyRingCollection.<init>(PGPPublicKeyRingCollection.java:62)
	...
Caused by: java.lang.ClassNotFoundException: org.bouncycastle.asn1.gnu.GNUObjectIdentifiers cannot be found by bcpg-jdk18on_1.78.1
	at org.eclipse.osgi.internal.loader.BundleLoader.generateException(BundleLoader.java:541)
	at org.eclipse.osgi.internal.loader.BundleLoader.findClass0(BundleLoader.java:536)
	at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:416)
	at org.eclipse.osgi.internal.loader.ModuleClassLoader.loadClass(ModuleClassLoader.java:168)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
	... 65 more

bcpg imports org.bouncycastle.asn1.gnu, but bcutil doesn't export it (it's marked as a private package).

(The class also is twice in the git rpo, once here and once there.)

Apart from that:

  • The bundles now import also packages that they export themselves. (For instance, bcpg imports org.bouncycastle.[bcpg,gpg,openpgp]*). I'm not sure that's intended or a good thing. There are no split packages, are there?
  • The org.bouncycastle* imports should not be optional. If a bundle is missing from the platform and a package cannot be found better fail fast during provisioning (bundle wiring) than later at run-time with a ClassNotFoundException.
  • The version range is "[1.78,2)". In my experience it's not a good idea to have different versions of BC installed, in particular not of bcprov. Do you really want to allow, e.g., using bcpg 1.78 with bcprov 1.79? It might be safer to require the same version (allowing patch versions, though) by using "[1.78,1.79)". Or maybe even "[1.78.1,1.79)".
  • The bundle name change is a bit inconvenient. It means that updating an existing installation becomes tricky. (In 1.77 the bundles were named just "bc*", now they are named "bc*-jdk18on".) For instance, upgrading an Eclipse instance that has bcprov 1.77 may result in an instance that has both bcprov 1.77 and bcprov-jdk18on 1.78.1 installed, which is going to give trouble. (Typical problem: security provider from one version installed, but then classes from the other version being used and getting ClassCastExceptions.) Not sure this can be solved via MANIFEST.MF.

@precoder
Copy link

I also tried with following 5 JARs from the betas site:
bcutil-jdk18on-1.78.1.jar
bcprov-jdk18on-1.78.1.jar
bcpkix-jdk18on-1.78.1.jar
bcpg-jdk18on-1.78.1.jar
bcmail-jdk18on-1.78.1.jar

I get also NoClassDefFoundError for: org.bouncycastle.asn1.oiw.OIWObjectIdentifiers

`Caused by: java.lang.ExceptionInInitializerError: Exception java.lang.NoClassDefFoundError: org/bouncycastle/asn1/oiw/OIWObjectIdentifiers [in thread "FelixStartLevel"]

Thanks for your effort 👍

@dghgit
Copy link
Contributor

dghgit commented Apr 16, 2024

Thanks for the additional info, I think this time I've actually managed to get the plugin and myself in agreement Have been able to set the bundle names back to what they were for 1.77 as well as well as deal with the versioning and the odd import issue. I found a couple more missing exports as well.

3c82b92198957dd6c1bba6ab16e131497257186187199158c33855bbbccc26c9 core/build/libs/core-1.78.1.jar
98857b92c8b8e6cafc86cb6612dd666f536a5dc900b155a003a50d0964ee25ac jmail/build/libs/bcjmail-jdk18on-1.78.1.jar
3a8706b961ca08da1f3ddf78f160d328f62bac1a2fe89fc73dd747f9de90f91b mail/build/libs/bcmail-jdk18on-1.78.1.jar
39c4df3975449e06cf60a3b665c0ca3164e9234fb2b95ad0bf58f27ca4548b40 mls/build/libs/bcmls-jdk18on-1.78.1.jar
0bc016838129e10c05035025994489f2d5ccf3658a975e0f6537ca6edf64e8b6 pg/build/libs/bcpg-jdk18on-1.78.1.jar
2a34524199f1c1d9cd79814f0e7c07e45f89d28431f220715a9b058dfb13435a pkix/build/libs/bcpkix-jdk18on-1.78.1.jar
3a53ca3b8048e561eef1151016fc5e07769a67d6cd405207239599fd5f77b17c prov/build/libs/bcprov-jdk18on-1.78.1.jar
77a93965d64e9bf6eb8f7711eb7dd0d8196d2d4efef5f91643e2ea0c637b4aa9 test/build/libs/test-1.78.1.jar
7447eee6e0a6cc4bd7d4f41ee590071079c27787cc58ab6a4e7bc160f66ba563 tls/build/libs/bctls-jdk18on-1.78.1.jar
9ac67614089053a0ebba86c7df330124ff2dff5cae13f2070df1f97c6c3a85c0 util/build/libs/bcutil-jdk18on-1.78.1.jar

Let me know, it it checks out, I'll put this one together as a release.

@tomaswolf
Copy link

Thanks for the update. In particular, thanks for changing back the Bundle-SymbolicNames.

Getting closer. This now works in Eclipse, but: the import version range is now "1.78.1". In OSGi, this means any version >= 1.78.1 is acceptable, even a future version 4.0.0. This is worse than "[1.78,2)".

This should be changed to "[1.78.1,1.79)".

@tomaswolf
Copy link

However, I run into troubles running unit tests. Might be a JUnit problem. In any case I get

java.lang.NoClassDefFoundError: org/bouncycastle/asn1/cryptlib/CryptlibObjectIdentifiers
	at org.bouncycastle.openpgp.PGPUtil$2.<init>(PGPUtil.java:71)
	at org.bouncycastle.openpgp.PGPUtil.<clinit>(PGPUtil.java:68)
	at org.eclipse.jgit.gpg.bc.internal.keys.KeyGripTest.readAsc(KeyGripTest.java:109)
	at org.eclipse.jgit.gpg.bc.internal.keys.KeyGripTest.testGrip(KeyGripTest.java:127)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.junit.runners.Suite.runChild(Suite.java:128)
	at org.junit.runners.Suite.runChild(Suite.java:27)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:93)
	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:40)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:529)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:756)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:452)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:210)
Caused by: java.lang.ClassNotFoundException: org.bouncycastle.asn1.cryptlib.CryptlibObjectIdentifiers
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
	... 39 more

To resolve this I have to add bcutil as an extra dependency to the bundle containing the tests.

Unclear where this comes from, but I do notice that the bcpg manifest has in its export packages:

org.bouncycastle.openpgp;uses:="org.bouncycastle.asn1,org.bouncycastle.bcpg,org.bouncycastle.bcpg.attr,org.bouncycastle.bcpg.sig,org.bouncycastle.gpg,org.bouncycastle.openpgp.operator,org.bouncycastle.util";version="1.78.1"

Class PGPUtil also imports various subpackages of asn1. Shouldn't those also appear in the "uses" clause?

@dghgit
Copy link
Contributor

dghgit commented Apr 16, 2024

I've done another update which should properly fix the version issue. Here's the checksums:

865d1692fd412079dc3828c827df35286ea20897c5a5439e885c82612bd7c501 core/build/libs/core-1.78.1.jar
bd17cb42bb2faf7103c707e06662f07fa6cec3d2c805bd3b4f19d5d61fa92413 jmail/build/libs/bcjmail-jdk18on-1.78.1.jar
f9b2ea4b3e3f4774140143cf19539297ef313e96a34d9a2b649697897ed15711 mail/build/libs/bcmail-jdk18on-1.78.1.jar
2cec4266c1771a5d81c3752b53e1ee5d981399edb83fcb7194290df01f5d33d5 mls/build/libs/bcmls-jdk18on-1.78.1.jar
27afc2ff05b47be2878de2c934eed548919e42eb6325d6f7d0936dddd221ec90 pg/build/libs/bcpg-jdk18on-1.78.1.jar
b7da20ed53cab20362706cce3df8093309e74bbe0e1ac166a1171f654acd6163 pkix/build/libs/bcpkix-jdk18on-1.78.1.jar
c842e97caf4bd23085f04b749c2be88065134e2a29fbc297181770b0f6c73ae7 prov/build/libs/bcprov-jdk18on-1.78.1.jar
ff65d110088461c4124d5fd53640c0b741275f47e74a4e6c478b2c4433e6bac9 test/build/libs/test-1.78.1.jar
080af1a6216e097253cd53ea3a2d47c2f2fd6d359511d8a64b9546029505de53 tls/build/libs/bctls-jdk18on-1.78.1.jar
2a0856f5fc5b0ca7aca922b3750a0b5997a8832419c805c5af724ae018d99480 util/build/libs/bcutil-jdk18on-1.78.1.jar

With bcpg, there is a dependency on it with bcutil now (the pom was missing this is as well, that's been fixed also). It's only a couple of classes at the moment (previously based in bcprov), but with the latest revisions to the PGP specification it's become obvious it's just going to get worse so we decided it was time to add the dependency.

Let me know how it goes. Thanks.

@tomaswolf
Copy link

Thanks; this now looks OK and works as far as I can test it. @precoder what about your problem case?

@dghgit
Copy link
Contributor

dghgit commented Apr 17, 2024

Thanks @tomaswolf yes @precoder let me know if you're issue is also solved with the latest (I'm confident it is but would like to be sure) and I'll get the update out the door and on Maven Central.

@precoder
Copy link

Now I tested my setup again and it works without any ClassNotFoundExceptions. Thanks for your fast response time and fix :)

@dghgit
Copy link
Contributor

dghgit commented Apr 18, 2024

1.78.1 is now starting to appear on Maven Central and bouncycastle.org.

@yschimke
Copy link
Contributor

This is broken for me.

Trusted-Library: true
Application-Library-Allowable-Codebase: *
Bundle-Version: 1..78.1
Bnd-LastModified: 1713490297960
Bundle-ManifestVersion: 2
Import-Package: org.bouncycastle.asn1;version="",org.bouncycastle.asn1

Causing this build failure https://github.com/square/okhttp/actions/runs/8751257626/job/24016483153?pr=8364#step:7:762

 OsgiTest > testMainModuleWithSiblings() FAILED
    java.lang.IllegalArgumentException: Invalid version 1..78.1 in file build/resources/test/okhttp3/osgi/workspace/cnf/repo/put2137661645660471094.bnd
        at aQute.bnd.deployer.repository.LocalIndexedRepo.putArtifact(LocalIndexedRepo.java:378)
        at aQute.bnd.deployer.repository.LocalIndexedRepo.put(LocalIndexedRepo.java:448)
        at okhttp3.osgi.OsgiTest.deployFile(OsgiTest.kt:131)
        at okhttp3.osgi.OsgiTest.deployClassPath(OsgiTest.kt:123)
        at okhttp3.osgi.OsgiTest.prepareWorkspace(OsgiTest.kt:86)
        at okhttp3.osgi.OsgiTest.createWorkspace(OsgiTest.kt:77)
        at okhttp3.osgi.OsgiTest.testMainModuleWithSiblings(OsgiTest.kt:54)

@dghgit
Copy link
Contributor

dghgit commented Apr 20, 2024

Oh boy... okay, I've fixed the ones available on the BC website, while dealing with that error I've also made the change @tomaswolf suggested of using "[1.78.1,1.79)" for imports as well.

If for some reason you're stuck on Maven Central, it's safe to keep using 1.78, there's 2 minor changes in 1.78.1, they're really edge conditions though, so annoying if you run into them, but if you don't, they simply don't matter. We can't replace the artifacts on Maven Central.

Let me know how you go.

@yschimke
Copy link
Contributor

I'm not blocked, I'll just skip that bump

@dghgit
Copy link
Contributor

dghgit commented Apr 21, 2024

Just so everyone is clear - this second issue applies to the jdk15to18 builds - they use a second, as it turns out, fragile, tool chain. The original issue reported here is still fixed. As things would have it, the jdk15to18 builds are actually okay for 1.78, the .1 release resulted in a script error which caused the invalid 1..78 to appear. People who must use Maven and are unaffected by the edge case that was also fixed in 1.78.1 can use the 1.78 release on Maven Central without OSGi issues - it's only if the rejecting invalid X.509 extensions issue affects you that you need 1.78.1 (which is now corrected for jdk15to18 on the BC website).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants