Invalid module descriptor of byte-buddy jar when JAVA_HOME is set to version 21

[mkoncek]

On Fedora 38.
Build the project using Maven.

$ JAVA_HOME=/usr/lib/jvm/java-21-openjdk mvn install

Check the module info of the compiled jar:

$ jar -f byte-buddy/target/byte-buddy-1.14.13-SNAPSHOT.jar --describe-module --release 9
releases: 9

java.lang.module.InvalidModuleDescriptorException: Package net.bytebuddy.description.modifier missing from ModulePackages class file attribute
        at java.base/jdk.internal.module.ModuleInfo.invalidModuleDescriptor(ModuleInfo.java:1212)
        at java.base/jdk.internal.module.ModuleInfo.doRead(ModuleInfo.java:330)
        at java.base/jdk.internal.module.ModuleInfo.read(ModuleInfo.java:129)
        at jdk.jartool/sun.tools.jar.Main.describeModule(Main.java:1942)
        at jdk.jartool/sun.tools.jar.Main.describeModuleFromEntries(Main.java:1911)
        at jdk.jartool/sun.tools.jar.Main.describeModule(Main.java:1825)
        at jdk.jartool/sun.tools.jar.Main.run(Main.java:404)
        at jdk.jartool/sun.tools.jar.Main.main(Main.java:1683)

This does not happen when using Java 17.

[raphw]

The plugin to generate the module-info works as expected, but indeed, ASM does not reproduce the packages that are provided to it when reading the generated module-info. I will look into it!

[raphw]

It seems like the ModulePackages attribute disappears somewhere during jar file packaging. It is generated correctly. But if I unzip the jar and check the contained module-info, it seems to disappear.

[raphw]

This seems to be related to the "jar" tool. With Java 8, everything works. The module-info class file is simply included as is. With Java 17 or 21, the ModulePackages attribute is removed.

[mkoncek]

I did test the module-info.class file myself and indeed running javap module-info.class returned the text representation of that file, so the file itself is probably ok.
Something must be wrong in some of the jar headers?

[raphw]

This seems to be related to the "jar" tool. With Java 8, everything works. The module-info class file is simply included as is. With Java 17 or 21, the ModulePackages attribute is removed by the "jar" tool. All other attributes remain intact.

I find this behaviour rather surprising but I assume that I need to specify something on the command line. I will investigate.

[raphw]

Turns out that the jar tool processes the package attribute of a module-info when placing it in a jar and removes all packages that are not contained. This breaks here since the shade plugin is adding class files later from a different location which is not present when the jar is first assembled. As a result, the module-info is incomplete. This does of course not happen with Java 8 as it is not aware of the module-info.

Strange feature of the jar tool, I say, but I worked around it by injecting the module-info class into the jar file now after the jar tool is used. I had to make some tweaks to the (my) plugin to support reproducible builds what did not work before, but this works now.

[raphw]

Can you try master now?

[mkoncek]

Turns out that the jar tool processes the package attribute of a module-info when placing it in a jar and removes all packages that are not contained. This breaks here since the shade plugin is adding class files later from a different location which is not present when the jar is first assembled. As a result, the module-info is incomplete. This does of course not happen with Java 8 as it is not aware of the module-info.

Strange feature of the jar tool, I say, but I worked around it by injecting the module-info class into the jar file now after the jar tool is used. I had to make some tweaks to the (my) plugin to support reproducible builds what did not work before, but this works now.

Simple check worked for me, I didn't check the initial failure (byte-buddy couldn't be used as a modular dependency when bootstrapping Maven with JDK 21, that takes some time to do).
Maven Jar plugin uses the default lifecycle phase package and I don't see a proper phase for modulemaker in this list after that, but initially I was in favor of changing the default lifecycle of the modulemaker so that it would be executed after Jar.
But it probably doesn't make sense.

[mkoncek]

Do you have an idea why this problem manifests with JDK 21? I can only think of natural code changes to the jar tool. I don't see any other reason for it.

[raphw]

Java 17 does not validate this aspect of a module info. Seems like that code was introduced in Java 20 and not backported.

[mkoncek]

With the backported changes to modulemaker and byte-buddy I was able to bootstrap Maven.
I am not sure if the fix in modulemake is "clean" or rather a workaround but it works for now, so this issue can be closed.

By the way, have you considered using a more standard approach to adding a module-info.class to your jars other than using your own Maven plugin? There are dozens of projects facing the same issue of adding it to their projects while remaining compatible with older versions of Java. I don't know if this project has any other special requirements.

[raphw]

When I added the plugin there was nothing really, but I am happy to revisit. Do you know of a good option? I do however wonder if they work with the shade plugin then.

[mkoncek]

Ah right, shade plugin, i don't really know what it does, we always remove it from our packages before our builds.
All I know is it renames class/package names.

[raphw]

And repackages. ASM did some major. API changes in the past, thus the repackaging.

[mkoncek]

So, first thing that comes to my mind is to have a module-info.java plus a two-step compilation in the pom where the first step compiles everything except the module info and the second step compiles the module info only, similar to this:
https://github.com/mizdebsk/java-deptools-native/blob/master/pom.xml#L101

[raphw]

The problem being is that I want to create the same jar with all toolchains from Java 6 upwards. With my plugin this is possible, but but I did not find other ways of doing it.