Please include hsdis in Adoptium builds

[theRealAph]

hsdis (a disassembler) is extremely useful when analysing HotSpot performance. At present Temurin builds don't support hsdis.

Capstone is a "univeral disassembler" project. It is easy to use Capstone to build OpenJDK's hsdis.
To build OpenJDK using Capstone you need to have Capstone installed. Typical ways of installation can be sudo apt install libcapstone-dev (on Debian and derivatives), or brew install capstone (on macOS with Homebrew). For Windows, you need to download the "Core Engine", and unzip it. See the Capstone Download page for up-to-date download links.

The easiest way to build with support for hsdis on RHEL/CentOS is something like this:

sudo dnf -y install capstone-devel
sh ./configure ... --enable-hsdis-bundling --with-hsdis=capstone

After this, the newly-built shared library jdk/lib/hsdis-XXX.so (not the built JDK, just hsdis) has a runtime dependency on libcapstone.so.4.x. So, to use the disassembly option in OpenJDK not only must a user install the Capstone library, but they must install Version 4 of it. This is rather unfortunate.

An alternative would be to link hsdis statically against libcapstone. This means that we have no runtime dependency on capstone. This would require me to make a small change to the OpenJDK configuration upstream.

As far as I'm aware there are no licence problems using Capstone in this way.

[tellison]

Thanks for opening the discussion.

Just to note that hsdis source code is provided under "The Universal Permissive License (UPL), Version 1.0"

[tellison]

@theRealAph Reading a bit around this tool for the first time, a popular backend option is to use --with-hsdis=binutils. Was there a reason to suggest using Capstone rather than Gnu Binutils?

For the record, the resulting hsdis plug-in binary is approx. 2.2Mb.

[gnu-andrew]

The Binutils backend is the one that has been in OpenJDK since launch - and before there were different backends - and suffers from a license incompatibility between it (GPLv3+) and HotSpot (GPLv2). That's one of the main reasons the other backends were introduced.

There is also now an LLVM backend which is what I'm planning to look into packaging in RHEL & Fedora, as they already have LLVM packages available.

[theRealAph]

@theRealAph Reading a bit around this tool for the first time, a popular backend option is to use --with-hsdis=binutils. Was there a reason to suggest using Capstone rather than Gnu Binutils?

For the record, the resulting hsdis plug-in binary is approx. 2.2Mb.

From a technical point of view I'd use Binutils, but there's no licence-conforming way to to do it, as far as I can see from GPL v3. But it's something of a legal grey area. I don't want to go there.

Hsdis built with Capstone is about the same size as Binutils, at 2.4M.

[sxa]

Noting that CentOS6 (which we use for building on x86 Linux for JDK17 and below) does not have capstone in its repositories so we may need to build from source on there.

[EDIT: The flags to enable hsdis only seem to be in later versions of the codebase, so we would only be able to use these options on systems which aready build with CentOS/RHEL7, so the CentOS6 thing is not a blocking issue unless this gets backported to JDK17]

[sxa]

Noting that openjdk only supports hsdis+capstone on x64 and aarch64

[sxa]

My current initial approach has been to install a suitable capstone as part of the machine setup process (within our ansible scripts) and then invoke the appropriate options during the openjdk build to pick it up. This should allow it to work on CentOS6 without the problems from the earlier comment, and also keep the library smaller (some default installations of capstone appear to contain support for every architecture in the library)

A second option would be to build (and probably therefore also download during each build) the library as part of the Temurin build process, so we'd rebuild it each time. Since this implementation is re-shipping the built library, this might be a preferable option. Thoughts?

[smlambert]

3rd option:

• Download as part of a dependencies job on our Jenkins server and pull it from lastSuccessfulArtifacts of that job, removes 3rd party flakiness, reduces chances of network issues and gets you a mechanism for pulling other versions (if later that is a requirement).

[sxa]

After a couple of discussions today I am now leaning more towards an option 2.5 (Somewhere between 2 and 3) of pulling from pre-built binaries on stored in the userContent path on the jenkins server (Possibly similar to how we have a cache of artefacts like gcc at present) and downloading them onto the machine during the build to reduce dependencies on external servers.
Option 2 isn't ideal since it wouldn't automatically guarantee reproducibility of the build.

[theRealAph]

The main advantage of Capstone is that it is small, and it has a BSD licence, thus GPL v2 compatible.
I'd like hdsis's footprint to be so small that people don't need it to be packaged separately. It is a fantastically-useful diagnostic tool.

Be aware that Capstone can be built in multiple ways. By default, it builds for all architectures. This is not what we want.
See here.

[sxa]

Be aware that Capstone can be built in multiple ways. By default, it builds for all architectures. This is not what we want.

Yep the draft PR I had was to only build for the specific architecture. The approach I'm trialing after my last comment is to store prebuilt versions for each architecture in a form we can download and use within the build process to ensure that we are using an identical binary for each build for reproducibility comparison purposes.

Broadly speaking I'm doing this to build the version, which we will cache and use in a build PR I'm about to look at putting together :-)

export CC=/usr/local/gcc11/bin/gcc-11.2
export CXX=/usr/local/gcc11/bin/g++-11.2
wget https://github.com/capstone-engine/capstone/archive/4.0.2.tar.gz
tar xpfz 4.0.2.tar.gz
cd capstone-4.0.2/
export CAPSTONE_ARCHS=x86
export PREFIX=/usr/local/capstone-4.0.2
./make.sh
./make.sh install
tar czf capstone-4.0.2-x86_64.tar.gz -C /usr/local capstone-4.0.2

[sxa]

An alternative would be to link hsdis statically against libcapstone.

I'm thinking we go for an initial approach of doing it dynamically against a suitable capstone and then subsequently look at adjusting that to do it statically if we make the changes upstream. WDYT @theRealAph?

Noting that if we don't statically link capstone4, CentOS Stream 9 (so likely RHEL9), Ubuntu 22.04 (Latest LTS) and Debian 12 Bookworm all have it. Earlier major versions do not.

I think I'm leaning back towards my option 1 above and installing as version in the playbooks which would have the additional advantage that it would be installed on the system for the purposes of testing until such time as we statically link it in.

[sxa]

PRs to achieve this:

• unixPB: Install capstone 4 from repositories or source infrastructure#3119
• Add hsdis support into the build for JDK20+ temurin-build#3414

[sxa]

Test builds of jdk-21+35 with this support added have been built for x64 and aarch64 (although they will not be retained there indefinitely). As a one-off for people to experiment with, I've made available a capstone download for a suitable 4.0.2 alongside the builds in those links.

[sxa]

The PMC decided today to not include it in the 20.0.1 October deliverable and to subsequently evaluate whether this was useful in a non-development build of the JDK. As an interim we can let users use the builds in the previous comment for evaluation purposes and feedback, or we can look at providing a separate hsdis library that people could add in if desired. This may be more useful once we get a statically linked version of capstone included into hsdis.

[theRealAph]

The PMC decided today to not include it in the 20.0.1 October deliverable and to subsequently evaluate whether this was useful in a non-development build of the JDK.

This is the JDK, not the JRE, right? So it's for Java developers. Some popular tools, especially JMH, need hsdis in order to provide full functionality. Sure, hsdis can be built externally, but you need JDK source to do it, and our Java developers shouldn't have to do that, and many of them don't even know how to do that.

Being able to see the generated code is very useful as an educational tool for helping Java developers understand performance issues. C/C++ compilers provide visible assembly-language, and IMVHO all Java developers have a right to this information too.

[jerboaa]

My understanding was that the current approach of building hsdis.so was to dynamically link to libcapstone.so. That means even if the lib is included, users would still need to have libcapstone.so installed. It would probably be best if that step wouldn't be needed. If we want to include it, there should be an automated test verifying that it works as expected. Could you please provide us some specific use-cases (with examples perhaps) that you think Java developers would use this for on release builds? Thanks!