Elasticsearch fails to start with both 32 bit and 64 bit libsystemd installed

[marshallmain]

Elasticsearch Version

8.15.0 snapshot

Installed Plugins

No response

Java Version

bundled

OS Version

Ubuntu Linux

Problem Description

Starting Elasticsearch immediately logs an error:

marshall@marshall-X570-AORUS-PRO-WIFI:~/dev/kibana$ yarn es snapshot -E xpack.security.authc.api_key.enabled=true --license trial
yarn run v1.22.19
$ node scripts/es snapshot -E xpack.security.authc.api_key.enabled=true --license trial
 info Installing from snapshot
 info version: 8.15.0
 info install path: /home/marshall/dev/kibana/.es/8.15.0
 info license: trial
 info Downloading snapshot manifest from https://storage.googleapis.com/kibana-ci-es-snapshots-daily/8.15.0/manifest-latest-verified.json
 info downloading artifact from https://storage.googleapis.com/kibana-ci-es-snapshots-daily/8.15.0/archives/20240422-131735_f9c414c5/elasticsearch-8.15.0-SNAPSHOT-linux-x86_64.tar.gz
 info downloading artifact checksum from https://storage.googleapis.com/kibana-ci-es-snapshots-daily/8.15.0/archives/20240422-131735_f9c414c5/elasticsearch-8.15.0-SNAPSHOT-linux-x86_64.tar.gz.sha512
 info checksum verified
 info extracting /home/marshall/dev/kibana/.es/cache/elasticsearch-8.15.0-SNAPSHOT-linux-x86_64.tar.gz
 info extracted to /home/marshall/dev/kibana/.es/8.15.0
 info created /home/marshall/dev/kibana/.es/8.15.0/ES_TMPDIR
 info setting secure setting bootstrap.password to changeme
 info Starting
   │ info bin/elasticsearch -E action.destructive_requires_name=true -E cluster.routing.allocation.disk.threshold_enabled=false -E ingest.geoip.downloader.enabled=false -E search.check_ccs_compatibility=true -E xpack.security.authc.api_key.enabled=true
   │ info ES_JAVA_OPTS: -Xms1536m -Xmx1536m
   │ERROR Apr 22, 2024 8:54:02 AM sun.util.locale.provider.LocaleProviderAdapter <clinit>
   │ERROR WARNING: COMPAT locale provider will be removed in a future release
   │ERROR OpenJDK 64-Bit Server VM warning: You have loaded library /usr/lib/i386-linux-gnu/libsystemd.so.0.26.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
   │      It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
   │ info [o.e.n.NativeAccess] [marshall-X570-AORUS-PRO-WIFI] Unable to load native provider. Native methods will be disabled. java.lang.UnsatisfiedLinkError: /usr/lib/i386-linux-gnu/libsystemd.so.0.26.0: /usr/lib/i386-linux-gnu/libsystemd.so.0.26.0: wrong ELF class: ELFCLASS32 (Possible cause: can't load IA 32 .so on a AMD 64 platform)
   │      	at java.base/jdk.internal.loader.NativeLibraries.load(Native Method)
   │      	at java.base/jdk.internal.loader.NativeLibraries$NativeLibraryImpl.open(NativeLibraries.java:331)
   │      	at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:197)
   │      	at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:139)
   │      	at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2418)
   │      	at java.base/java.lang.Runtime.load0(Runtime.java:852)
   │      	at java.base/java.lang.System.load(System.java:2025)
   │      	at org.elasticsearch.nativeaccess@8.15.0-SNAPSHOT/org.elasticsearch.nativeaccess.jdk.JdkSystemdLibrary.<clinit>(JdkSystemdLibrary.java:29)
   │      	at org.elasticsearch.nativeaccess@8.15.0-SNAPSHOT/org.elasticsearch.nativeaccess.lib.NativeLibraryProvider.getLibrary(NativeLibraryProvider.java:58)
   │      	at org.elasticsearch.nativeaccess@8.15.0-SNAPSHOT/org.elasticsearch.nativeaccess.LinuxNativeAccess.<init>(LinuxNativeAccess.java:20)
   │      	at org.elasticsearch.nativeaccess@8.15.0-SNAPSHOT/org.elasticsearch.nativeaccess.NativeAccessHolder.<clinit>(NativeAccessHolder.java:28)
   │      	at org.elasticsearch.nativeaccess@8.15.0-SNAPSHOT/org.elasticsearch.nativeaccess.NativeAccess.instance(NativeAccess.java:22)
   │      	at org.elasticsearch.server@8.15.0-SNAPSHOT/org.elasticsearch.bootstrap.Elasticsearch.initializeNatives(Elasticsearch.java:285)
   │      	at org.elasticsearch.server@8.15.0-SNAPSHOT/org.elasticsearch.bootstrap.Elasticsearch.initPhase2(Elasticsearch.java:169)
   │      	at org.elasticsearch.server@8.15.0-SNAPSHOT/org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:74)
   │ info [o.e.n.NativeAccess] [marshall-X570-AORUS-PRO-WIFI] Cannot check if running as root because native access is not available

This leads to a number of messages about zstd is null and shard failures, e.g.

│ info [o.e.c.r.a.AllocationService] [marshall-X570-AORUS-PRO-WIFI] failing shard [FailedShard[routingEntry=[.security-7][0], node[taa-iE1pTke-lCYe7gWuzA], [P], recovery_source[existing store recovery; bootstrap_history_uuid=false], s[INITIALIZING], a[id=aG-UKf6nRzW5RA4cjICx2w], unassigned_info[[reason=ALLOCATION_FAILED], at[2024-04-22T15:54:15.434Z], failed_attempts[1], delayed=false, last_node[taa-iE1pTke-lCYe7gWuzA], details[failed shard on node [taa-iE1pTke-lCYe7gWuzA]: shard failure, reason [refresh failed source[schedule]], failure java.lang.NullPointerException: Cannot invoke "org.elasticsearch.nativeaccess.Zstd.compressBound(int)" because "zstd" is null

And Elasticsearch crashes before finishing startup.

Steps to Reproduce

Attempt to start Elasticsearch on a system with both 32 bit and 64 bit libsystemd installations.

marshall@marshall-X570-AORUS-PRO-WIFI:~$ sudo apt list | grep libsystemd

libsystemd-dev/eoan-security 242-7ubuntu3.11 amd64
libsystemd-dev/eoan-security 242-7ubuntu3.11 i386
libsystemd0/eoan-security,now 242-7ubuntu3.11 amd64 [installed,automatic]
libsystemd0/eoan-security,now 242-7ubuntu3.11 i386 [installed,automatic]

Logs (if relevant)

No response

[elasticsearchmachine]

Pinging @elastic/es-core-infra (Team:Core/Infra)