Loading native lib from regular system lib path?

[StefanOltmann]

Hydraulic Conveyor does extract and sign the native libraries. They are moved to the regular system libs. After that it’s not working anymore.

Is there a system property I can set so they will be found again?

https://hydraulic.dev/blog/11-in-jar-signing.html
https://raw.githubusercontent.com/hydraulic-software/conveyor/master/configs/jvm/extract-native-libraries.conf

This is how skiko deals with that:
https://github.com/JetBrains/skiko/blob/5b5ee682e2a1586ea53cc214960fba280d559ca2/skiko/src/jvmMain/kotlin/org/jetbrains/skiko/Library.kt#L91

[StefanOltmann]

I discovered the existence of PYTORCH_LIBRARY_PATH, which appears to be tailored for PyTorch, suggesting its use for a local installation. The approach of inspecting the system library path utilized by Skiko should be incorporated here as well. Alternatively, consider implementing a universal variable applicable to all engines, ensuring it does not disrupt other applications.

djl/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/jni/LibUtils.java

Line 200 in 063fb8d

private static LibTorch findOverrideLibrary() {

[StefanOltmann]

Setting PYTORCH_LIBRARY_PATH did not help. So that's not the answer.

[frankfliu]

each engine has their own LIBRARY_PATH env var (e.g. MXNET_LIBRARY_PATH).

@StefanOltmann
What error are facing with PYTORCH_LIBRARY_PATH? Are you using Windows? Can you try 0.27.0-SNASPHOST version. We fixed some issue related loading PyTorch.

[StefanOltmann]

I am using the snapshot version on macOS. I copied the native libs somewhere else and set the system property, it just does not work. It gives no exception, it just detects nothing in my images.

Hydraulic Conveyor extracts the native libs and signs them before copying them. Works with other native libs, but somehow not in this case.

[frankfliu]

@StefanOltmann

Can you enable debug log, that can help troubleshooting native library loading issue.
On macOS, it should just work. The easiest way for you is copy a DJL cache folder and set ENGINE_CACHE_DIR or DJL_CACHE_DIR env var. See: https://docs.djl.ai/docs/development/cache_management.html

[StefanOltmann]

That are other variables. 👀

So just copying the files in the PyTorch cache folder to another location and set PYTORCH_LIBRARY_PATH accordingly is not expected to work?

I feel once I get to this I may need to debug the code and find out where it fails exactly.

[frankfliu]

One issue with PYTORCH_LIBRARY_PATH is we have to guess PyTorch version and flavor.
If you set the following env var, we should be able to load it correctly:

export PYTORCH_VERSION=2.1.1
export PYTORCH_FLAVOR=cpu

[StefanOltmann]

Thank you.

I came a lot closer to the problem.

Even though the PYTORCH_LIBRARY_PATH folder contains a libdjl_torch.dylib file a new one is always extracted to .djl.ai/pytorch/2.1.1-cpu-osx-aarch64/0.27.0-libdjl_torch.dylib

Hydraulic Conveyor extracts all native libs into a new directory. So the JAR files don't contain them anymore. I assume that DJL fails because it still wants to extract that lib.

Can we make that the location of libdjl_torch.dylib can also be specified?
Or allow it being contained in the pytorch folder?

Unfortunately Hydraulic Conveyor does not have an option to turn extraction off for just a single JAR. And generally it works great for every other native lib like skiko, lwjgl, sqlitejdbc and others.

[frankfliu]

You are right. PYTORCH_LIBRARY_PATH was design to point to a python installation. Java process may not have permission to write to that folder. So DJL will extract JNI into cache directory.

Can you show me what's the Hydraulic Conveyor created the folder structure?

[StefanOltmann]

It extracts the native libs found in the JAR files and moves them into the Ashampoo Photos.app/Contents/runtime/Contents/Home/lib folder.

These are the ones:

• libdjl_torch.dylib
• libtorch.dylib
• libc10.dylib
• libtorch_cpu.dylib

The other libs used (like skiko) don't even require a variable to be set. They check first if the lib is already present at the default library path and if not, they look for other places. Skiko only extracts it libs to .skiko in the home folder as a last resort.

[StefanOltmann]

Looks like this has been fixed. Thank you!

0fa0db2

[StefanOltmann]

@frankfliu one more thing: Could we also check for the JNI lib name without the version prefix? Or rename it in the JAR?

[frankfliu]

See: https://github.com/deepjavalibrary/djl/blob/master/engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/jni/LibUtils.java#L240