Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kscript is broken on ArchLinux #371

Closed
CLOVIS-AI opened this issue Jun 29, 2022 · 20 comments · May be fixed by #383
Closed

Kscript is broken on ArchLinux #371

CLOVIS-AI opened this issue Jun 29, 2022 · 20 comments · May be fixed by #383

Comments

@CLOVIS-AI
Copy link

CLOVIS-AI commented Jun 29, 2022

Installed from the AUR.

Error message:

$ kscript 'println("Hello world")'
/usr/bin/kscript: line 49: /usr/share/kotlin/bin/kotlin: No such file or directory

Correct path to the Kotlin binary:

$ command -v kotlin
/usr/bin/kotlin

I don't really know what information could be of use, don't hesitate to ask for specific information you need.

@aartiPl
Copy link
Collaborator

aartiPl commented Jul 3, 2022

Thanks for the report!

Can you please execute in the console:

  1. echo $KOTLIN_HOME
  2. KOTLIN_RUNNER=1 JAVACMD=echo kotlinc

@CLOVIS-AI
Copy link
Author

$ echo $KOTLIN_HOME

$ KOTLIN_RUNNER=1 JAVACMD=echo kotlinc
-Xmx256M -Xms32M -Dkotlin.home=/usr/share/kotlin -cp /usr/share/kotlin/lib/kotlin-runner.jar org.jetbrains.kotlin.runner.Main

@aartiPl
Copy link
Collaborator

aartiPl commented Jul 12, 2022

The second line is explaining why you get /usr/share/kotlin path for kotlin. (I am not exactly sure how this line work, but it is used to guess KOTLIN_HOME, if it is not set explicitly).
To make kscript working set your KOTLIN_HOME env variable to /usr , and it should start working. I will keep this issue open to allow other comments and suggestions.

@CLOVIS-AI
Copy link
Author

It does not seem to fix the problem.

# .zshrc
export KOTLIN_HOME=/usr
$ echo $KOTLIN_HOME
/usr
$ kscript 'println("Hello world!")'
[kscript] [ERROR] Compilation of scriplet failed:
[kscript] [ERROR] Command     : bash -c /usr/share/kotlin/bin/kotlinc   -d '/home/ivan/.kscript/cache/jar_1075c8348f9943a25627037568233d8f/scriplet.jar' '/home/ivan/.kscript/cache/jar_1075c8348f9943a25627037568233d8f/Scriplet.kts' '/home/ivan/.kscript/cache/jar_1075c8348f9943a25627037568233d8f/Main_Scriplet.kt'
[kscript] [ERROR] Exit Code   : 127   
[kscript] [ERROR] Stdout      : 
[kscript] [ERROR] Stderr      : 
[kscript] [ERROR] bash: line 1: /usr/share/kotlin/bin/kotlinc: No such file or directory
[kscript] [ERROR]

I'm not sure why the error message is visually different yet complains about the same thing.

@aartiPl
Copy link
Collaborator

aartiPl commented Jul 15, 2022

That's strange. Below you can see the code which is used to resolve kotlin home:

obraz

https://github.com/holgerbrandl/kscript/blob/master/src/main/kotlin/kscript/app/model/ConfigBuilder.kt

As you can see KOTLIN_HOME env goes first while resolving kotlinc.

Another relevant piece of code:

obraz

https://github.com/holgerbrandl/kscript/blob/master/src/kscript

But here also everything looks okay for me.

@sfesenko
Copy link
Contributor

As you can see KOTLIN_HOME env goes first while resolving kotlinc.
But here also everything looks okay for me.

It's not ok, since kscript makes wrong assumption about location of kotlin binaries, and doesn't check their actual location.
In archlinux scripts are located in $PATH, not under $KOTLIN_HOME.

pacman -Ql kotlin | grep /bin/

kotlin /usr/bin/
kotlin /usr/bin/kotlin
kotlin /usr/bin/kotlinc
kotlin /usr/bin/kotlinc-js
kotlin /usr/bin/kotlinc-jvm

@aartiPl
Copy link
Collaborator

aartiPl commented Aug 28, 2022

Is KOTLIN_HOME set by some installer e. g. SdkMan, or Kotlin installer? In other distros, KOTLIN_HOME is a separate directory that contains bin and lib. Here is a screenshot from Ubuntu:

obraz

So if it is different in archlinux, I guess there will be needed a patch for KScript to correct that behavior on that specific platform.

@CLOVIS-AI
Copy link
Author

You can see the list of files here: https://archlinux.org/packages/community/any/kotlin/ (in the section ‘package contents’).

Basically it is just another system package with binaries in /usr/bin, JARS in /usr/share, etc.

@sfesenko
Copy link
Contributor

  1. everything fine with sdkman, this issue about Archlinux and kotlin package from official Archlinux repo (as mentioned by @CLOVIS-AI )
  2. Assumption, that kotlin binaries are placed in $KOTLIN_HOME/bin is just wrong.
  3. Implementation also doesn't look correct:
    1. GUESS_KOTLIN_HOME=$(KOTLIN_RUNNER=1 JAVACMD=echo kotlinc) ## here it assumes that kotlinc is present in $PATH
    2. KOTLIN_BIN="$KOTLIN_HOME/bin/" ## then it require, that kotlin is present in $KOTLIN_HOME/bin/ (without any checks)
    3. anyway, KOTLIN_HOME value will be ignored later, so nothing will work:

$ kscript
/home/sfesenko/.sdkman/candidates/kscript/current/bin/kscript: line 50: /usr/share/kotlin/bin/kotlin: No such file or directory

$ KOTLIN_HOME=/usr kscript
kscript - Enhanced scripting support for Kotlin on *nix-based systems.
...
Copyright : 2022 Holger Brandl
License : MIT
Version : v4.1.1
Website : https://github.com/holgerbrandl/kscript

$ KOTLIN_HOME=/usr kscript println
[kscript] [ERROR] Compilation of scriplet failed:
[kscript] [ERROR] Command : 'bash -c /usr/share/kotlin/bin/kotlinc -d '/home/sfesenko/.cache/kscript/jar_b58e7483b3914effc9ffa9c6ce4fb765/scriplet.jar' '/home/sfesenko/.cache/kscript/jar_b58e7483b3914effc9ffa9c6ce4fb765/Scriplet.kts' '/home/sfesenko/.cache/kscript/jar_b58e7483b3914effc9ffa9c6ce4fb765/Main_Scriplet.kt''
[kscript] [ERROR] Exit Code : 127
[kscript] [ERROR] Stdout : ''
[kscript] [ERROR] Stderr : 'bash: line 1: /usr/share/kotlin/bin/kotlinc: No such file or directory[nl]'
[kscript] [ERROR]

Since it already assumes, that kotlin/kotlinc are available in $PATH, all logic around $KOTLIN_HOME/bin/ may be removed and issue should be fixed.
Also I believe, that bash is not required to run kotlin/kotlinc (they don't have to be bash scripts)
As alternative option - kscript may embed kotlin/kotlinc logic, and just run java command, so it won't depend on bash / cmd/ kotlin / kotlinc / etc and won't run jvm multiple times (kscript will require either $KOTLIN_HOME or kotlin in $PATH to calculate $KOTLIN_HOME),

@aartiPl
Copy link
Collaborator

aartiPl commented Aug 30, 2022

KScript is using KOTLIN_HOME in CommandResolver.kt in two places, so it is not enough to rely on PATH. I want to rework how scripting execution happens so that dependency on this environment variable might be relaxed.

BTW. Has KScript worked previously on ArchLinux?

@sfesenko
Copy link
Contributor

sfesenko commented Aug 31, 2022

4.0.0 is last version that works
Added dockerfile as test

this succeed

docker build -t kts --build-arg VERSION=4.0.0 .

but this failed

docker build -t kts --build-arg VERSION=4.1.0 .

Dockerfile

FROM archlinux

RUN pacman -Sy --noconfirm \
  which \
  jdk-openjdk \
  unzip \
  zip \
  kotlin 

RUN curl -s "https://get.sdkman.io" | bash

ARG VERSION=4.1.0
RUN . ~/.bashrc && sdk i kscript $VERSION

# Run test
RUN . ~/.bashrc; kscript 'println("Hi")'

ENTRYPOINT . ~/.bashrc; kscript 'println("Hi")'

@aartiPl
Copy link
Collaborator

aartiPl commented Sep 1, 2022

Thanks a lot for checking! I plan to do one more big rework in kscript so that refactoring might resolve this issue. However, I will keep this issue open as it is an important use case.

@DareFox
Copy link

DareFox commented Oct 12, 2022

Tried to fix problem of ignored KOTLIN_HOME=/usr variable value, that @sfesenko mentioned. Turns out kotlinc automatically overwrites it with /usr/share/kotlin value

image

The only way I found to bypass this behavior is to change the name of KOTLIN_HOME in bash launch script to this:

src/kscript

## run it using command substitution to have just the user process once kscript is done
- COMMAND=$("${KOTLIN_BIN}kotlin" -classpath "${JAR_PATH}" kscript.app.KscriptKt "$OSTYPE" "$@")
+ COMMAND=$(export KOTLIN_HOME_BYPASS=$KOTLIN_HOME; "${KOTLIN_BIN}kotlin" -classpath "${JAR_PATH}" kscript.app.KscriptKt "$OSTYPE" "$@")

And read its value from ConfigBuilder
src/main/kotlin/../model/ConfigBuilder.kt

private fun resolveKotlinHome(osType: OsType): OsPath = path(
+   System.getenv("KOTLIN_HOME_BYPASS") ?:
    System.getenv("KOTLIN_HOME") ?:
    ShellUtils.guessKotlinHome(osType) ?:
    throw IllegalStateException("KOTLIN_HOME is not set and could not be inferred from context.")
)

This solved the problem with kscript finding kotlinc binary, but now I get a "NoClassDefFoundError" error because the stdlib JAR files are exist separately in /usr/share/kotlin/libs

Exception in thread "main" java.lang.NoClassDefFoundError: kotlin/script/templates/standard/ScriptTemplateWithArgs
        at java.base/java.lang.ClassLoader.defineClass1(Native Method)
        at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1017)
        at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
        at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555)
        at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458)
        at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452)
        at java.base/java.security.AccessController.doPrivileged(Native Method)
        at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
        at Main_Scriplet$Companion.main(Main_Scriplet.kt:5)
        at Main_Scriplet.main(Main_Scriplet.kt)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.jetbrains.kotlin.runner.AbstractRunner.run(runners.kt:70)
        at org.jetbrains.kotlin.runner.Main.run(Main.kt:188)
        at org.jetbrains.kotlin.runner.Main.main(Main.kt:198)
Caused by: java.lang.ClassNotFoundException: kotlin.script.templates.standard.ScriptTemplateWithArgs
        at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:476)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)

@DareFox
Copy link

DareFox commented Oct 12, 2022

Found fix, just create symlink to kotlinc in /usr/share/kotlin/bin

 sudo mkdir /usr/share/kotlin/bin
 sudo ln -s /usr/bin/kotlin /usr/share/kotlin/bin/kotlin   
 sudo ln -s /usr/bin/kotlinc /usr/share/kotlin/bin/kotlinc   

image

Maybe add this commands to PKGBUILD on Aur?

@aartiPl
Copy link
Collaborator

aartiPl commented Oct 12, 2022

Found fix, just create symlink to kotlinc in /usr/share/kotlin/bin

 sudo mkdir /usr/share/kotlin/bin
 sudo ln -s /usr/bin/kotlin /usr/share/kotlin/bin/kotlin   
 sudo ln -s /usr/bin/kotlinc /usr/share/kotlin/bin/kotlinc   

@DareFox - this is a nice workaround - thanks for bringing it here.

Another workaround would be to install Kotlin using sdkman (alongside the system-installed one), as in kscript installation guide. That should also fix the problem, although I haven't tested it on ArchLinux.

@CLOVIS-AI
Copy link
Author

Maybe add this commands to PKGBUILD on Aur?

Is it first-party? Who should be contacted to do it?

Another workaround would be to install Kotlin using sdkman

ArchLinux users are blessed with a good quality package manager, we tend to avoid installing software by other means for security/stability reasons.

@aartiPl
Copy link
Collaborator

aartiPl commented Oct 22, 2022

@DareFox - I think those symbolic links should be added in the Kotlin package rather than in the KScript. This way, the behavior on the ArchLinux will be consistent with the Kotlin installation directory structure on other systems. And in fact, I think this is the best of all solutions discussed above.

Can you try to push the change to the maintainer of the Kotlin package on ArchLinux?

@aartiPl
Copy link
Collaborator

aartiPl commented Oct 31, 2022

(at) HERE - let me discuss other solutions and why I don't think they are the best way to proceed. Let's assume we have the following structure on disk:

Additional Kotlin versions:
/opt/kotlin/1.7.20/* [bin/, lib/... directory]
/opt/kotlin/1.6.20/* [bin/, lib/... directory]
....
Default ArchLinux Kotlin version:
/usr/bin/kotlin
/usr/bin/kotlinc
/usr/share/kotlin/* [lib/*... directory]

  1. Revert to the state of kscript as it was in version 4.0.0 and before (executing 'kotlinc'/'kotlin' without the absolute path and relying on the 'PATH' env variable)

In such a case, when we change 'KOTLIN_HOME' to one of the versions in '/opt', we will still have in 'PATH' binaries from the other version of Kotlin. So it may be wrong and will not work with libraries in 'KOTLIN_HOME'. We can, of course, change 'PATH' so that it will execute the correct version of 'kotlin' and 'kotlinc' scripts, but IMHO it's not really how "*_HOME" type variables work. They should direct to the whole, consistent environment connected with the Kotlin version.

  1. Use Kotlin compiler jar directly without executing shell scripts

This way, we can eliminate dependency on 'kotlin'/'kotlinc', but we do not have any guarantees that it will not break in new versions of Kotlin: calling jar files directly, without using scripts, is possible (and it works - I have tested it), but it is not a public API of Kotlin, and it can break at any time in a new version of Kotlin.

  1. In the 'kscript' installation package, add symbolic links between Kotlin executables '/usr/bin/' and 'KOTLIN_HOME/bin/'

It's not a 'kscript' thing to provide those links. The best solution is to provide links in the Kotlin package. It is consistent with the way Kotlin is distributed.

To summarize, I am reluctant to change 'kscript' to fix this problem because it seems that the issue should be resolved in the Kotlin package in ArchLinux.

If anyone can contact the Kotlin package maintainer for ArchLinux, and ask to add symbolic links between 'KOTLIN_HOME/bin/' and '/usr/bin/', that would be the best solution.

@aartiPl
Copy link
Collaborator

aartiPl commented Dec 6, 2022

I have added information about the problem in the README file with a link to this ticket.

@aartiPl aartiPl closed this as not planned Won't fix, can't repro, duplicate, stale Dec 6, 2022
@aartiPl
Copy link
Collaborator

aartiPl commented Jan 6, 2023

@ here - I need some help with releasing to ArchLinux. Please have a look at:
#376 (comment)

If you know how to solve that problem, please comment:
#376

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants