Java: error message broken in KDBExceptions (Internal Sorry)

[markus2330]

https://build.libelektra.org/jenkins/blue/organizations/jenkins/libelektra/detail/master/826/pipeline/422

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.6.0:compile (default-compile) on project libelektra4j: Execution default-compile of goal org.apache.maven.plugins:maven-compiler-plugin:3.6.0:compile failed: Plugin org.apache.maven.plugins:maven-compiler-plugin:3.6.0 or one of its dependencies could not be resolved: Failed to collect dependencies at org.apache.maven.plugins:maven-compiler-plugin:jar:3.6.0 -> org.apache.maven:maven-plugin-api:jar:3.0 -> org.apache.maven:maven-artifact:jar:3.0: Failed to read artifact descriptor for org.apache.maven:maven-artifact:jar:3.0: Could not transfer artifact org.apache:apach[  8%] Built target elektra-filecheck-objects

e:pom:6 from/to central (https://repo.maven.apache.org/maven2): /home/jenkins/.m2/repository/org/apache/apache/6/apache-6.pom.part (No such file or directory) -> [Help 1]

[ERROR] 

[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.

[ERROR] Re-run Maven using the -X switch to enable full debug logging.

[ERROR] 

[ERROR] For more information about the errors and possible solutions, please read the following articles:

[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginResolutionException

[markus2330]

Happened again https://build.libelektra.org/jenkins/blue/organizations/jenkins/libelektra/detail/master/835/pipeline

WARNING: An illegal reflective access operation has occurred

WARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$1 (file:/usr/share/maven/lib/guice.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)

WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$1

WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations

WARNING: All illegal access operations will be denied in a future release

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.6.0:compile (default-compile) on project libelektra4j: Execution default-compile of goal org.apache.maven.plugins:maven-compiler-plugin:3.6.0:compile failed: Plugin org.apache.maven.plugins:maven-compiler-plugin:3.6.0 or one of its dependencies could not be resolved: Failed to collect dependencies at org.apache.maven.plugins:maven-compiler-plugin:jar:3.6.0 -> org.apache.maven:maven-core:jar:3.0 -> org.apache.maven:maven-settings-builder:jar:3.0 -> org.sonatype.aether:aether-util:jar:1.7: Failed to read artifact descriptor for org.sonatype.aether:aether-util:jar:1.7: Could not transfer artifact org.sonatype.aether:aether-parent:pom:1.7 from/to central (https://repo.maven.apache.org/maven2): /home/jenkins/.m2/repository/org/sonatype/aether/aether-parent/1.7/aether-parent-1.7.pom.part (No such file or directory) -> [Help 1]

[ERROR] 

[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.

[ERROR] Re-run Maven using the -X switch to enable full debug logging.

[ERROR] 

[ERROR] For more information about the errors and possible solutions, please read the following articles:

[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginResolutionException

make[2]: *** [src/bindings/jna/CMakeFiles/jna.dir/build.make:75: src/bindings/jna/libelektra4j/target/libelektra4j-0.9.0.jar] Error 1

make[1]: *** [CMakeFiles/Makefile2:17116: src/bindings/jna/CMakeFiles/jna.dir/all] Error 2

@Piankero Can you take a look at the maven files?

[ghost]

This error happens randomly and not with every build?

I could only suggest to upgrade the maven-compiler-plugin from 3.6.0 to 3.8.1 and hope that the transitive sonar dependency is available there. The sonar 1.7 dependency is outdated anyway by years (2010)

[markus2330]

Yes, it happens randomly.

Do we need the sonar dependency at all?

Can we upgrade to maven-compiler-plugin 3.8.1 and still support Apache Maven 3.3.9?

[ghost]

Do we need the sonar dependency at all?

The sonar dependency is not used by us directly but by the maven-compiler-plugin.

Can we upgrade to maven-compiler-plugin 3.8.1 and still support Apache Maven 3.3.9?

The version of maven and the version of the maven-compiler-plugin have no correlation at all. You should be able to upgrade it without a problem

[markus2330]

Thank you, could you please upgrade the version?

[markus2330]

It seems like the upgrade did not help, the first build after merging the PR already failed:

https://build.libelektra.org/jenkins/blue/organizations/jenkins/libelektra/detail/master/848/pipeline

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile) on project libelektra4j: Execution default-compile of goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile failed: Plugin org.apache.maven.plugins:maven-compiler-plugin:3.8.1 or one of its dependencies could not be resolved: Failed to collect dependencies at org.apache.maven.plugins:maven-compiler-plugin:jarScanning dependencies of target elektra-lineendings-objects

:3.8.1 -> org.apache.maven:maven-core:jar:3.0 -> org.apache.maven:maven-settings-builder:jar:3.0 -> org.sonatype.aether:aether-impl:jar:1.7: Failed to read artifact descriptor for org.sonatype.aether:aether-impl:jar:1.7: Could not transfer artifact org.sonatype.aether:aether-parent:pom:1.7 from/to central (https://repo.maven.apache.org/maven2): /home/jenkins/.m2/repository/org/sonatype/aether/aether-parent/1.7/aether-parent-1.7.pom.part (No such file or directory) -> [Help 1]

[ERROR] 

[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.

[ERROR] Re-run Maven using the -X switch to enable full debug logging.

[ERROR] 

[ERROR] For more information about the errors and possible solutions, please read the following articles:

[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginResolutionException

Scanning dependencies of target elektra-logchange-objects

Scanning dependencies of target elektra-list-objects

[ghost]

Is the build multithreaded? It would explain the error (https://jira.apache.org/jira/browse/MDEP-518)

[markus2330]

Do you mean if we call maven several times concurrently? This might be possible.

[ghost]

Then this is an open maven issue unfortunately.

[markus2330]

The Java build is very simple, it would be a shame to not get this reliable. Maybe we simply forgot a dependency on CMake level (e.g. between testjna_maven and the binding?).

If it really is an internal problem in maven, in the last post of the maven issue a solution is described, can you try it?

[ghost]

If it really is an internal problem in maven, in the last post of the maven issue a solution is described, can you try it?

The last post describes an extension which severely alters the behavior of maven in terms of jar management. We would just get another dependency into the build which is probably not very stable. Also we probably would limit us with upgrading maven in the future because it alters the behavior like this. Also I currently do not have the resources to patch every build job with this

[markus2330]

As said before, I do not think we need such an extension as our Java builds are completely standard.

More likely is that two maven processes are started at the same time. It should not be too difficult to avoid that JNI is only build if JNA already was built.

[markus2330]

#2967 now contains a full list of problems, I hope we can fix this soon to have one problem less 👍

[markus2330]

happened in https://build.libelektra.org/jenkins/blue/organizations/jenkins/libelektra/detail/PR-1950/70/pipeline

[markus2330]

Failed again in the same PR https://build.libelektra.org/jenkins/blue/organizations/jenkins/libelektra/detail/PR-1950/74/pipeline

[markus2330]

failed again https://travis-ci.org/ElektraInitiative/libelektra/jobs/599129506

[ghost]

failed again https://travis-ci.org/ElektraInitiative/libelektra/jobs/599129506

actually this build job did not fail at all

@sanssecours (or @Chemin1 ): How exactly does the jenkins build work in regards of caches. Does every build load a docker image and downloads all dependencies of maven itself into the container? Or is there a general folder which is used by all builds to reduce disk/network load?

[sanssecours]

I do not know about any special code to download Maven packages on the Jenkins build server. I would assume cmake uses mvn to download the packages for the build, just like it would if you build Elektra locally. I am not really familiar with the Jenkins setup, though.

[ghost]

I would assume cmake uses mvn to download the packages for the build

mvn does implicitly download dependencies when jna is build. Any dependency is usually downloaded in a local .m2 repository under the home path. Does jenkins do any build/downloads etc in a local running docker container or does it use a standard temporary build directory from a jenkins?

[sanssecours]

Does jenkins do any build/downloads etc in a local running docker container or does it use a standard temporary build directory from a jenkins?

I only know that we already add Google Test to most of our Docker images. For example, here is the relevant code for Debian Buster:

libelektra/scripts/docker/debian/buster/Dockerfile

Lines 87 to 94 in 990b866

	ENV GTEST_ROOT=/opt/gtest
	ARG GTEST_VER=release-1.8.1
	RUN mkdir -p ${GTEST_ROOT} \
	&& cd /tmp \
	&& curl -o gtest.tar.gz \
	-L https://github.com/google/googletest/archive/${GTEST_VER}.tar.gz \
	&& tar -zxvf gtest.tar.gz --strip-components=1 -C ${GTEST_ROOT} \
	&& rm gtest.tar.gz

.

[dominicjaeger]

How exactly does the jenkins build work in regards of caches. Does every build load a docker image and downloads all dependencies of maven itself into the container? Or is there a general folder which is used by all builds to reduce disk/network load?

Unfortunately, I don't have the slightest clue.

@Mistreated Do you know more about this?

[ghost]

I am just confused how docker interacts with the jenkins directories. The problem of this issue does only occur if multiple maven instances try to download into the same directory when resolving dependencies. If however this is done via docker this should not happen unless the same volumes are mounted for all docker images which include the mvn directory. This would also explain why this issue only happens on the jenkins server since travis uses VMs which are correctly encapsulated.

This line suggests at least that volumes are mounted but I do not know exactly how it is done on the jenkins as $PWD can be anything.

[mpranj]

This line suggests at least that volumes are mounted but I do not know exactly how it is done on the jenkins as $PWD can be anything.

I think that is just a tutorial, not what's actually happening on our jenkins. Access to the local git mirror is the only mount I see in our Jenkinsfile. I don't think there are any other direct interactions with the host filesystem. Docs are published via SSH and artifacts are stored via archiveArtifacts. It seems less likely to me, that multiple docker instances are accessing the same maven directory. Maybe something different is going on?

[markus2330]

actually this build job did not fail at all

Seems like Travis does not keep the failed build after retry, at least I also cannot find it anymore.

How exactly does the jenkins build work in regards of caches.

It has nothing to do with Jenkins as it also fails locally and on Travis.

The problem of this issue does only occur if multiple maven instances try to download into the same directory when resolving dependencies.

This is a step into the right direction.

As already said, Elektra does at least two mvn calls during build (jni and jna). The dependency between them most likely is incorrect, leading to concurrent executions. Please investigate this.

[ghost]

It has nothing to do with Jenkins as it also fails locally and on Travis.

Ok, this was not clear to me.

As already said, Elektra does at least two mvn calls during build (jni and jna). The dependency between them most likely is incorrect, leading to concurrent executions. Please investigate this.

I opened #3113

[markus2330]

Let us observe if the error occurs again.

[ghost]

Have you seen this error again? We could probably close it and reopen if a build fails

[markus2330]

I did not see it again. So it seems like you fixed it 💯

[mpranj]

@markus2330 seems like this happened on master now. https://build.libelektra.org/blue/organizations/jenkins/libelektra/detail/master/35/pipeline

Output was: log.txt

Is that a temporary failure we can ignore for the release?

[markus2330]

It is maybe not temporary [0] but I think we can ignore it for the release.

[0] #3269 seems to have been temporary because there was some "connection reset" but here I do not see it and KDBTest.test_kdbGet_shouldPass:46 » Internal Sorry, module (null) issued error... very much looks like there was something in the KDB.

The error message is completely wrong, this is an issue for @Piankero.

I reopen to track this new issue (in future better open new issues, reusing issues is always somewhat problematic).

[mpranj]

Sorry, I did not want to re-use issues. I glanced over it and thought it was the same.

[markus2330]

No need to be sorry, I was the one who reused it 😉

[ghost]

The error message is completely wrong, this is an issue for @Piankero.

I cannot reproduce this error unfortunately and also do not have an idea how this can actually happen because Elektra returned -1 to the binding with an "InternalError" (

libelektra/src/bindings/jna/libelektra4j/src/main/java/org/libelektra/exception/mapper/ExceptionMapperService.java

Lines 26 to 29 in 09f2fdc

	else if (errorNumber.equals (InternalException.errorNumber ()))
	{
	return new InternalException (k);
	}

) but is not able to read it a few lines later

libelektra/src/bindings/jna/libelektra4j/src/main/java/org/libelektra/exception/KDBException.java

Lines 54 to 57 in 09f2fdc

	public String getErrorNumber ()
	{
	return errorKey.getMeta ("error/number").getString ();
	}

As if the key with its metadata was deleted in between. Did this error happen again in between the release an now?

[markus2330]

Do you have an explanation why the error message is so distorted?

Please add a Java unit test about an error happening in Elektra (e.g. with the error plugin). The error plugin might also be useful for the Error tutorial you started to write.

[ghost]

Please add a Java unit test about an error happening in Elektra (e.g. with the error plugin). The error plugin might also be useful for the Error tutorial you started to write.

This class actually integration tests every exception with the error plugin.

This tests if the metadata is correctly extracted.

Do you have an explanation why the error message is so distorted?

What you mean by "distorted"? Do you mean the dots at the end or what exactly?

[markus2330]

This class actually integration tests every exception with the error plugin.

Only if an exception is thrown but not which message it renders.

This tests if the metadata is correctly extracted.

This test also has nothing to do with errors coming from Elektra.

What you mean by "distorted"?

See title of this issue and log.txt:

Sorry, module (null) issued error (null):
(null)
Configfile: user/sw/tests/jna/1

	at org.libelektra.KDBTest.test_kdbGet_shouldPass(KDBTest.java:46)

Do you mean the dots at the end or what exactly?

Which part of this error message do you believe to be correct? Also this is completely broken:

Internal Sorry, module (null) issued error..

[ghost]

Which part of this error message do you believe to be correct? Also this is completely broken:

This message here is just a truncated output from maven as summary and maven appends dots to indicate that there is more text to come. It would do that for all tests that have failed. It has nothing to do with the actual message.

This test also has nothing to do with errors coming from Elektra.

It unit tests the correct metadata extraction and an integration test for this seems a bit of an overkill. I can add an extra integration test for the text parsing but I can assure you that this will not fix the existing problem. It looks to be a random error to me and never happened again since then ...

[markus2330]

This message here is just a truncated output from maven as summary and maven appends dots to indicate that there is more text to come. It would do that for all tests that have failed. It has nothing to do with the actual message.

So maven shuffles around the words? Even if it does it does not explain the broken first message.

It unit tests the correct metadata extraction and an integration test for this seems a bit of an overkill. I can add an extra integration test for the text parsing but I can assure you that this will not fix the existing problem. It looks to be a random error to me and never happened again since then ...

"Random errors" only occur if there are some timeout problems or the software is broken. I do not see where in this case there could be a timeout.

But concentrate on an amazing tutorial first.