java.io.IOException: Cannot use custom test directory 'build/Test/*/junit' that does not exist!

[garcanam]

Describe the bug
The content of the project is the Tosca Quickstarter.
An error occurs at the end of the build process because the system doesn't find out the custom test directory defined in Jenkins file.
Don't hesitate to contact me if you need more info.

Expected behavior
The build finishes SUCCESS.

Screenshots
Build executed with the current ods-jenkins-shared-library. (Finished: FAILURE)

Build executed with the previous ods-jenkins-shared-library (before the last upgrade). (Finished: SUCCESS)

Affected version:

• OpenShift: 4.10
• OpenDevStack: 4.1

Additional context
After a quick assessment comparing both ods-jenkins-shared-library versions:

Regarding the numbers that appear in the picture:

1. [Red Side] It throws an exception in case the defined custom path doesn't exist.
2. [Red Side] However, if the path is not customized, the system creates it.
3. [Green Side] The system always creates the defined directory (customized or not).

We don't know if it is an error or if it is correct and all testing quickstarters must change something to work properly.

[braisvq1996]

It seems other testing quickstarters can also be affected.
I changed the testResults param to "build/Test/dummy/junit" and pipeline fails

[renedupont]

I'm encountering it as well
java.io.IOException: Cannot use custom test directory 'docker/build/test-results/test' that does not exist!
but only for reruns of the pipeline (without any changes to the code) so that the image build is skipped due to the odsComponentFindOpenShiftImageOrElse(context) stage.

[clemensutschig]

@renedupont - What's the expectation here - that ods creates it? .. my intention (it was my change) is that one MUST point this to an existing dir :)

[renedupont]

@clemensutschig I assume that error occurs in my case because the custom test dir (with reports as content) is normally being created during the build stage which is never happening when odsComponentFindOpenShiftImageOrElse(context) finds the image and hence skips the build stage?

In that case, my expectation would have been that the pipeline run does not try to stash test reports that have not been created and then fails because of it.
It is also kinda unexpected that this happens after there has been a Finished ODS PIpeline message:

[my-proj] ***** Finished ODS Pipeline ***** (took 146889 ms)
[Pipeline] }
[Pipeline] // stage
[Pipeline] echo
Collecting test results, if available ...
[Pipeline] fileExists
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] End of Pipeline
java.io.IOException: Cannot use custom test directory 'docker/build/test-results/test' that does not exist!

[clemensutschig]

@renedupont - that's fair, it's a finally block - that's why you see it the way you do :) we could just go and create the dir early on (which is maybe the quickest fix ... )

[renedupont]

@clemensutschig Yes, true :).
I am not understanding yet though why this test-results stashing is happening at that point at all. There won't be anything in that folder to stash in any case if the build stage was skipped due to odsComponentFindOpenShiftImageOrElse(context).
Wouldn't it make more sense to check at this point whether the pipeline ran through odsComponentFindOpenShiftImageOrElse(context) and in case it did, to skip stashing test-results?

[clemensutschig]

we cant "see" what stages you ran . we can just check if the dir is there - and dont fail

[renedupont]

Sounds good 👍🏻.
Just thinking about the case where someone misconfigured the build by putting the test-results neither in the default nor a custom dir... should the pipeline fail in that case? I think we wouldn't be able to differentiate between that and my case above.

[fkressmann]

I have also just stumbled across this problem.
I second what @renedupont said, when not running tests (for whatever reason), a finally stage that collects reports should not fail if there are none.
Especially cause the standard dir gets created, too if it doesn't exist, it's kinda inconsequent to fail in case a custom dir doesn't.
I don't like the idea to just create the dir in the Jenkinsfile cause that would need to be done independent from the tests at the beginning and clutters the Jenkinsfile. I don't want to setup test dirs if I'm not going to test.

My preferred solution: Warn if the custom dir doesn't exist and either create it or skip the rest of the test result collection.
/Felix