Gradle fails to produce TestNG reports if NullPointerException is thrown in listener #29098
Labels
Comments
|
I have also tested this with disabling Gradle reports and specifying different path for TestNG reports as shown below in my Although this disables generating Gradle HTML and XMLreports as it can also be verified from logs: but it still doesn't produce TestNG reports. It only produces following files in specified directory as indicated in logs: Previously I though that TestNG reports were not getting generated because Gradle also generates report in same folder so it might be getting overwritten by Gradle so I have now also tested with specifying different paths for Gradle reports as well as even disabling the gradle reports but it does not solve the problem. What is really strange is this issue only happens with a specific build and not others. Only difference between these builds is that the problematic build has different test groups to include for TestNG and has many tests to run. |
|
I was finally able to triage this issue and it turned out that Gradle was not able to execute tests because a NullPointerException thrown in the TestNG listener implementation class. You can reproduce this issue by throwing a NullPointerException after the test execution in TestNG listener. I have intentionally declared the String as null in listener class to produce the NullPointerException. This causes the gradle test execution to fail. Steps to reproduce:DebugTest.class DebugListener.class build.gradle Run your tests with commandline as It will cause the gradle test execution to fail. You can view the following output in console and also in the HTML reports produced by Gradle. i am not sure whether this behaviour is intended in Gradle. Even if the tests are failing, we should still be able to see the testng-results.xml reports getting properly generated but the only reports that are generated for TestNG are: This error does not occur if NullPointerException directly in test methods. In that case, gradle is still able to handle this gracefully and execute tests. The only time, gradle is unable to execute tests if this issue happens in listener and then we get this error: |
itkhanz
changed the title
|
The issue is in the backlog of the relevant team and is prioritized by them. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Current Behavior
I am facing this issue that when there are large number of tests and test task runs for longer duration, then Gradle test task does not generate TestNG reports (emailable-report.html, testng-results.xml, as well as junitreports and other TestNG related JS and CSS files). I am expecting it to be generated under
build/reports/tests/test/path but any TestNG related files are not created.I have opened a separate discussion theread with more details, screenshots and logs here:
https://discuss.gradle.org/t/gradle-does-not-generate-testng-report-for-longer-builds/48433
Expected Behavior
Gradle should always generate TestNG reports on running
gradle clean testif listerner are enabled in build.gradle as:-or if there is an alternative way to correctly configure the TestNG reports to be generated, then please specify it. I assume TestNG reports are not being generated because somehow Gradle html and xml reports are interfering with it.
Context (optional)
This issue only happens if my gradle task (gradle clean test) takes long time (1 hour +) and has many tests (50+). otherwise it produces TestNG reports successfully if the test task finishes relatively quickly
Steps to Reproduce
Run a Gradle TestNG test task which takes about 02 hours to run and has 60+ tests.
Gradle version
8.7
Build scan URL (optional)
No response
Your Environment (optional)
The text was updated successfully, but these errors were encountered: