testng-failed.xml does not contain failed methods from all the classes while running the tests in parallel.

[deepakguna]

Problem Statement

testng-failed.xml does not contain failed methods from all the classes while running the tests in parallel.

Any relate message in "Error Log" view

"Windows -> Show View -> Others -> Error Log"

The Dependency Management tool for your project

• [ *] Maven
• Gradle
• Ant
• Eclipse Buildpath (aka. Use "TestNG Library" for your project in Eclipse)

Operating System

• [ *] Windows
• Linux
• OSX

TestNG Version
7.1.0

Eclipse version
Version: 2019-06 (4.12.0)
Build id: 20190614-1200

TestNG for Eclipse Plugin
Version: TestNG 7.0.0.201908240652

Expected behavior
testng-failed.xml should contain failed methods from all the classes while running the tests in parallel.

JDK 13.0.1 Testng 7.1.0

When running the tests with Testng there are failures from the methods in both Class A & B, how ever testng-failed.xml contains only either of the class? What may go wrong here

Actual behavior
testng-failed.xml is inconsistent while running in parallel. On Many occasion it includes failures from only one test/class while there are failures in other test/class.

Test case sample
I have two test classes TestRed and TestRed . Each of the class have 2 @test methods and one of them will fail. testng-failed.xml is inconsistent while running in parallel. On Many occasion it includes failures from only one test/class while there are failures in other test/class.

package test;

import org.testng.Assert;
import org.testng.annotations.Test;

public class TestRed {

@Test
public void testAInRed() {
	Assert.assertTrue(false, "will fail");
}
@Test
public void testBInRed() {
	Assert.assertTrue(true, "will pass");
}

}
Here is my suite xml

Below is the testng-failed.xml :

I was not able to reproduce this issue with the test runner. However i was consistently able to reproduce this issue 2/5 times by running from the suite.xml

It works fine in TestNG v 6.14.3 but not in 7.1.0