-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ITestListener's methods get called multiple times for one test, when there are multiple tests in the xml #956
Comments
Hello, I have a similar issue, my testng.xml looks like the following: <!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="suite1" verbose="1">
<listeners>
<listener class-name="org.testng.reporters.SuiteHTMLReporter"/>
<listener class-name="org.testng.reporters.TestHTMLReporter"/>
</listeners>
<test name="package1">
<classes>
<class name="xxx.xxx.xxx.Account"/>
<class name="xxx.xxx.xxx.Corporate"/>
<class name="xxx.xxx.xxx.Department"/>
<class name="xxx.xxx.xxx.Login"/>
<class name="xxx.xxx.xxx.Subscriber"/>
</classes>
</test>
</suite> I have a Listener which I use for all the classes above, that implements ITestListener. Every time a test is run, I see the following output in the log:
The methods inside CustomListener look like the following: @Override
public void onTestStart(ITestResult iTestResult) {
log.info("Test started: " + iTestResult.getName());
}
@Override
public void onStart(ITestContext iTestContext) {
log.info("Start of execution " + iTestContext.getName());
} The problem occurs with all the methods inside the listener, i.e. onTestSuccess, onTestFailure, etc. Any ideas? |
I had this very same problem, and got round it by adding a boolean e.g. testStartListenerYetToRun to the base class and setting it to True in the @BeforeMethod (also in the base class), and adding an if condition to the CustomListener e.g.: @Override
public void onTestStart(ITestResult iTestResult) {
if (((AbstractTestNGOrderGridTests)iTestResult.getInstance()).testStartListenerYetToRun) {
((AbstractTestNGOrderGridTests) iTestResult.getInstance()).testStartListenerYetToRun = false;
log.info("Test started: " + iTestResult.getName());
}
} You will need a second boolean for the onStart method. The methods will of course still be invoked once per class but at least the code will only be executed once. |
@libertylocked - Is the below sample what you were using as well ? @stevie1877 - Are you trying to do this on TestNG 6.10 ? If not, can you please try this without your work around and see if you are able to reproduce this ? If you are still able to recreate it, it would be great if you could please help share a sample that I can use to reproduce this issue, so that we can get this fixed. Here's what I have as test classes and suite xml file that I tried using to recreate the problem [ using TestNG v6.10 ] I am not able to reproduce this problem using the below samples on TestNG 6.10 package org.rationale.emotions.github.issue956;
import org.testng.annotations.Listeners;
import org.testng.annotations.Test;
@Listeners(ListenerFor956.class)
public class AccountTest {
@Test
public void testMethod() {
System.err.println("accountTest()");
}
} package org.rationale.emotions.github.issue956;
import org.testng.annotations.Listeners;
import org.testng.annotations.Test;
@Listeners (ListenerFor956.class)
public class CorporateTest {
@Test
public void testMethod() {
System.err.println("corporateTest()");
}
} package org.rationale.emotions.github.issue956;
import org.testng.ITestContext;
import org.testng.ITestListener;
import org.testng.ITestResult;
public class ListenerFor956 implements ITestListener {
public ListenerFor956() {
System.err.println("New instance created ");
}
@Override
public void onTestStart(ITestResult result) {
String print = result.getTestClass().getRealClass().getName() + "." + result.getMethod()
.getConstructorOrMethod().getMethod().getName() + "()";
System.err.println("Test started: " + print);
}
@Override
public void onTestSuccess(ITestResult result) {}
@Override
public void onTestFailure(ITestResult result) {}
@Override
public void onTestSkipped(ITestResult result) {}
@Override
public void onTestFailedButWithinSuccessPercentage(ITestResult result) {}
@Override
public void onStart(ITestContext context) {
System.err.println("Start of execution <" + context.getName() + ">");
}
@Override
public void onFinish(ITestContext context) {}
} <?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="1265_Suite" parallel="false" verbose="2">
<test name="956">
<classes>
<class name="org.rationale.emotions.github.issue956.AccountTest"/>
<class name="org.rationale.emotions.github.issue956.CorporateTest"/>
</classes>
</test>
</suite> Here's the output
ping @juherr (Just keeping your fyi'ed) |
Thanks @krmahadevan , no I can't replicate the issue using 6.10 - I have been using TestNG 6.1.1, so quite an old version. I will update to 6.10 as this clearly solves the problem. Many thanks. |
@juherr - I guess we can close this issue as well. |
@krmahadevan Could you check if the current tests are covering this case? |
I am experiencing this issue with TestNG 6.11.0 |
@snehalizaj Could you share a project sample? |
For what it's worth, I could not duplicate this machine locally on my windows box, but I was able to get a repro for our jenkins machine for testNG 6.10 |
What about v6.11 or the latest v6.12 ? ( 6.12 is only on Jcenter and is yet
to make it to Maven Central).
Can you please try and let us know? Also pls help share a sample to
recreate the problem.
|
Upgrading TestNG from 6.9 to 6.11 solved it for me. Thanks! |
Hi! I'm using latest version of TestNG 6.11, but it does not resolve this issue for me. My xml config looks so: <?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Test all" >
<listeners>
<listener class-name="com.selenium.test.testng.listeners.TestListener"/>
</listeners>
<test name="CheckoutDesktop">
<classes>
<class name="com.selenium.test.testng.tests.CheckoutTestsDesktop"/>
</classes>
</test>
<test name="CheckoutMobile">
<classes>
<class name="com.selenium.test.testng.tests.CheckoutTestsMobile"/>
</classes>
</test>
</suite> When I run test in Intellij IDEA all methods of TestListener called two times. |
@ils-808 - Do you have a test that we can use to reproduce the problem ? |
@krmahadevan, I found that it was my fault. Version 6.11 works fine, thank you! |
I am facing the same issue . In my scenario, there is only one testcase in single class. I am using Testing version 7.1.0 and I am facing the issue. I have tried testing version below 6.10 but facing the same. Kindly help. |
@Abhijitdatta pls share a sample that can be used to reproduce the problem |
TestNG version on eclipseIDE: 7.1.0 |
@seleniumsb - If the problem is only when running tests via Eclipse, then you should be filing an issue on https://github.com/cbeust/testng-eclipse/issues If this problem can be reproduced even via running from maven command line, then please create a new bug with a full fledged example that can be used to reproduce the problem. |
@krmahadevan , this is following scenario. This is the simple Testcase @Test
public void getTitleTest()
{
String actualTitle = driver.getTitle();
System.out.println(actualTitle);
String expectedTitle = "Free CRM #1 cloud software for any business large or small123";
Assert.assertEquals(actualTitle, expectedTitle);
} Base class scereenshot method public void getScreenshot(String testcasename, WebDriver driver) throws IOException
{
System.out.println("in screenshot method");
TakesScreenshot ts = (TakesScreenshot)driver;
System.out.println("in line1");
File source = ts.getScreenshotAs(OutputType.FILE);
System.out.println("in line2");
String destinationFile = System.getProperty("user.dir")+"\\reports\\"+testcasename+".png";
FileUtils.copyFile(source, new File (destinationFile));
} intentionally expected title is given wrong to capture screenshot.. @test method is getting failed for two times . for 1st time, its assertion issue and for 2nd time, it's null pointer exception. |
@Abhijitdatta - Thats not a sample that can be used to reproduce the problem. Please share a simple standalone complete example (without any 3rd party dependencies such as selenium) with the listener that is getting invoked multiple times, so that we can reproduce the problem. The information shared is NOT sufficient to debug |
I've reproduced the issue.
Two test classes:
Two tests declared in testng.xml:
pom.xml:
When I call maven clean test on the output i can see that after first test listener method was called twice and after second test it was called once:
When I declare only one test in testng.xml:
I can see that listener method was called once:
|
This is fixed via PR #2222 and is available in TestNG |
Similar to #171, this applies to ITestListener instances.
When there are multiple and the xmls all use the same test listener, onFinish() gets called multiple times for one test, which does not seem intended.
The text was updated successfully, but these errors were encountered: