Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Intermingled test from different test groups will result in missing test cases in junit.xml #1741

Open
AlanRosenthal opened this issue Jan 9, 2023 · 1 comment

Comments

@AlanRosenthal
Copy link

AlanRosenthal commented Jan 9, 2023

Take this example:

TEST_GROUP(group1)
{
};

TEST(group1, test1)
{
    CHECK_EQUAL(1,1);
}

TEST(group1, test2)
{
    CHECK_EQUAL(1,1);
}

TEST(group1, test3)
{
    CHECK_EQUAL(1,2);
}

TEST_GROUP(group2)
{
};

TEST(group2, test1)
{
    CHECK_EQUAL(1,1);
}

running testname -ojunit results in cpputest_group1.xml:

<?xml version="1.0" encoding="UTF-8" ?>
<testsuite errors="0" failures="1" hostname="localhost" name="group1" tests="3" time="0.000" timestamp="2023-01-09T17:43:30">
<properties>
</properties>
<testcase classname="group1" name="test3" assertions="1" time="0.000" file="test.cpp" line="78">
<failure message="test.cpp:80: expected &lt;1&gt;{newline}	but was  &lt;2&gt;{newline}	difference starts at position 0 at: &lt;          2         &gt;{newline}	                                               ^" type="AssertionFailedError">
</failure>
</testcase>
<testcase classname="group1" name="test2" assertions="6" time="0.000" file="test.cpp" line="73">
</testcase>
<testcase classname="group1" name="test1" assertions="6" time="0.000" file="test.cpp" line="68">
</testcase>
<system-out></system-out>
<system-err></system-err>
</testsuite>

if we swap mix the groups a bit:

TEST_GROUP(group1)
{
};

TEST(group1, test1)
{
    CHECK_EQUAL(1,1);
}

TEST(group1, test2)
{
    CHECK_EQUAL(1,1);
}

TEST_GROUP(group2)
{
};

TEST(group2, test1)
{
    CHECK_EQUAL(1,1);
}

TEST(group1, test3)
{
    CHECK_EQUAL(1,2);
}

the resulting xml file doesn't contain test3!

<?xml version="1.0" encoding="UTF-8" ?>
<testsuite errors="0" failures="0" hostname="localhost" name="group1" tests="2" time="0.000" timestamp="2023-01-09T17:46:18">
<properties>
</properties>
<testcase classname="group1" name="test2" assertions="6" time="0.000" file="test.cpp" line="73">
</testcase>
<testcase classname="group1" name="test1" assertions="6" time="0.000" file="test.cpp" line="68">
</testcase>
<system-out></system-out>
<system-err></system-err>
</testsuite>

the test case is run, and the error is reported to the console.

@basvodde
Copy link
Member

Same as this one: #168 right?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants