Quickstart atf c test

Writing an ATF test in C

The ATF test framework provides a library and API for writing test programs. Currently, ATF programs can only be written in C, C++, or Bourne shell script.

Example ATF test - C

The following example consists of one executable program, atf_tests2, and one Kyuafile which references the executable. The executable and the Kyuafile must be in the same directory.

atf_tests2.c - test program with multiple test cases

/*
 * This test program must be compiled into a binary named atf_test2.
 *
 * For example:
 *    cc -o atf_tests2 -I/usr/local/include -L/usr/local/lib atf_test2.c
 *
 */
/*
 * atf-c.h must be included for the ATF C API.
 *
 */
#include <atf-c.h>

#include <stdio.h>
#include <string.h>

/*
 *  Test 1
 *    - To define a test, we must invoke:
 *          ATF_TC()        -> constructs the test case
 *          ATF_TC_HEAD()   -> define metadata for test csae
 *          ATF_TC_BODY()   -> implement the test case
 *
 *   - At the end of the program, we must implement ATF_TP_ADD_TCS()
 *     and call ATF_ADD_TP_TC() to add this test case to the
 *     list of test cases which will be run. 
 */
ATF_TC(test1);
ATF_TC_HEAD(test1, tc)
{
	/*
	 * Define a textual description for the test case,
	 * which will be included in test reports.
	 */
	atf_tc_set_md_var(tc, "descr", "This is test 1");
}
ATF_TC_BODY(test1, tc)
{
	/*
	 * Implement the code for test1
	 *
	 */
	char buf[255];

	/* The ATF_REQUIRE() macro is used to check
	 * the return code of snprintf().  If it fails, the test case
	 * exits, and the error is propagated to kyua. 
	 */
	ATF_REQUIRE(snprintf(buf, sizeof(buf), "%s %d",
	    "Test", 1) > 0);

	/* The ATF_CHECK_STREQ() macro is used to check
	 * the contents of the buffer.  Alternatively,
	 * we could have done ATF_REQUIRE(strcmp("Test 1", buf));
	 */
	ATF_CHECK_STREQ("Test 1", buf);

	/* At the end of a test case BODY, we should not return
	 * or exit.
	 */
}

/*
 *  Test 2
 */
ATF_TC(test2);
ATF_TC_HEAD(test2, tc)
{
	atf_tc_set_md_var(tc, "descr", "This test case validates the proper "
	    "truncation of the output string from snprintf when it does not "
	    "fit the provided buffer.");
}

ATF_TC_BODY(test2, tc)
{
	char buf[255];

	/* This is a similar test to the above, but in this case we do the
	 * test ourselves and forego the ATF_* macros.  Note that we use the
	 * atf_tc_fail() function instead of exit(2) or similar because we
	 * want Kyua to have access to the failure message.
	 *
	 * In general, prefer using the ATF_* macros wherever possible.  Only
	 * resort to manual tests when the macros are unsuitable (and consider
	 * filing a feature request to get a new macro if you think your case
	 * is generic enough). */
	if (snprintf(buf, sizeof(buf), "0123456789abcdef") != 16)
		atf_tc_fail("snprintf did not return the expected number "
		    "of characters");

	ATF_CHECK(strcmp(buf, "012345678") == 0);
}

/*
 *  Test 3
 */
ATF_TC(test3);
ATF_TC_HEAD(test3, tc)
{
	atf_tc_set_md_var(tc, "descr", 
	    "This is writes a sequence of bytes to a file, then verifies the file.");
}
ATF_TC_BODY(test3, tc)
{
	const char *test_sequence = "This is a test sequence\n";

	FILE *fp = fopen("testfile.txt", "w");
	ATF_REQUIRE(fprintf(fp, "%s", test_sequence) > 0);
	fclose(fp);

	/* We can use the atf_utils_compare_file() helper function
	 * in the ATF C library.
	 */
	ATF_REQUIRE(atf_utils_compare_file("testfile.txt", test_sequence));
}

/*
 * We must define an ATF_TP_ADD_TCS method which
 * adds each testcase to the list which will be run.
 *
 * This function should not do anything other than
 * this registration.
 */
ATF_TP_ADD_TCS(tp)
{
	ATF_TP_ADD_TC(tp, test1);
	ATF_TP_ADD_TC(tp, test2);
	ATF_TP_ADD_TC(tp, test3);

	return atf_no_error();
}

/*
 * The test case .c file must not define a main() method
 *
 */

Kyuafile

-- Comments in Kyuafiles must start with two hyphens

-- The syntax version must be defined to 2 at the
-- beginning of all Kyuafiles. 
syntax(2)

-- The name of the test suite must be defined.
test_suite('suite2')

-- This specifies the test programs
-- The atf_tests.c program must be compiled
-- to an executable named atf_tests2.
atf_test_program{name='atf_tests2'}

Listing the tests

To list the tests which will be run, type:

kyua list

The output will look like this:

atf_tests2:test1
atf_tests2:test2
atf_tests2:test3

Running the tests

To run the all tests, type:

kyua test

The output of the test run will look like:

atf_tests2:test1  ->  passed  [0.005s]
atf_tests2:test2  ->  failed: 1 checks failed; see output for more details  [0.005s]
atf_tests2:test3  ->  passed  [0.005s]

Results file id is Users_crodrigues_kyua.wiki_QuickStart.20141027-210832-970162
Results saved to /home/crodrigues/.kyua/store/results.Users_crodrigues_kyua.wiki_QuickStart.20141027-210832-970162.db

2/3 passed (1 failed)

To run only the first test, type:

kyua test atf_tests2:test1

The output of the test run will look like:

atf_tests2:test1  ->  passed  [0.014s]

Results file id is usr_home_crodrigues_kyua-quickstart_kyua.wiki_QuickStart.20141009-003148-760012
Results saved to /home/crodrigues/.kyua/store/results.usr_home_crodrigues_kyua-quickstart_kyua.wiki_QuickStart.20141009-003148-760012.db

1/1 passed (0 failed)

To run the third test case outside of kyua (for debugging purposes), type:

./atf_tests2 test3

The output of the test will look like:

./atf_tests2 test3
atf_tests2: WARNING: Running test cases outside of kyua(1) is unsupported
atf_tests2: WARNING: No isolation nor timeout control is being applied; you may get unexpected failures; see atf-test-case(4)
passed

Generating test reports

After running the tests, you can generate test reports.

• To generate a report in text format, type:

    kyua report
  

• To generate a report in HTML format, type:

    kyua report-html
  

• To generate a report in JUnit XML, type:

    kyua report-junit
  

Further references

• ATF C API
• ATF C++ API
• kyua ATF test interface
• The FreeBSD source tree has a heavily commented example ATF test program.