Writing Unit Tests

ArrayFire uses the Google Test framework for implementing unit tests for the library functions. Pick any C++ source file in the folder arrayfire/test to get a quick look of how the unit tests in general look like. You can follow the below procedure to implement the unit tests. Using these guidelines is not mandatory, they are only provided in case someone needs a starting point.

• Name the unit test source file after the new function you are adding. For example, if you are implementing median filter and the API for it says medfilt, name the file medfilt.cpp.
• Add the license statement.
• Make sure to include the following headers
  • gtest/gtest.h - to use Google Test framework.
  • arrayfire.h - to use arrayfire library.
  • af/dim4.hpp & af/traits.hpp - these are auxiliary headers used by ArrayFire.
  • testHelpers.hpp - this is optional, but recommended. It has utility functions you can use to read test data from disk, as well as macros that simplify assertion of two af::array objects' (or a std::vector and an af::array) equality in terms of type, size, and elements (either strict equality or "near" equality). It also has a macro for simplifying assertion of whether a C API function returns a successful error code (AF_SUCCESS).
• Create a template class such as below to setup tests for multiple types.

template<typename T>
class MedianFilter : public ::testing::Test
{ // You can name the class after the function 
  // you writing tests for
    public:
        virtual void SetUp() {
            // Any initialization that all tests 
            // need to see can go here
        }
};

• Create the list of data types for which you need to test the function.

    typedef ::testing::Types<float, double, int, 
                             uint, char, uchar> TestTypes;

• Register the types with the class you created earlier.

TYPED_TEST_CASE(MedianFilter, TestTypes);

• That's it, that is pretty much what you need to write a unit test. Your unit test skeleton would like below.

TYPED_TEST(MedianFilter, SYMMETRIC_PAD) 
// second argument to TYPED_TEST is your test name
{   
    // Your testing code goes here.

    // Use the keyword TypeParam to refer to the types 
    // you registered with class MedianFilter

    // The tests are instantiated and run for each type you have 
    // registered during the call TYPED_TEST_CASE
}

• You can also use TEST() macro instead of TYPED_TEST() in which case none of the above steps are required to write a unit test. Your unit test would like below. Depending on the type of testing style your function requires choose whatever fits yours needs. In the case of TEST(), one would have to write tests for each type individually which may be what some one needs in certain situations.

TEST(MedianFilter, SYMMETRIC_PAD_F32)
{
 // Your testing code goes here
}

• To simplify your testing code, you can use the provided custom macros in testhelpers.hpp for asserting two af::array objects' equality (in either the C or C++ API). There's also a version of the macro that compares a std::vector with an af::array, in case the reference data must be in the form of a vector. In both cases, the macros compare the reference vector/array and the output array's types, dimension sizes, and elements. The macro also displays easy-to-read error messages if any of those criteria differs between the reference and the output.
  • There are 5 macro calls:
    • ASSERT_ARRAYS_EQUAL(EXPECTED, ACTUAL), for strict equality between two arrays.
    • ASSERT_VEC_ARRAY_EQ(EXPECTED_VEC, EXPECTED_ARR_DIMS, ACTUAL_ARR), for strict equality between a vector and an array. Notice that you have to provide (in the second argument) the dimensions of the expected array that the vector represents. The dimensions must be a af::dim4 type.
    • ASSERT_ARRAYS_NEAR(EXPECTED, ACTUAL, MAX_ABSDIFF), for approximate equality between two arrays. The last argument is the maximum decimal threshold that the two arrays can deviate from each other and still be considered an "equal". This threshold must be a float type.
    • ASSERT_VEC_ARRAY_NEAR(EXPECTED_VEC, EXPECTED_ARR_DIMS, ACTUAL_ARR, MAX_ABSDIFF), for approximate equality between a vector and an array.
    • ASSERT_SUCCESS(CALL), for C API calls only, checks if the function returns AF_SUCCESS as its error code.
  • The error messages display the kind of inequality (type, dimensions, or elements) that the reference and output have, the variable/expression names involved, and any relevant information about the unequal criteria. In the case of inequality of the elements, the error message displays a 1-D slice of the arrays along the first dimension of the inequality's position. The slice displays five elements before and after the inequality's position, and is truncated if the context width goes beyond the beginning or end of the arrays. Here's an example (notice that the actual inequality's indices, output value, and reference value are highlighted using brackets):

VALUE DIFFERS at (125, 82):
Viewing slice (120:130, 82), dims are (200, 100)
        120 121 122 123 124 [125] 126 127 128 129 130
 out: { 173  50 245 245  75   [2]  10  10  76 239 188 }
gold: { 173  50 245 245  75 [138]  10  10  76 239 188 }

Existing unit tests under arrayfire/test/*.cpp are always a good starting point for getting an idea of how unit tests can be written if you are not familiar with Google Test framework.

After writing unit tests, one would need to provide ArrayFire build system some reference data against which the output generated by the function implementations should be tested. You can use the formats listed in this page to upload your test data to the arrayfire-data repository.