CeedlingPacket.md

[All code is copyright © 2010-2021 Ceedling Project by Mike Karlesky, Mark VanderVoord, and Greg Williams.

This Documentation Is Released Under a Creative Commons 3.0 Attribution Share-Alike License]

What the What?

Assembling build environments for C projects - especially with automated unit tests - is a pain. Whether it's Make or Rake or Premake or what-have-you, set up with an all-purpose build environment tool is tedious and requires considerable glue code to pull together the necessary tools and libraries. Ceedling allows you to generate an entire test and build environment for a C project from a single YAML configuration file. Ceedling is written in Ruby and works with the Rake build tool plus other goodness like Unity and CMock — the unit testing and mocking frameworks for C. Ceedling and its complementary tools can support the tiniest of embedded processors, the beefiest 64 bit power houses available, and everything in between.

For a build project including unit tests and using the default toolchain gcc, the configuration file could be as simple as this:

:project:
  :build_root: project/build/
  :release_build: TRUE

:paths:
  :test:
    - tests/**
  :source:
    - source/**

From the command line, to build the release version of your project, you would simply run ceedling release. To run all your unit tests, you would run ceedling test:all. That's it!

Of course, many more advanced options allow you to configure your project with a variety of features to meet a variety of needs. Ceedling can work with practically any command line toolchain and directory structure – all by way of the configuration file. Further, because Ceedling piggy backs on Rake, you can add your own Rake tasks to accomplish project tasks outside of testing and release builds. A facility for plugins also allows you to extend Ceedling's capabilities for needs such as custom code metrics reporting and coverage testing.

What's with this Name?

Glad you asked. Ceedling is tailored for unit tested C projects and is built upon / around Rake (Rake is a Make replacement implemented in the Ruby scripting language). So, we've got C, our Rake, and the fertile soil of a build environment in which to grow and tend your project and its unit tests. Ta da - Ceedling.

What Do You Mean "tailored for unit tested C projects"?

Well, we like to write unit tests for our C code to make it lean and mean (that whole Test-Driven Development thing). Along the way, this style of writing C code spawned two tools to make the job easier: a unit test framework for C called Unity and a mocking library called CMock. And, though it's not directly related to testing, a C framework for exception handling called CException also came along.

These tools and frameworks are great, but they require quite a bit of environment support to pull them all together in a convenient, usable fashion. We started off with Rakefiles to assemble everything. These ended up being quite complicated and had to be hand-edited or created anew for each new project. Ceedling replaces all that tedium and rework with a configuration file that ties everything together.

Though Ceedling is tailored for unit testing, it can also go right ahead and build your final binary release artifact for you as well. Or, Ceedling and your tests can live alongside your existing release build setup. That said, Ceedling is more powerful as a unit test build environment than it is a general purpose release build environment; complicated projects including separate bootloaders or multiple library builds, etc. are not its strong suit.

Hold on. Back up. Ruby? Rake? YAML? Unity? CMock? CException?

Seem overwhelming? It's not bad at all, and for the benefits tests bring us, it's all worth it.

Ruby is a handy scripting language like Perl or Python. It's a modern, full featured language that happens to be quite handy for accomplishing tasks like code generation or automating one's workflow while developing in a compiled language such as C.

Rake is a utility written in Ruby for accomplishing dependency tracking and task automation common to building software. It's a modern, more flexible replacement for Make). Rakefiles are Ruby files, but they contain build targets similar in nature to that of Makefiles (but you can also run Ruby code in your Rakefile).

YAML is a "human friendly data serialization standard for all programming languages." It's kinda like a markup language, but don't call it that. With a YAML library, you can serialize data structures to and from the file system in a textual, human readable form. Ceedling uses a serialized data structure as its configuration input.

Unity is a unit test framework for C. It provides facilities for test assertions, executing tests, and collecting / reporting test results. Unity derives its name from its implementation in a single C source file (plus two C header files) and from the nature of its implementation - Unity will build in any C toolchain and is configurable for even the very minimalist of processors.

CMock is a tool written in Ruby able to generate entire mock functions in C code from a given C header file. Mock functions are invaluable in interaction-based unit testing. CMock's generated C code uses Unity.

CException is a C source and header file that provide a simple exception mechanism for C by way of wrapping up the setjmp / longjmp standard library calls. Exceptions are a much cleaner and preferable alternative to managing and passing error codes up your return call trace.

Notes

• YAML support is included with Ruby - requires no special installation or configuration.

• Unity, CMock, and CException are bundled with Ceedling, and Ceedling is designed to glue them all together for your project as seamlessly as possible.

Installation & Setup: What Exactly Do I Need to Get Started?

As a Ruby gem:

1. Download and install Ruby

2. Use Ruby's command line gem package manager to install Ceedling: gem install ceedling (Unity, CMock, and CException come along with Ceedling for free)

3. Execute Ceedling at command line to create example project or an empty Ceedling project in your filesystem (executing ceedling help first is, well, helpful).

Gem install notes:

1. Steps 1-2 are a one time affair for your local environment. When steps 1-2 are completed once, only step 3 is needed for each new project.

Getting Started after Ceedling is installed:

1. Once Ceedling is installed, you'll want to start to integrate it with new and old projects alike. If you wanted to start to work on a new project named foo, Ceedling can create the skeleton of the project using ceedling new foo. Likewise if you already have a project named bar and you want to integrate Ceedling into it, you would run ceedling new bar and Ceedling will create any files and directories it needs to run.

2. Now that you have Ceedling integrated with a project, you can start using it. A good starting point to get use to Ceedling either in a new project or an existing project is creating a new module to get use to Ceedling by issuing the command ceedling module:create[unicorn].

General notes:

1. Certain advanced features of Ceedling rely on gcc and cpp as preprocessing tools. In most linux systems, these tools are already available. For Windows environments, we recommend the mingw project (Minimalist GNU for Windows). This represents an optional, additional setup / installation step to complement the list above. Upon installing mingw ensure your system path is updated or set [:environment][:path] in your project.yml file (see environment section later in this document).

2. To use a project file name other than the default project.yml or place the project file in a directory other than the one in which you'll run Rake, create an environment variable CEEDLING_MAIN_PROJECT_FILE with your desired project file path.

3. To better understand Rake conventions, Rake execution, and Rakefiles, consult the Rake tutorial, examples, and user guide.

4. When using Ceedling in Windows environments, a test file name may not include the sequences “patch” or “setup”. The Windows Installer Detection Technology (part of UAC), requires administrator privileges to execute file names with these strings.

Now What? How Do I Make It GO?

We're getting a little ahead of ourselves here, but it's good context on how to drive this bus. Everything is done via the command line. We'll cover conventions and how to actually configure your project in later sections.

To run tests, build your release artifact, etc., you will be interacting with Rake on the command line. Ceedling works with Rake to present you with named tasks that coordinate the file generation and build steps needed to accomplish something useful. You can also add your own independent Rake tasks or create plugins to extend Ceedling (more on this later).

• ceedling [no arguments]:

  Run the default Rake task (conveniently recognized by the name default by Rake). Neither Rake nor Ceedling provide a default task. Rake will abort if run without arguments when no default task is defined. You can conveniently define a default task in the Rakefile discussed in the preceding setup & installation section of this document.

• ceedling -T:

  List all available Rake tasks with descriptions (Rake tasks without descriptions are not listed). -T is a command line switch for Rake and not the same as tasks that follow.

• ceedling <tasks...> --trace:

  For advanced users troubleshooting a confusing build error, debug Ceedling or a plugin, --trace provides a stack trace of dependencies walked during task execution and any Ruby failures along the way. Note that --trace is a command line switch for Rake and is not the same as tasks that follow.

• ceedling environment:

  List all configured environment variable names and string values. This task is helpful in verifying the evaluation of any Ruby expressions in the [:environment] section of your config file. : Note: Ceedling may set some convenience environment variables by default.

• ceedling paths:*:

  List all paths collected from [:paths] entries in your YAML config file where * is the name of any section contained in [:paths]. This task is helpful in verifying the expansion of path wildcards / globs specified in the [:paths] section of your config file.

• ceedling files:assembly

• ceedling files:include

• ceedling files:source

• ceedling files:support

• ceedling files:test

  List all files and file counts collected from the relevant search paths specified by the [:paths] entries of your YAML config file. The files:assembly task will only be available if assembly support is enabled in the [:release_build] section of your configuration file.

• ceedling options:*:

  Load and merge configuration settings into the main project configuration. Each task is named after a *.yml file found in the configured options directory. See documentation for the configuration setting [:project][:options_paths] and for options files in advanced topics.

• ceedling test:all:

  Run all unit tests (rebuilding anything that's changed along the way).

• ceedling test:build_only:

  Build all unit tests, object files and executable but not run them.

• ceedling test:delta:

  Run only those unit tests for which the source or test files have changed (i.e. incremental build). Note: with the [:project][:use_test_preprocessor] configuration file option set, runner files are always regenerated limiting the total efficiency this text execution option can afford.

• ceedling test:*:

  Execute the named test file or the named source file that has an accompanying test. No path. Examples: ceedling test:foo.c or ceedling test:test_foo.c

• ceedling test:pattern[*]:

  Execute any tests whose name and/or path match the regular expression pattern (case sensitive). Example: ceedling "test:pattern[(I|i)nit]" will execute all tests named for initialization testing. Note: quotes may be necessary around the ceedling parameter to distinguish regex characters from command line operators.

• ceedling test:path[*]:

  Execute any tests whose path contains the given string (case sensitive). Example: ceedling test:path[foo/bar] will execute all tests whose path contains foo/bar. Note: both directory separator characters / and \ are valid.

• ceedling test:* --test_case=<test_case_name> Execute test case which match test_case_name. Option available only after setting up cmdline_args to true under test_runner in project.yml:

  :test_runner:
    :cmdline_args: true
  

  For instance, if you have file test_gpio.c with defined 3 tests:

  • test_gpio_start
  • test_gpio_configure_proper
  • test_gpio_configure_fail_pin_not_allowed

  and you want to run only configure tests, you can call:

  ceedling test:gpio --test_case=configure

  ---

  Limitation

  The Unity implementation use test case name as substring which will be search in your test case names. If you pass only gpio and your file under test contains gpio in the name, it will run all tests from it. This is connected with the logic, how Unity generates test_<file_name_runner.c> files. In such case, it is suggested to use full name of test case.

  ---

• ceedling test:* --exclude_test_case=<test_case_name> Execute test case which does not match test_case_name. Option available only after setting up cmdline_args to true under test_runner in project.yml:

  :test_runner:
    :cmdline_args: true
  

  For instance, if you have file test_gpio.c with defined 3 tests:

  • test_gpio_start
  • test_gpio_configure_proper
  • test_gpio_configure_fail_pin_not_allowed

  and you want to run only start tests, you can call:

  ceedling test:gpio --exclude_test_case=configure

  ---

  Limitation

  The Unity implementation use test case name as substring which will be search in your test case names. If you pass only gpio and your file under test contains gpio in the name, it will run all tests from it. This is connected with the logic, how Unity generates test_<file_name_runner.c> files. In such case, it is suggested to use full name of test case.

  ---

• ceedling release:

  Build all source into a release artifact (if the release build option is configured).

• ceedling release:compile:*:

  Sometimes you just need to compile a single file dagnabit. Example: ceedling release:compile:foo.c

• ceedling release:assemble:*:

  Sometimes you just need to assemble a single file doggonit. Example: ceedling release:assemble:foo.s

• ceedling module:create[Filename]:

• ceedling module:create[<Path:>Filename]:

  It's often helpful to create a file automatically. What's better than that? Creating a source file, a header file, and a corresponding test file all in one step!

  There are also patterns which can be specified to automatically generate a bunch of files. Try ceedling module:create[Poodles,mch] for example!

  The module generator has several options you can configure. F.e. Generating the source/header/test file in a subdirectory (by adding when calling module:create). For more info, refer to the Module Generator section.

• ceedling module:stub[Filename]:

• ceedling module:stub[<Path:>Filename]:

  So what happens if you've created your API in your header (maybe even using TDD to do so?) and now you need to start to implement the corresponding C module? Why not get a head start by using ceedilng module:stub[headername] to automatically create a function skeleton for every function declared in that header? Better yet, you can call this again whenever you add new functions to that header to add just the new functions, leaving the old ones alone!

• ceedling logging <tasks...>:

  Enable logging to /logs. Must come before test and release tasks to log their steps and output. Log names are a concatenation of project, user, and option files loaded. User and option files are documented in the advanced topics section of this document.

• ceedling verbosity[x] <tasks...>:

  Change the default verbosity level. [x] ranges from 0 (quiet) to 4 (obnoxious). Level [3] is the default. The verbosity task must precede all tasks in the command line list for which output is desired to be seen. Verbosity settings are generally most meaningful in conjunction with test and release tasks.

• ceedling summary:

  If plugins are enabled, this task will execute the summary method of any plugins supporting it. This task is intended to provide a quick roundup of build artifact metrics without re-running any part of the build.

• ceedling clean:

  Deletes all toolchain binary artifacts (object files, executables), test results, and any temporary files. Clean produces no output at the command line unless verbosity has been set to an appreciable level.

• ceedling clobber:

  Extends clean task's behavior to also remove generated files: test runners, mocks, preprocessor output. Clobber produces no output at the command line unless verbosity has been set to an appreciable level.

• ceedling options:export:

  This allows you to export a snapshot of your current tool configuration as a yaml file. You can specify the name of the file in brackets [blah.yml] or let it default to tools.yml. In either case, the produced file can be used as the tool configuration for you project if desired, and modified as you wish.

To better understand Rake conventions, Rake execution, and Rakefiles, consult the Rake tutorial, examples, and user guide.

At present, none of Ceedling's commands provide persistence. That is, they must each be specified at the command line each time they are needed. For instance, Ceedling's verbosity command only affects output at the time it's run.

Individual test and release file tasks are not listed in -T output. Because so many files may be present it's unwieldy to list them all.

Multiple rake tasks can be executed at the command line (order is executed as provided). For example, ceedling clobber test:all release will removed all generated files; build and run all tests; and then build all source - in that order. If any Rake task fails along the way, execution halts before the next task.

The clobber task removes certain build directories in the course of deleting generated files. In general, it's best not to add to source control any Ceedling generated directories below the root of your top-level build directory. That is, leave anything Ceedling & its accompanying tools generate out of source control (but go ahead and add the top-level build directory that holds all that stuff). Also, since Ceedling is pretty smart about what it rebuilds and regenerates, you needn't clobber often.

Important Conventions

Directory Structure, Filenames & Extensions

Much of Ceedling's functionality is driven by collecting files matching certain patterns inside the paths it's configured to search. See the documentation for the [:extension] section of your configuration file (found later in this document) to configure the file extensions Ceedling uses to match and collect files. Test file naming is covered later in this section.

Test files and source files must be segregated by directories. Any directory structure will do. Tests can be held in subdirectories within source directories, or tests and source directories can be wholly separated at the top of your project's directory tree.

Search Path Order

When Ceedling searches for files (e.g. looking for header files to mock) or when it provides search paths to any of the default gcc toolchain executables, it organizes / prioritizes its search paths. The order is always: test paths, support paths, source paths, and then include paths. This can be useful, for instance, in certain testing scenarios where we desire Ceedling or a compiler to find a stand-in header file in our support directory before the actual source header file of the same name.

This convention only holds when Ceedling is using its default tool configurations and / or when tests are involved. If you define your own tools in the configuration file (see the [:tools] section documented later in this here document), you have complete control over what directories are searched and in what order. Further, test and support directories are only searched when appropriate. That is, when running a release build, test and support directories are not used at all.

Source Files & Binary Release Artifacts

Your binary release artifact results from the compilation and linking of all source files Ceedling finds in the specified source directories. At present only source files with a single (configurable) extension are recognized. That is, *.c and *.cc files will not both be recognized - only one or the other. See the configuration options and defaults in the documentation for the [:extension] sections of your configuration file (found later in this document).

Test Files & Executable Test Fixtures

Ceedling builds each individual test file with its accompanying source file(s) into a single, monolithic test fixture executable. Test files are recognized by a naming convention: a (configurable) prefix such as "test_" in the file name with the same file extension as used by your C source files. See the configuration options and defaults in the documentation for the [:project] and [:extension] sections of your configuration file (found later in this document). Depending on your configuration options, Ceedling can recognize a variety of test file naming patterns in your test search paths. For example: test_some_super_functionality.c, TestYourSourceFile.cc, or testing_MyAwesomeCode.C could each be valid test file names. Note, however, that Ceedling can recognize only one test file naming convention per project.

Ceedling knows what files to compile and link into each individual test executable by way of the #include list contained in each test file. Any C source files in the configured search directories that correspond to the header files included in a test file will be compiled and linked into the resulting test fixture executable. From this same #include list, Ceedling knows which files to mock and compile and link into the test executable (if you use mocks in your tests). That was a lot of clauses and information in a very few sentences; the example that follows in a bit will make it clearer.

By naming your test functions according to convention, Ceedling will extract and collect into a runner C file calls to all your test case functions. This runner file handles all the execution minutiae so that your test file can be quite simple and so that you never forget to wire up a test function to be executed. In this generated runner lives the main() entry point for the resulting test executable. There are no configuration options for the naming convention of your test case functions. A test case function signature must have these three elements: void return, void parameter list, and the function name prepended with lowercase "test". In other words, a test function signature should look like this: void test``[any name you like]``(void).

A commented sample test file follows on the next page. Also, see the sample project contained in the Ceedling documentation bundle.

// test_foo.c -----------------------------------------------
#include "unity.h"     // compile/link in Unity test framework
#include "types.h"     // header file with no *.c file -- no compilation/linking
#include "foo.h"       // source file foo.c under test
#include "mock_bar.h"  // bar.h will be found and mocked as mock_bar.c + compiled/linked in;
                       // foo.c includes bar.h and uses functions declared in it
#include "mock_baz.h"  // baz.h will be found and mocked as mock_baz.c + compiled/linked in
                       // foo.c includes baz.h and uses functions declared in it


void setUp(void) {}    // every test file requires this function;
                       // setUp() is called by the generated runner before each test case function

void tearDown(void) {} // every test file requires this function;
                       // tearDown() is called by the generated runner after each test case function

// a test case function
void test_Foo_Function1_should_Call_Bar_AndGrill(void)
{
    Bar_AndGrill_Expect();                    // setup function from mock_bar.c that instructs our
                                              // framework to expect Bar_AndGrill() to be called once
    TEST_ASSERT_EQUAL(0xFF, Foo_Function1()); // assertion provided by Unity
                                              // Foo_Function1() calls Bar_AndGrill() & returns a byte
}

// another test case function
void test_Foo_Function2_should_Call_Baz_Tec(void)
{
    Baz_Tec_ExpectAnd_Return(1);       // setup function provided by mock_baz.c that instructs our
                                       // framework to expect Baz_Tec() to be called once and return 1
    TEST_ASSERT_TRUE(Foo_Function2()); // assertion provided by Unity
}

// end of test_foo.c ----------------------------------------

From the test file specified above Ceedling will generate test_foo_runner.c; this runner file will contain main() and call both of the example test case functions.

The final test executable will be test_foo.exe (for Windows machines or test_foo.out for linux systems - depending on default or configured file extensions). Based on the #include list above, the test executable will be the output of the linker having processed unity.o, foo.o, mock_bar.o, mock_baz.o, test_foo.o, and test_foo_runner.o. Ceedling finds the files, generates mocks, generates a runner, compiles all the files, and links everything into the test executable. Ceedling will then run the test executable and collect test results from it to be reported to the developer at the command line.

For more on the assertions and mocks shown, consult the documentation for Unity and CMock.

The Magic of Dependency Tracking

Ceedling is pretty smart in using Rake to build up your project's dependencies. This means that Ceedling automagically rebuilds all the appropriate files in your project when necessary: when your configuration changes, Ceedling or any of the other tools are updated, or your source or test files change. For instance, if you modify a header file that is mocked, Ceedling will ensure that the mock is regenerated and all tests that use that mock are rebuilt and re-run when you initiate a relevant testing task. When you see things rebuilding, it's for a good reason. Ceedling attempts to regenerate and rebuild only what's needed for a given execution of a task. In the case of large projects, assembling dependencies and acting upon them can cause some delay in executing tasks.

With one exception, the trigger to rebuild or regenerate a file is always a disparity in timestamps between a target file and its source - if an input file is newer than its target dependency, the target is rebuilt or regenerated. For example, if the C source file from which an object file is compiled is newer than that object file on disk, recompilation will occur (of course, if no object file exists on disk, compilation will always occur). The one exception to this dependency behavior is specific to your input configuration. Only if your logical configuration changes will a system-wide rebuild occur. Reorganizing your input configuration or otherwise updating its file timestamp without modifying the values within the file will not trigger a rebuild. This behavior handles the various ways in which your input configuration can change (discussed later in this document) without having changed your actual project YAML file.

Ceedling needs a bit of help to accomplish its magic with deep dependencies. Shallow dependencies are straightforward: a mock is dependent on the header file from which it's generated, a test file is dependent upon the source files it includes (see the preceding conventions section), etc. Ceedling handles these "out of the box." Deep dependencies are specifically a C-related phenomenon and occur as a consequence of include statements within C source files. Say a source file includes a header file and that header file in turn includes another header file which includes still another header file. A change to the deepest header file should trigger a recompilation of the source file, a relinking of all the object files comprising a test fixture, and a new execution of that test fixture.

Ceedling can handle deep dependencies but only with the help of a C preprocessor. Ceedling is quite capable, but a full C preprocessor it ain't. Your project can be configured to use a C preprocessor or not. Simple projects or large projects constructed so as to be quite flat in their include structure generally don't need deep dependency preprocessing - and can enjoy the benefits of faster execution. Legacy code, on the other hand, will almost always want to be tested with deep preprocessing enabled. Set up of the C preprocessor is covered in the documentation for the [:project] and [:tools] section of the configuration file (later in this document). Ceedling contains all the configuration necessary to use the gcc preprocessor by default. That is, as long as gcc is in your system search path, deep preprocessing of deep dependencies is available to you by simply enabling it in your project configuration file.

Ceedling's Build Output

Ceedling requires a top-level build directory for all the stuff that it, the accompanying test tools, and your toolchain generate. That build directory's location is configured in the [:project] section of your configuration file (discussed later). There can be a ton of generated files. By and large, you can live a full and meaningful life knowing absolutely nothing at all about the files and directories generated below the root build directory.

As noted already, it's good practice to add your top-level build directory to source control but nothing generated beneath it. You'll spare yourself headache if you let Ceedling delete and regenerate files and directories in a non-versioned corner of your project's filesystem beneath the top-level build directory.

The artifacts directory is the one and only directory you may want to know about beneath the top-level build directory. The subdirectories beneath artifacts will hold your binary release target output (if your project is configured for release builds) and will serve as the conventional location for plugin output. This directory structure was chosen specifically because it tends to work nicely with Continuous Integration setups that recognize and list build artifacts for retrieval / download.

The Almighty Project Configuration File (in Glorious YAML)

Please consult YAML documentation for the finer points of format and to understand details of our YAML-based configuration file. We recommend Wikipedia's entry on YAML for this. A few highlights from that reference page:

• YAML streams are encoded using the set of printable Unicode characters, either in UTF-8 or UTF-16

• Whitespace indentation is used to denote structure; however tab characters are never allowed as indentation

• Comments begin with the number sign ( # ), can start anywhere on a line, and continue until the end of the line unless enclosed by quotes

• List members are denoted by a leading hyphen ( - ) with one member per line, or enclosed in square brackets ( [ ] ) and separated by comma space ( , )

• Hashes are represented using the colon space ( : ) in the form key: value, either one per line or enclosed in curly braces ( { } ) and separated by comma space ( , )

• Strings (scalars) are ordinarily unquoted, but may be enclosed in double-quotes ( " ), or single-quotes ( ' )

• YAML requires that colons and commas used as list separators be followed by a space so that scalar values containing embedded punctuation can generally be represented without needing to be enclosed in quotes

• Repeated nodes are initially denoted by an ampersand ( & ) and thereafter referenced with an asterisk ( * )

Notes on what follows:

• Each of the following sections represent top-level entries in the YAML configuration file.

• Unless explicitly specified in the configuration file, default values are used by Ceedling.

• These three settings, at minimum, must be specified:

  • [:project][:build_root]
  • [:paths][:source]
  • [:paths][:test]

• As much as is possible, Ceedling validates your settings in properly formed YAML.

• Improperly formed YAML will cause a Ruby error when the YAML is parsed. This is usually accompanied by a complaint with line and column number pointing into the project file.

• Certain advanced features rely on gcc and cpp as preprocessing tools. In most linux systems, these tools are already available. For Windows environments, we recommend the mingw project (Minimalist GNU for Windows).

• Ceedling is primarily meant as a build tool to support automated unit testing. All the heavy lifting is involved there. Creating a simple binary release build artifact is quite trivial in comparison. Consequently, most default options and the construction of Ceedling itself is skewed towards supporting testing though Ceedling can, of course, build your binary release artifact as well. Note that complex binary release artifacts (e.g. application + bootloader or multiple libraries) are beyond Ceedling's release build ability.

Conventions / features of Ceedling-specific YAML:

• Any second tier setting keys anywhere in YAML whose names end in _path or _paths are automagically processed like all Ceedling-specific paths in the YAML to have consistent directory separators (i.e. "/") and to take advantage of inline Ruby string expansion (see [:environment] setting below for further explanation of string expansion).

Let's Be Careful Out There: Ceedling performs validation on the values you set in your configuration file (this assumes your YAML is correct and will not fail format parsing, of course). That said, validation is limited to only those settings Ceedling uses and those that can be reasonably validated. Ceedling does not limit what can exist within your configuration file. In this way, you can take full advantage of YAML as well as add sections and values for use in your own custom plugins (documented later). The consequence of this is simple but important. A misspelled configuration section name or value name is unlikely to cause Ceedling any trouble. Ceedling will happily process that section or value and simply use the properly spelled default maintained internally - thus leading to unexpected behavior without warning.

project: global project settings

• build_root:

  Top level directory into which generated path structure and files are placed. Note: this is one of the handful of configuration values that must be set. The specified path can be absolute or relative to your working directory.

  Default: (none)

• use_exceptions:

  Configures the build environment to make use of CException. Note that if you do not use exceptions, there's no harm in leaving this as its default value.

  Default: TRUE

• use_mocks:

  Configures the build environment to make use of CMock. Note that if you do not use mocks, there's no harm in leaving this setting as its default value.

  Default: TRUE

• use_test_preprocessor:

  This option allows Ceedling to work with test files that contain conditional compilation statements (e.g. #ifdef) and header files you wish to mock that contain conditional preprocessor statements and/or macros.

  Ceedling and CMock are advanced tools with sophisticated parsers. However, they do not include entire C language preprocessors. Consequently, with this option enabled, Ceedling will use gcc's preprocessing mode and the cpp preprocessor tool to strip down / expand test files and headers to their applicable content which can then be processed by Ceedling and CMock.

  With this option enabled, the gcc & cpp tools must exist in an accessible system search path and test runner files are always regenerated.

  Default: FALSE

• use_preprocessor_directives:

  After standard preprocessing when use_test_preprocessor is used macros are fully expanded to C code. Some features, for example TEST_CASE() or TEST_RANGE() from Unity require not-fully preprocessed file to be detected by Ceedling. To do this gcc directives-only option is used to expand only conditional compilation statements, handle directives, but do not expand macros preprocessor and leave the other content of file untouched.

  With this option enabled, use_test_preprocessor must be also enabled and gcc must exist in an accessible system search path. For other compilers behavior can be changed by test_file_preprocessor_directives compiler tool.

  Default: FALSE

• use_deep_dependencies:

  The base rules and tasks that Ceedling creates using Rake capture most of the dependencies within a standard project (e.g. when the source file accompanying a test file changes, the corresponding test fixture executable will be rebuilt when tests are re-run). However, deep dependencies cannot be captured this way. If a typedef or macro changes in a header file three levels of #include statements deep, this option allows the appropriate incremental build actions to occur for both test execution and release builds.

  This is accomplished by using the dependencies discovery mode of gcc. With this option enabled, gcc must exist in an accessible system search path.

  Default: FALSE

• generate_deep_dependencies:

  When use_deep_dependencies is set to TRUE, Ceedling will run a separate build step to generate the deep dependencies. If you are using gcc as your primary compiler, or another compiler that can generate makefile rules as a side effect of compilation, then you can set this to FALSE to avoid the extra build step but still use the deep dependencies data when deciding which source files to rebuild.

  Default: TRUE

• test_file_prefix:

  Ceedling collects test files by convention from within the test file search paths. The convention includes a unique name prefix and a file extension matching that of source files.

  Why not simply recognize all files in test directories as test files? By using the given convention, we have greater flexibility in what we do with C files in the test directories.

  Default: "test_"

• options_paths:

  Just as you may have various build configurations for your source codebase, you may need variations of your project configuration.

  By specifying options paths, Ceedling will search for other project YAML files, make command line tasks available (ceedling options:variation for a variation.yml file), and merge the project configuration of these option files in with the main project file at runtime. See advanced topics.

  Note these Rake tasks at the command line - like verbosity or logging control - must come before the test or release task they are meant to modify.

  Default: [] (empty)

• release_build:

  When enabled, a release Rake task is exposed. This configuration option requires a corresponding release compiler and linker to be defined (gcc is used as the default).

  More release configuration options are available in the release_build section.

  Default: FALSE

Example [:project] YAML blurb

:project:
  :build_root: project_awesome/build
  :use_exceptions: FALSE
  :use_test_preprocessor: TRUE
  :use_deep_dependencies: TRUE
  :options_paths:
    - project/options
    - external/shared/options
  :release_build: TRUE

Ceedling is primarily concerned with facilitating the somewhat complicated mechanics of automating unit tests. The same mechanisms are easily capable of building a final release binary artifact (i.e. non test code; the thing that is your final working software that you execute on target hardware).

• use_backtrace_gdb_reporter: Set this value to true if you project use gcc compiler and you want to collect backtrace from test runners which fail with Segmentation fault error. The .fail files will contain testsuite with information, which test failed. Backtrace is fully integrated with junit_tests_report plugin.

  Default: FALSE

• output:

  The name of your release build binary artifact to be found in /artifacts/release. Ceedling sets the default artifact file extension to that as is explicitly specified in the [:extension] section or as is system specific otherwise.

  Default: project.exe or project.out

• use_assembly:

  If assembly code is present in the source tree, this option causes Ceedling to create appropriate build directories and use an assembler tool (default is the GNU tool as - override available in the [:tools] section.

  Default: FALSE

• artifacts:

  By default, Ceedling copies to the /artifacts/release directory the output of the release linker and (optionally) a map file. Many toolchains produce other important output files as well. Adding a file path to this list will cause Ceedling to copy that file to the artifacts directory. The artifacts directory is helpful for organizing important build output files and provides a central place for tools such as Continuous Integration servers to point to build output. Selectively copying files prevents incidental build cruft from needlessly appearing in the artifacts directory. Note that inline Ruby string replacement is available in the artifacts paths (see discussion in the [:environment] section).

  Default: [] (empty)

Example [:release_build] YAML blurb

:release_build:
  :output: top_secret.bin
  :use_assembly: TRUE
  :artifacts:
    - build/release/out/c/top_secret.s19

paths: options controlling search paths for source and header (and assembly) files

• test:

  All C files containing unit test code. Note: this is one of the handful of configuration values that must be set.

  Default: [] (empty)

• source:

  All C files containing release code (code to be tested). Note: this is one of the handful of configuration values that must be set.

  Default: [] (empty)

• support:

  Any C files you might need to aid your unit testing. For example, on occasion, you may need to create a header file containing a subset of function signatures matching those elsewhere in your code (e.g. a subset of your OS functions, a portion of a library API, etc.). Why? To provide finer grained control over mock function substitution or limiting the size of the generated mocks.

  Default: [] (empty)

• include:

  Any header files not already in the source search path. Note there's no practical distinction between this search path and the source search path; it's merely to provide options or to support any peculiar source tree organization.

  Default: [] (empty)

• test_toolchain_include:

  System header files needed by the test toolchain - should your compiler be unable to find them, finds the wrong system include search path, or you need a creative solution to a tricky technical problem. Note that if you configure your own toolchain in the [:tools] section, this search path is largely meaningless to you. However, this is a convenient way to control the system include path should you rely on the default gcc tools.

  Default: [] (empty)

• release_toolchain_include:

  Same as preceding albeit related to the release toolchain.

  Default: [] (empty)

• <custom>

  Any paths you specify for custom list. List is available to tool configurations and/or plugins. Note a distinction. The preceding names are recognized internally to Ceedling and the path lists are used to build collections of files contained in those paths. A custom list is just that - a custom list of paths.

Notes on path grammar within the [:paths] section:

• Order of search paths listed in [:paths] is preserved when used by an entry in the [:tools] section

• Wherever multiple path lists are combined for use Ceedling prioritizes path groups as follows: test paths, support paths, source paths, include paths.

  This can be useful, for instance, in certain testing scenarios where we desire Ceedling or the compiler to find a stand-in header file before the actual source header file of the same name.

• Paths:

  1. can be absolute or relative

  2. can be singly explicit - a single fully specified path

  3. can include a glob operator (more on this below)

  4. can use inline Ruby string replacement (see [:environment] section for more)

  5. default as an addition to a specific search list (more on this in the examples)

  6. can act to subtract from a glob included in the path list (more on this in the examples)

Globs as used by Ceedling are wildcards for specifying directories without the need to list each and every required search path. Ceedling globs operate just as Ruby globs except that they are limited to matching directories and not files. Glob operators include the following * ** ? [-] {,} (note: this list is space separated and not comma separated as commas are used within the bracket operators).

• *:

  All subdirectories of depth 1 below the parent path and including the parent path

• **:

  All subdirectories recursively discovered below the parent path and including the parent path

• ?:

  Single alphanumeric character wildcard

• [x-y]:

  Single alphanumeric character as found in the specified range

• {x,y}:

  Single alphanumeric character from the specified list

Example [:paths] YAML blurbs

:paths:
  :source:              #together the following comprise all source search paths
    - project/source/*  #expansion yields all subdirectories of depth 1 plus parent directory
    - project/lib       #single path
  :test:                #all test search paths
    - project/**/test?  #expansion yields any subdirectory found anywhere in the project that
                        #begins with "test" and contains 5 characters

:paths:
  :source:                           #all source search paths
    - +:project/source/**            #all subdirectories recursively discovered plus parent directory
    - -:project/source/os/generated  #subtract os/generated directory from expansion of above glob
                                     #note that '+:' notation is merely aesthetic; default is to add

  :test:                             #all test search paths
    - project/test/bootloader        #explicit, single search paths (searched in the order specified)
    - project/test/application
    - project/test/utilities

  :custom:                           #custom path list
    - "#{PROJECT_ROOT}/other"        #inline Ruby string expansion

Globs and inline Ruby string expansion can require trial and error to arrive at your intended results. Use the ceedling paths:* command line options (documented in preceding section) to verify your settings.

Ceedling relies on file collections automagically assembled from paths, globs, and file extensions. File collections greatly simplify project set up. However, sometimes you need to remove from or add individual files to those collections.

• test:

  Modify the collection of unit test C files.

  Default: [] (empty)

• source:

  Modify the collection of all source files used in unit test builds and release builds.

  Default: [] (empty)

• assembly:

  Modify the (optional) collection of assembly files used in release builds.

  Default: [] (empty)

• include:

  Modify the collection of all source header files used in unit test builds (e.g. for mocking) and release builds.

  Default: [] (empty)

• support:

  Modify the collection of supporting C files available to unit tests builds.

  Default: [] (empty)

• libraries:

  Add a collection of library paths to be included when linking.

  Default: [] (empty)

Note: All path grammar documented in [:paths] section applies to [:files] path entries - albeit at the file path level and not the directory level.

Example [:files] YAML blurb

:files:
  :source:
    - callbacks/comm.c        # entry defaults to file addition
    - +:callbacks/comm*.c     # add all comm files matching glob pattern
    - -:source/board/atm134.c # not our board
  :test:
    - -:test/io/test_output_manager.c # remove unit tests from test build

environment: inserts environment variables into the shell instance executing configured tools

Ceedling creates environment variables from any key / value pairs in the environment section. Keys become an environment variable name in uppercase. The values are strings assigned to those environment variables. These value strings are either simple string values in YAML or the concatenation of a YAML array.

Ceedling is able to execute inline Ruby string substitution code to set environment variables. This evaluation occurs when the project file is first processed for any environment pair's value string including the Ruby string substitution pattern #{…}. Note that environment value strings that begin with this pattern should always be enclosed in quotes. YAML defaults to processing unquoted text as a string; quoting text is optional. If an environment pair's value string begins with the Ruby string substitution pattern, YAML will interpret the string as a Ruby comment (because of the #). Enclosing each environment value string in quotes is a safe practice.

[:environment] entries are processed in the configured order (later entries can reference earlier entries).

Special case: PATH handling

In the specific case of specifying an environment key named path, an array of string values will be concatenated with the appropriate platform-specific path separation character (e.g. ':' on linux, ';' on Windows). All other instances of environment keys assigned YAML arrays use simple concatenation.

Example [:environment] YAML blurb

:environment:
  - :license_server: gizmo.intranet        #LICENSE_SERVER set with value "gizmo.intranet"
  - :license: "#{`license.exe`}"           #LICENSE set to string generated from shelling out to
                                           #execute license.exe; note use of enclosing quotes

  - :path:                                 #concatenated with path separator (see special case above)
     - Tools/gizmo/bin                     #prepend existing PATH with gizmo path
     - "#{ENV['PATH']}"                    #pattern #{…} triggers ruby evaluation string substitution
                                           #note: value string must be quoted because of '#'

  - :logfile: system/logs/thingamabob.log  #LOGFILE set with path for a log file

extension: configure file name extensions used to collect lists of files searched in [:paths]

• header:

  C header files

  Default: .h

• source:

  C code files (whether source or test files)

  Default: .c

• assembly:

  Assembly files (contents wholly assembly instructions)

  Default: .s

• object:

  Resulting binary output of C code compiler (and assembler)

  Default: .o

• executable:

  Binary executable to be loaded and executed upon target hardware

  Default: .exe or .out (Win or linux)

• testpass:

  Test results file (not likely to ever need a new value)

  Default: .pass

• testfail:

  Test results file (not likely to ever need a new value)

  Default: .fail

• dependencies:

  File containing make-style dependency rules created by gcc preprocessor

  Default: .d

Example [:extension] YAML blurb

:extension:
  :source: .cc
  :executable: .bin

defines: command line defines used in test and release compilation by configured tools

• test:

  Defines needed for testing. Useful for:

  1. test files containing conditional compilation statements (i.e. tests active in only certain contexts)

  2. testing legacy source wherein the isolation of source under test afforded by Ceedling and its complementary tools leaves certain symbols unset when source files are compiled in isolation

  Default: [] (empty)

• test_preprocess:

  If [:project][:use_test_preprocessor] or [:project][:use_deep_dependencies] is set and code is structured in a certain way, the gcc preprocessor may need symbol definitions to properly preprocess files to extract function signatures for mocking and extract deep dependencies for incremental builds.

  Default: [] (empty)

• <test_name>:

  Replace standard test definitions for specified <test_name>definitions. For example:

  :defines:
    :test:
      - FOO_STANDARD_CONFIG
    :test_foo_config:
      - FOO_SPECIFIC_CONFIG

ceedling test:foo_config will now have FOO_SPECIFIC_CONFIG defined instead of FOO_STANDARD_CONFIG. None of the other tests will have FOO_SPECIFIC_SPECIFIC.

Default: [] (empty)

• release:

  Defines needed for the release build binary artifact.

  Default: [] (empty)

• release_preprocess:

  If [:project][:use_deep_dependencies] is set and code is structured in a certain way, the gcc preprocessor may need symbol definitions to properly preprocess files for incremental release builds due to deep dependencies.

  Default: [] (empty)

• use_test_definition:

  When this option is used the -D<test_name> flag is added to the build option.

  Default: FALSE

Example [:defines] YAML blurb

:defines:
  :test:
    - UNIT_TESTING  #for select cases in source to allow testing with a changed behavior or interface
    - OFF=0
    - ON=1
    - FEATURE_X=ON
  :source:
    - FEATURE_X=ON

libraries: command line defines used in test and release compilation by configured tools

Ceedling allows you to pull in specific libraries for the purpose of release and test builds. It has a few levels of support for this. Start by adding a :libraries main section in your configuration. In this section, you can optionally have the following subsections:

• test:

  Library files that should be injected into your tests when linking occurs. These can be specified as either relative or absolute paths. These files MUST exist when the test attempts to build.

• release:

  Library files that should be injected into your release when linking occurs. These can be specified as either relative or absolute paths. These files MUST exist when the release attempts to build UNLESS you are using the subprojects plugin. In that case, it will attempt to build that library for you as a dynamic dependency.

• system:

  These libraries are assumed to be in the tool path somewhere and shouldn't need to be specified. The libraries added here will be injected into releases and tests. For example if you specify -lm you can include the math library. The -l portion is only necessary if the :flag prefix below doesn't specify it already for you other libraries.

• flag:

  This is the method of adding an argument for each library. For example, gcc really likes it when you specify “-l${1}”

• path_flag:

  This is the method of adding an path argument for each library path. For example, gcc really likes it when you specify “-L "${1}"”

Notes:

• If you've specified your own link step, you are going to want to add ${4} to your argument list in the place where library files should be added to the command call. For gcc, this is often the very end. Other tools may vary.

flags: configure per-file compilation and linking flags

Ceedling tools (see later [:tools] section) are used to configure compilation and linking of test and source files. These tool configurations are a one-size-fits-all approach. Should individual files require special compilation or linking flags, the settings in the [:flags] section work in conjunction with tool definitions by way of argument substitution to achieve this.

• release:

  [:compile] or [:link] flags for release build

• test:

  [:compile] or [:link] flags for test build

Notes:

• Ceedling works with the [:release] and [:test] build contexts as-is; plugins can add additional contexts

• Only [:compile] and [:link] are recognized operations beneath a context

• File specifiers do not include a path or file extension

• File specifiers are case sensitive (must match original file name)

• File specifiers do support regular expressions if encased in quotes

• '*' is a special (optional) file specifier to provide flags to all files not otherwise specified

Example [:flags] YAML blurb

:flags:
  :release:
    :compile:
      :main:       # add '-Wall' to compilation of main.c
        - -Wall
      :fan:        # add '--O2' to compilation of fan.c
        - --O2
      :'test_.+':   # add '-pedantic' to all test-files
        - -pedantic
      :*:          # add '-foo' to compilation of all files not main.c or fan.c
        - -foo
  :test:
    :compile:
      :main:       # add '--O1' to compilation of main.c as part of test builds including main.c
        - --O1
    :link:
      :test_main:  # add '--bar --baz' to linking of test_main.exe
        - --bar
        - --baz

import: Load additional config files

In some cases it is nice to have config files (project.yml, options files) which can load other config files, for commonly re-used definitions (target processor, common code modules, etc).

These can be recursively nested, the included files can include other files.

To import config files, either provide an array of files to import, or use hashes to set imports. The former is useful if you do not anticipate needing to replace a given file for different configurations (project: or options:). If you need to replace/remove imports based on different configuration files, use the hashed version. The two methods cannot be mixed in the same .yml.

Example [:import] YAML blurb using array

:import:
  - path/to/config.yml
  - path/to/another/config.yml

Example [:import] YAML blurb using hashes

:import:
  :configA: path/to/config.yml
  :configB: path/to/another/config.yml

Ceedling sets values for a subset of CMock settings. All CMock options are available to be set, but only those options set by Ceedling in an automated fashion are documented below. See CMock documentation.

cmock: configure CMock's code generation options and set symbols used to modify CMock's compiled features Ceedling sets values for a subset of CMock settings. All CMock options are available to be set, but only those options set by Ceedling in an automated fashion are documented below. See CMock documentation.

• enforce_strict_ordering:

  Tests fail if expected call order is not same as source order

  Default: TRUE

• mock_path:

  Path for generated mocks

  Default: /tests/mocks

• defines:

  List of conditional compilation symbols used to configure CMock's compiled features. See CMock documentation to understand available options. No symbols must be set unless defaults are inappropriate for your specific environment. All symbols are used only by Ceedling to compile CMock C code; contents of [:defines] are ignored by CMock's Ruby code when instantiated.

  Default: [] (empty)

• verbosity:

  If not set, defaults to Ceedling's verbosity level

• plugins:

  If [:project][:use_exceptions] is enabled, the internal plugins list is pre-populated with 'cexception'.

  Whether or not you have included [:cmock][:plugins] in your configuration file, Ceedling automatically adds 'cexception' to the plugin list if exceptions are enabled. To add to the list Ceedling provides CMock, simply add [:cmock][:plugins] to your configuration and specify your desired additional plugins.

  Each of the plugins have their own additional documentation.

• includes:

  If [:cmock][:unity_helper] set, pre-populated with unity_helper file name (no path).

  The [:cmock][:includes] list works identically to the plugins list above with regard to adding additional files to be inserted within mocks as #include statements.

The last four settings above are directly tied to other Ceedling settings; hence, why they are listed and explained here. The first setting above, [:enforce_strict_ordering], defaults to FALSE within CMock. It is set to TRUE by default in Ceedling as our way of encouraging you to use strict ordering. It's a teeny bit more expensive in terms of code generated, test execution time, and complication in deciphering test failures. However, it's good practice. And, of course, you can always disable it by overriding the value in the Ceedling YAML configuration file.

cexception: configure symbols used to modify CException's compiled features

• defines:

  List of conditional compilation symbols used to configure CException's features in its source and header files. See CException documentation to understand available options. No symbols must be set unless the defaults are inappropriate for your specific environment.

  Default: [] (empty)

unity: configure symbols used to modify Unity's compiled features

• defines:

  List of conditional compilation symbols used to configure Unity's features in its source and header files. See Unity documentation to understand available options. No symbols must be set unless the defaults are inappropriate for your specific environment. Most Unity defines can be easily configured through the YAML file.

  Default: [] (empty)

Example [:unity] YAML blurbs

:unity: #itty bitty processor & toolchain with limited test execution options
  :defines:
    - UNITY_INT_WIDTH=16           #16 bit processor without support for 32 bit instructions
    - UNITY_EXCLUDE_FLOAT          #no floating point unit

:unity: #great big gorilla processor that grunts and scratches
  :defines:
    - UNITY_SUPPORT_64                    #big memory, big counters, big registers
    - UNITY_LINE_TYPE=\"unsigned int\"    #apparently we're using really long test files,
    - UNITY_COUNTER_TYPE=\"unsigned int\" #and we've got a ton of test cases in those test files
    - UNITY_FLOAT_TYPE=\"double\"         #you betcha

Notes on Unity configuration:

• Verification - Ceedling does no verification of your configuration values. In a properly configured setup, your Unity configuration values are processed, collected together with any test define symbols you specify elsewhere, and then passed to your toolchain during test compilation. Unity's conditional compilation statements, your toolchain's preprocessor, and/or your toolchain's compiler will complain appropriately if your specified configuration values are incorrect, incomplete, or incompatible.

• Routing $stdout - Unity defaults to using putchar() in C's standard library to display test results. For more exotic environments than a desktop with a terminal (e.g. running tests directly on a non-PC target), you have options. For example, you could create a routine that transmits a character via RS232 or USB. Once you have that routine, you can replace putchar() calls in Unity by overriding the function-like macro UNITY_OUTPUT_CHAR. Consult your toolchain and shell documentation. Eventhough this can also be defined in the YAML file most shell environments do not handle parentheses as command line arguments very well. To still be able to add this functionality all necessary options can be defined in the unity_config.h. Unity needs to be told to look for the unity_config.h in the YAML file, though.

Example [:unity] YAML blurbs

:unity:
  :defines:
  	- UNITY_INCLUDE_CONFIG_H

Example unity_config.h

#ifndef UNITY_CONFIG_H
#define UNITY_CONFIG_H

#include "uart_output.h" //Helper library for your custom environment

#define UNITY_INT_WIDTH 16
#define UNITY_OUTPUT_START() uart_init(F_CPU, BAUD) //Helperfunction to init UART
#define UNITY_OUTPUT_CHAR(a) uart_putchar(a) //Helperfunction to forward char via UART
#define UNITY_OUTPUT_COMPLETE() uart_complete() //Helperfunction to inform that test has ended

#endif

tools: a means for representing command line tools for use under Ceedling's automation framework

Ceedling requires a variety of tools to work its magic. By default, the GNU toolchain (gcc, cpp, as) are configured and ready for use with no additions to the project configuration YAML file. However, as most work will require a project-specific toolchain, Ceedling provides a generic means for specifying / overriding tools.

• test_compiler:

  Compiler for test & source-under-test code

  • ${1}: input source
  • ${2}: output object
  • ${3}: optional output list
  • ${4}: optional output dependencies file

  Default: gcc

• test_linker:

  Linker to generate test fixture executables

  • ${1}: input objects
  • ${2}: output binary
  • ${3}: optional output map
  • ${4}: optional library list
  • ${5}: optional library path list

  Default: gcc

• test_fixture:

  Executable test fixture

  • ${1}: simulator as executable with${1} as input binary file argument or native test executable

  Default: ${1}

• test_includes_preprocessor:

  Extractor of #include statements

  • ${1}: input source file

  Default: cpp

• test_file_preprocessor:

  Preprocessor of test files (macros, conditional compilation statements)

  • ${1}: input source file
  • ${2}: preprocessed output source file

  Default: gcc

• test_file_preprocessor_directives:

  Preprocessor of test files to expand only conditional compilation statements, handle directives, but do not expand macros

  • ${1}: input source file
  • ${2}: not-fully preprocessed output source file

  Default: gcc

• test_dependencies_generator:

  Discovers deep dependencies of source & test (for incremental builds)

  • ${1}: input source file
  • ${2}: compiled object filepath
  • ${3}: output dependencies file

  Default: gcc

• release_compiler:

  Compiler for release source code

  • ${1}: input source
  • ${2}: output object
  • ${3}: optional output list
  • ${4}: optional output dependencies file

  Default: gcc

• release_assembler:

  Assembler for release assembly code

  • ${1}: input assembly source file
  • ${2}: output object file

  Default: as

• release_linker:

  Linker for release source code

  • ${1}: input objects
  • ${2}: output binary
  • ${3}: optional output map
  • ${4}: optional library list
  • ${5}: optional library path list

  Default: gcc

• release_dependencies_generator:

  Discovers deep dependencies of source files (for incremental builds)

  • ${1}: input source file
  • ${2}: compiled object filepath
  • ${3}: output dependencies file

  Default: gcc

A Ceedling tool has a handful of configurable elements:

1. [:executable] - Command line executable (required)

2. [:arguments] - List of command line arguments and substitutions (required)

3. [:name] - Simple name (e.g. "nickname") of tool beyond its executable name (if not explicitly set then Ceedling will form a name from the tool's YAML entry name)

4. [:stderr_redirect] - Control of capturing $stderr messages {:none, :auto, :win, :unix, :tcsh}. Defaults to :none if unspecified; create a custom entry by specifying a simple string instead of any of the available symbols.

5. [:background_exec] - Control execution as background process {:none, :auto, :win, :unix}. Defaults to :none if unspecified.

6. [:optional] - By default a tool is required for operation, which means tests will be aborted if the tool is not present. However, you can set this to TRUE if it's not needed for testing.

Tool Element Runtime Substitution

To accomplish useful work on multiple files, a configured tool will most often require that some number of its arguments or even the executable itself change for each run. Consequently, every tool's argument list and executable field possess two means for substitution at runtime. Ceedling provides two kinds of inline Ruby execution and a notation for populating elements with dynamically gathered values within the build environment.

Tool Element Runtime Substitution: Inline Ruby Execution

In-line Ruby execution works similarly to that demonstrated for the [:environment] section except that substitution occurs as the tool is executed and not at the time the configuration file is first scanned.

• #{...}:

  Ruby string substitution pattern wherein the containing string is expanded to include the string generated by Ruby code between the braces. Multiple instances of this expansion can occur within a single tool element entry string. Note that if this string substitution pattern occurs at the very beginning of a string in the YAML configuration the entire string should be enclosed in quotes (see the [:environment] section for further explanation on this point).

• {...} :

  If an entire tool element string is enclosed with braces, it signifies that Ceedling should execute the Ruby code contained within those braces. Say you have a collection of paths on disk and some of those paths include spaces. Further suppose that a single tool that must use those paths requires those spaces to be escaped, but all other uses of those paths requires the paths to remain unchanged. You could use this Ceedling feature to insert Ruby code that iterates those paths and escapes those spaces in the array as used by the tool of this example.

Tool Element Runtime Substitution: Notational Substitution

A Ceedling tool's other form of dynamic substitution relies on a '$' notation. These '$' operators can exist anywhere in a string and can be decorated in any way needed. To use a literal '$', escape it as '\$'.

• $:

  Simple substitution for value(s) globally available within the runtime (most often a string or an array).

• ${#}:

  When a Ceedling tool's command line is expanded from its configured representation and used within Ceedling Ruby code, certain calls to that tool will be made with a parameter list of substitution values. Each numbered substitution corresponds to a position in a parameter list. Ceedling Ruby code expects that configured compiler and linker tools will contain ${1} and ${2} replacement arguments. In the case of a compiler ${1} will be a C code file path, and ${2} will be the file path of the resulting object file. For a linker ${1} will be an array of object files to link, and ${2} will be the resulting binary executable. For an executable test fixture ${1} is either the binary executable itself (when using a local toolchain such as gcc) or a binary input file given to a simulator in its arguments.

Example [:tools] YAML blurbs

:tools:
  :test_compiler:
     :executable: compiler              #exists in system search path
     :name: 'acme test compiler'
     :arguments:
        - -I"$": COLLECTION_PATHS_TEST_TOOLCHAIN_INCLUDE               #expands to -I search paths
        - -I"$": COLLECTION_PATHS_TEST_SUPPORT_SOURCE_INCLUDE_VENDOR   #expands to -I search paths
        - -D$: COLLECTION_DEFINES_TEST_AND_VENDOR  #expands to all -D defined symbols
        - --network-license             #simple command line argument
        - -optimize-level 4             #simple command line argument
        - "#{`args.exe -m acme.prj`}"   #in-line ruby sub to shell out & build string of arguments
        - -c ${1}                       #source code input file (Ruby method call param list sub)
        - -o ${2}                       #object file output (Ruby method call param list sub)
  :test_linker:
     :executable: /programs/acme/bin/linker.exe    #absolute file path
     :name: 'acme test linker'
     :arguments:
        - ${1}               #list of object files to link (Ruby method call param list sub)
        - -l$-lib:           #inline yaml array substitution to link in foo-lib and bar-lib
           - foo
           - bar
        - -o ${2}            #executable file output (Ruby method call param list sub)
  :test_fixture:
     :executable: tools/bin/acme_simulator.exe  #relative file path to command line simulator
     :name: 'acme test fixture'
     :stderr_redirect: :win                     #inform Ceedling what model of $stderr capture to use
     :arguments:
        - -mem large   #simple command line argument
        - -f "${1}"    #binary executable input file to simulator (Ruby method call param list sub)

Resulting command line constructions from preceding example [:tools] YAML blurbs

> compiler -I"/usr/include” -I”project/tests”
  -I"project/tests/support” -I”project/source” -I”project/include”
  -DTEST -DLONG_NAMES -network-license -optimize-level 4 arg-foo
  arg-bar arg-baz -c project/source/source.c -o
  build/tests/out/source.o

[notes: (1.) "arg-foo arg-bar arg-baz" is a fabricated example string collected from $stdout as a result of shell execution of args.exe (2.) the -c and -o arguments are fabricated examples simulating a single compilation step for a test; ${1} & ${2} are single files]

> \programs\acme\bin\linker.exe thing.o unity.o
  test_thing_runner.o test_thing.o mock_foo.o mock_bar.o -lfoo-lib
  -lbar-lib -o build\tests\out\test_thing.exe

[note: in this scenario ${1} is an array of all the object files needed to link a test fixture executable]

> tools\bin\acme_simulator.exe -mem large -f "build\tests\out\test_thing.bin 2>&1”

[note: (1.) :executable could have simply been ${1} - if we were compiling and running native executables instead of cross compiling (2.) we're using $stderr redirection to allow us to capture simulator error messages to $stdout for display at the run's conclusion]

Notes:

• The upper case names are Ruby global constants that Ceedling builds

• "COLLECTION_" indicates that Ceedling did some work to assemble the list. For instance, expanding path globs, combining multiple path globs into a convenient summation, etc.

• At present, $stderr redirection is primarily used to capture errors from test fixtures so that they can be displayed at the conclusion of a test run. For instance, if a simulator detects a memory access violation or a divide by zero error, this notice might go unseen in all the output scrolling past in a terminal.

• The preprocessing tools can each be overridden with non-gcc equivalents. However, this is an advanced feature not yet documented and requires that the replacement toolchain conform to the same conventions used by gcc.

Ceedling Collection Used in Compilation:

• COLLECTION_PATHS_TEST:

  All test paths

• COLLECTION_PATHS_SOURCE:

  All source paths

• COLLECTION_PATHS_INCLUDE:

  All include paths

• COLLECTION_PATHS_SUPPORT:

  All test support paths

• COLLECTION_PATHS_SOURCE_AND_INCLUDE:

  All source and include paths

• COLLECTION_PATHS_SOURCE_INCLUDE_VENDOR:

  All source and include paths + applicable vendor paths (e.g. CException's source path if exceptions enabled)

• COLLECTION_PATHS_TEST_TOOLCHAIN_INCLUDE:

  All test toolchain include paths

• COLLECTION_PATHS_TEST_SUPPORT_SOURCE_INCLUDE:

  All test, source, and include paths

• COLLECTION_PATHS_TEST_SUPPORT_SOURCE_INCLUDE_VENDOR:

  All test, source, include, and applicable vendor paths (e.g. Unity's source path plus CMock and CException's source paths if mocks and exceptions are enabled)

• COLLECTION_PATHS_RELEASE_TOOLCHAIN_INCLUDE:

  All release toolchain include paths

• COLLECTION_DEFINES_TEST_AND_VENDOR:

  All symbols specified in [:defines][:test] + symbols defined for enabled vendor tools - e.g. [:unity][:defines], [:cmock][:defines], and [:cexception][:defines]

• COLLECTION_DEFINES_RELEASE_AND_VENDOR:

  All symbols specified in [:defines][:release] plus symbols defined by [:cexception][:defines] if exceptions are enabled

Notes:

• Other collections exist within Ceedling. However, they are only useful for advanced features not yet documented.

• Wherever multiple path lists are combined for use Ceedling prioritizes path groups as follows: test paths, support paths, source paths, include paths. This can be useful, for instance, in certain testing scenarios where we desire Ceedling or the compiler to find a stand-in header file before the actual source header file of the same name.

plugins: Ceedling extensions

• load_paths:

  Base paths to search for plugin subdirectories or extra ruby functionalit

  Default: [] (empty)

• enabled:

  List of plugins to be used - a plugin's name is identical to the subdirectory that contains it (and the name of certain files within that subdirectory)

  Default: [] (empty)

Plugins can provide a variety of added functionality to Ceedling. In general use, it's assumed that at least one reporting plugin will be used to format test results. However, if no reporting plugins are specified, Ceedling will print to $stdout the (quite readable) raw test results from all test fixtures executed.

Example [:plugins] YAML blurb

:plugins:
  :load_paths:
    - project/tools/ceedling/plugins  #home to your collection of plugin directories
    - project/support                 #maybe home to some ruby code your custom plugins share
  :enabled:
    - stdout_pretty_tests_report      #nice test results at your command line
    - our_custom_code_metrics_report  #maybe you needed line count and complexity metrics, so you
                                      #created a plugin to scan all your code and collect that info

• stdout_pretty_tests_report:

  Prints to $stdout a well-formatted list of ignored and failed tests, final test counts, and any extraneous output (e.g. printf statements or simulator memory errors) collected from executing the test fixtures. Meant to be used with runs at the command line.

• stdout_ide_tests_report:

  Prints to $stdout simple test results formatted such that an IDE executing test-related Rake tasks can recognize file paths and line numbers in test failures, etc. Thus, you can click a test result in your IDE's execution window and jump to the failure (or ignored test) in your test file (obviously meant to be used with an IDE like Eclipse, etc).

• xml_tests_report:

  Creates an XML file of test results in the xUnit format (handy for Continuous Integration build servers or as input to other reporting tools). Produces a file report.xml in /artifacts/tests.

• bullseye:

  Adds additional Rake tasks to execute tests with the commercial code coverage tool provided by Bullseye. See readme.txt inside the bullseye plugin directory for configuration and use instructions. Note: Bullseye only works with certain compilers and linkers (healthy list of supported toolchains though).

• gcov:

  Adds additional Rake tasks to execute tests with the GNU code coverage tool gcov. See readme.txt inside the gcov directory for configuration and use instructions. Only works with GNU compiler and linker.

• warnings_report:

  Scans compiler and linker $stdout / $stderr output for the word 'warning' (case insensitive). All code warnings (or tool warnings) are logged to a file warnings.log in the appropriate <build root>/artifacts directory (e.g. test/ for test tasks, release/ for a release build, or even bullseye/ for bullseye runs).

Module Generator

Ceedling includes a plugin called module_generator that will create a source, header and test file for you. There are several possibilities to configure this plugin through your project.yml to suit your project's needs.

Directory Structure

The default configuration for directory/project structure is:

:module_generator:
  :project_root: ./
  :source_root: src/
  :test_root: test/

You can change these variables in your project.yml file to comply with your project's directory structure.

If you call ceedling module:create, it will create three files:

1. A source file in the source_root
2. A header file in the source_root
3. A test file in the test_root

If you want your header file to be in another location, you can specify the ':inc_root:" in your project.yml file:

:module_generator:
  :inc_root: inc/

The module_generator will then create the header file in your defined ':inc_root:'. By default, ':inc_root:' is not defined so the module_generator will use the source_root.

Sometimes, your project can't be divided into a single src, inc, and test folder. You have several directories with sources/..., something like this for example: <project_root>

• myDriver
  • src
  • inc
  • test
• myOtherDriver
  • src
  • inc
  • test
• ...

Don't worry, you don't have to manually create the source/header/test files. The module_generator can accept a path to create a source_root/inc_root/test_root folder with your files: ceedling module:create[<module_root_path>:<module_name>]

F.e., applied to the above project structure: ceedling module:create[myOtherDriver:driver] This will make the module_generator run in the subdirectory 'myOtherDriver' and generate the module files for you in that directory. So, this command will generate the following files:

1. A source file 'driver.c' in <project_root>/myOtherDriver/<source_root>
2. A header file 'driver.h' in <project_root>/myOtherDriver/<source_root> (or <inc_root> if specified)
3. A test file 'test_driver.c' in <project_root>/myOtherDriver/<test_root>

Naming

By default, the module_generator will generate your files in lowercase. ceedling module:create[mydriver] and ceedling module:create[myDriver](note the uppercase) will generate the same files:

1. mydriver.c
2. mydriver.h
3. test_mydriver.c

You can configure the module_generator to use a differect naming mechanism through the project.yml:

:module_generator:
  :naming: "camel"

There are other possibilities as well (bumpy, camel, snake, caps). Refer to the unity module generator for more info (the unity module generator is used under the hood by module_generator).

Boilerplate header

There are two ways of adding a boilerplate header comment to your generated files:

• With a defined string in the project.yml file:

:module_generator:
  :boilerplates:
    :src: '/* This is Boilerplate code. */'

Using the command ceedling module:create[foo] it creates the source module as follows:

/* This is Boilerplate code. */
#include "foo.h"

It would be the same for :tst: and :inc: adding its respective options.

• Defining an external file with boileplate code:

:module_generator:
  :boilerplate_files:
    :src: '<template_folder>\src_boilerplate.txt'
    :inc: '<template_folder>\inc_boilerplate.txt'
    :tst: '<template_folder>\tst_boilerplate.txt'

For whatever file names in whichever folder you desire.

Advanced Topics (Coming)

Modifying Your Configuration without Modifying Your Project File: Option Files & User Files

Modifying your project file without modifying your project file

Debugging and/or printf()

When you gotta get your hands dirty...

Ceedling Plays Nice with Others - Using Ceedling for Tests Alongside Another Release Build Setup

You've got options.

Adding Handy Rake Tasks for Your Project (without Fancy Pants Custom Plugins)

Add a file rakefile.rb at the root of your project that loads Ceedling. This differs whether you are using the gem version or a local Ceedling version.

Gem Version:

require('ceedling')
Ceedling.load_project

Local Ceedling Version (assuming local ceedling is in vendor/ceedling):

PROJECT_CEEDLING_ROOT = "vendor/ceedling"
load "#{PROJECT_CEEDLING_ROOT}/lib/ceedling.rb"
Ceedling.load_project

Now you simply add your rake task to the file e.g.:

desc "Print hello world in sh" # Only tasks with description are listed by ceedling -T
task :hello_world do
  sh "echo Hello World!"
end

The task can now be called with: ceedling hello_world

Working with Non-Desktop Testing Environments

For those crazy platforms lacking command line simulators and for which cross-compiling on the desktop just ain't gonna get it done.

Creating Custom Plugins

Oh boy. This is going to take some explaining.