CIAT (pronounced “dog”) provides a system for writing high-level acceptance tests for compilers and interpreters. Each acceptance test is entered into a single file, and that file identifies the elements of a test.
Suppose you have a compiler written in Java that compiles a language named Hobbes. Your compiler targets the Parrot Virtual Machine. So you want to provide source code which is compiled with a Java program and that result is interpreted by Parrot.
Input files should be named with a .ciat extension and saved in a ciat folder.
A sample input file (simpleinteger5.ciat) for the scenario described above might look like this:
Compiles a simple integer. ==== source 5 ==== compilation .sub main print 5 print "\n" .end ==== execution 5
This file specifies four elements: description, source, compilation, and execution. The description is always the first element, always unlabeled, and used prominently in the HTML report. All of the other elements are dependent on the processors that you use.
In this example, we’re using a “Java compiler” (a compiler written in Java) and a “Parrot executor”. CIAT’s “Java compiler” runs your compiler over the source, and that output is compared to the compilation element. Then the “Parrot executor” is executed with the generated compilation, and that output is compared to the execution element.
If any processor fails, either due to an error while running or a failure during checking the output, the remaining processors are not executed.
Some processors will use optional elements in a test file. For example, the “Parrot executor” knows about command-line arguments:
Compiles a simple integer and ignores the command-line arguments. ==== source 5 ==== compilation .sub main print 5 print "\n" .end ==== command line 89 pqp ==== execution 5
When the “Parrot executor” is run on the compilation, it’ll also pass in 89 pqp as command-line arguments.
This sample Rakefile will pull everything together:
require 'ciat' require 'ciat/processors/java' require 'ciat/processors/parrot' def compiler classpath = Dir.glob('../lib/*.jar').join(':') + ":../bin" CIAT::Compilers::Java.new(classpath, 'org.norecess.hobbes.drivers.PIRCompiler') end def executor CIAT::Processors::Parrot.new end CIAT::RakeTask.new do |t| t.processors << compiler t.processors << executor end
This rakefile will find all of the .ciat files inside a ciat directory, each one representing a test. Each test will be executed, and the results are put into a folder named temp, including the HTML report report.html. All of these settings can be tweaked; see the documentation for CIAT::RakeTask for more information.
-
Pronounce “CIAT” as “dog”. (See this funny video.)
-
Must have
diffexecutable. -
You have to provide your own target-code processors (e.g.,
parrotfor the Parrot Virtual Machine,spimfor MIPS emulation, etc.)
-
Install
diff. -
Install Ruby and Ruby Gems.
-
sudo gem sources -a http://gems.github.com(only needed once) -
sudo gem install jdfrens-ciat
(The MIT License)
Copyright © 2008 Jeremy D. Frens
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the ‘Software’), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED ‘AS IS’, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.