TestSuite
In order to meet the goals of issues 6 (cross-browser compatibility) and 15 (performance benchmarking), we need to have a test-suite that would execute the compiled code in all (most) of the major browsers.
The best solution appears to be using Selenium 2, which also has Haskell bindings (see package 'webdriver' on Hackage). This would allow us to test the output of GHCJS with most major browsers (Firefox, IE, Chrome, Opera). The only inconvenience seems to be the requirement for the test machine to have X11 (or another window system) enabled.
The test-suite is going to support two types of tests: simple and interaction.
Simple tests will only be able to test batch-mode computations. A test will produce a single boolean value --true or false-- which will represent the success or failure of the test case. A proposal has been made to use Template Haskell and have tests compare the results of different Haskell expressions compiled to JavaScript with the same expressions, but pre-computed with Template Haskell. Another possibility might be to utilize QuickCheck/Arbitrary to generate random Haskell expressions to test.
Two test interfaces were proposed, that differ in how the test result is reflected.
-
VarResult -- the result is stored in a variable. The test program in Haskell will have to declare and export (using FFI?) a top-level variable named "result :: Bool" that should be assigned the test result. The variable should be exported to avoid minifying it and, thus, changing it's name in the resulting JavaScript code. The test-suite would then load the test file (wrap it in an HTML page, maybe) and check the value of the variable after a timeout (say, 1 second) by executing the following JavaScript code in the context of the page: "result;" -- which would return a boolean back to the test driver.
-
DOMResult -- the result is stored in a DOM element. The test program should created a DOM element of type with an id="result" and either 1 or 0 as it's text. This would require us to have FFI bindings to, at least, 2 DOM functions (createElement and appendChild) as well as being able to set the ID property. Alternatively, a helper function (say, recordTestResult could be included with the compiled code that would perform the necessary modifications, so we would only need to bindings to that function.
These tests will be able to check interactive computations, and will be useful on a later stage, when bindings to DOM are developed. The provisional format is as follows: each test takes a WebDriver script (WebDriver Bool) which is executed in the context of the test page. The script would be able to select different elements, click on buttons, fill and submit forms and then verify results via inspection of the DOM tree.
The first milestone would be to support Simple/VarResult tests without using Template Haskell and automated generation of test-cases.