add todo and done features

[andrewfowlie]

• I have reviewed the Contributor Guidelines.
• I have reviewed the Code of Conduct and agree to abide by it

This is a continuation of PR38 (#38). Unfortunately, that PR was so old and I encountered some problems rebasing against master, so it was faster for me to start from scratch.

I have added the todo directive as described in the TAP specification - https://testanything.org/tap-version-13-specification.html

The syntax is similar to the skip directive:

@test "todo test that passes" {
  todo "this todo is done"
  return 0
}

@test "todo test that fails" {
  todo "this todo is outstanding"
  return 1
}

This results in

 ✓ todo test that passes (done: this todo is done)
 - todo test that fails (todo: this todo is outstanding)
   (in test file todo.tap, line 7)
     `todo "this todo is outstanding"' failed

2 tests, 0 failures, 1 todo, 1 done

and in TAP format:

1..2
ok 1 todo test that passes # done this todo is done
not ok 2 todo test that fails # todo this todo is outstanding
# (in test file todo.tap, line 7)
#   `todo "this todo is outstanding"' failed

[martin-schulze-vireso]

I am sorry that this has been lying around so long and would require a rebase/rewrite again.

Looking through your code, I see that there are no tests pertaining to done. Looking more closely, it seems that your added tests (fixture) never get called. Additionally, it looks like it might be possible to combine todo and skip. What should the outcome of this be? Furthermore, should a done test only print a report or generate a non-zero exit code, to hightlight that a todo is not a todo anymore?

[martin-schulze-vireso]

Reading the TAP specification more closely, I only see todo being mentioned, not done:

TODO tests

If the directive starts with # TODO, the test is counted as a todo test, and the text after TODO is the explanation.

not ok 13 # TODO bend space and time
Note that if the TODO has an explanation it must be separated from TODO by a space. These tests represent a feature to be implemented or a bug to be fixed and act as something of an executable “things to do” list. They are not expected to succeed. Should a todo test point begin succeeding, the harness should report it as a bonus. This indicates that whatever you were supposed to do has been done and you should promote this to a normal test point.

So I think the correct way to write the done case would be:

ok 1 Test name # TODO todo explanation

Then there is the question if a not ok # TODO should make give a non-zero exit code. The TAP spec seems to suggest to ignore these test failures.