Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for additional test reporters #39

Open
fatuhoku opened this issue Mar 27, 2013 · 2 comments
Open

Support for additional test reporters #39

fatuhoku opened this issue Mar 27, 2013 · 2 comments

Comments

@fatuhoku
Copy link

The default output format is great; but it's not suitable for consumption by build management, reporting, or CI systems.

To make test-framework become a viable part of a software development pipleine, it's useful to support other formats of test result outputs.

For instance, Mocha supports a vast range of variably useful reporters:

$ mocha --reporters

    dot - dot matrix
    doc - html documentation
    spec - hierarchical spec list
    json - single json object
    progress - progress bar
    list - spec-style listing
    tap - test-anything-protocol
    landing - unicode landing strip
    xunit - xunit reportert
    teamcity - teamcity ci support  ***
    html-cov - HTML test coverage
    json-cov - JSON test coverage
    min - minimal reporter (great with --watch)
    json-stream - newline delimited json events
    markdown - markdown documentation (github flavour)
    nyan - nyan cat!

For instance, Mocha's spec reporter looks like this:
Mocha's spec reporter

This issue requires refactoring that would abstract from the way test progress and information is printed onto standard out at the moment into something more versatile.

Test reporters can have

  • streaming output (like test-framework's current one, or progress bar): more suitable for development use in the terminal
  • holistic (like the single JSON object that gets spit out right at the end): more suitable for consumption by other build processes.

Some part of this is necessary for #38 to be implemented. It's questionnable whether this issue can become a part of the #38 altogether.

@fatuhoku
Copy link
Author

Much of the existing test runner/reporter lives in Run.hs. We must tear apart logic that creates Doc from putDoc (which outputs it directly onto standard output). This would provide several extension points where Docs can be output. The need for this is most evident in the amount of functions in the IO monad in that source file.

@UnkindPartition
Copy link
Contributor

Is there more to this than alternative runners? test-framework already supports XML output as an alternative runner, for example.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants