Support merging results?

[jonspalmer]

How complex would it be to support merging test results? We're running tests in parallel, its easy to configure each one to spit out separate files but it would be great to be able to merge results across the entire test suite into one report of flamechart.

[palkan]

This is one of the features from our Roadmap.

It's kinda complex feature, 'cause it depends on others:

1. first, we need a way to dump reports into files (Saving reports to file for later analysis #113); the idea is to provide Formatters API
2. then we need to add the merging functionality itself: given N reports (in some format) produce an aggregated report (again, in some format); the main question here is where to trigger this aggregation? Probably, we need a CLI tool: test-prof generate factory-flame path/to/report1 path/to/report2 or smth like that.

The 1) seems clear (from the implementation point of view), the 2) is not as clear: we need to think about a good UX for that.

So, will be glad to hear your suggestions)

[jonspalmer]

Sounds right. #113 seems pretty straight forward. IMO you'd dump the results to json or yml. I'd look to how Ruby exports a very simple json object for coverage reports (which makes them super easy to merge)

On 2. I'd imagine a set of rake tests for the CLI. Supporting glob patterns would be simplest. I'd think of it as:
bundle exec rake test_prof:generate --type factory-flame --path path/to/**/*.fprof.json

Loving the results I'm getting so far btw.

[modosc]

is this in progress at all or would you accept a pr / help

[palkan]

The full-featured support is planned for the v2 release cycle (no roadmap publicly available yet), which is going to standardize the profilers and tools APIs.

So, for the current version (which is going to be released as v1 later this year) I do plan to introduce such massive changes.

On the other hand, if you have some specific ideas or request (e.g., adding JSON support for a particular profiler)—I would be glad to accept a PR 🙂.