Repeat failed tests

[rentalhost]

It's a complement for #1022 closed by 5.0 version.

I guess that a command line like --repeat should be useful to try first lasts failed tests.

It should works like that: when you run phpunit, it'll create a temporary file that should have the info about any failed tests (basically: class and test name and test index if it uses a data provider).

[
    { "class": "MyTest", "method": "testSingle", "index": null },
    { "class": "MyTest", "method": "testWithDataProvider", "index": 14 },
    { "class": "MyTest", "method": "testWithDataProvider", "index": 19 },
]

After an error, user could uses the command --repeat to make phpunit run initially this failed tests. If no more error is found, then phpunit could continues to run all others tests (except the previously faileds).

---

Example case:

> phpunit
FAILURES!
Tests: 100, Assertions: 500, Failures: 3.

1. MyTest::testSingle() expects string by get int;
2. MyTest::testWithDataProvider()#14 expects 5 but get 6;
3. MyTest::testWithDataProvider()#19 expects 8 but get 90;

After fix two errors only:

> phpunit --repeat
Rerunning last fails...

FAILURES!
Tests: 100, Assertions: 500, Failures: 1.

1. MyTest::testWithDataProvider()#14 expects 5 but get 18;

After fix last error:

> phpunit --repeat
Rerunning last fails... OK.
Running all others tests...

OK (100 tests, 500 assertions)

---

Additional commands:

• User could uses --repeat-fails to repeat only last failed tests (without rerun all);

[sebastianbergmann]

@lstrojny Are you still interested in moving https://github.com/lstrojny/phpunit-clever-and-smart into PHPUnit's core?

[lstrojny]

@sebastianbergmann Do I need to do work for it? 😉 In all seriousness, I am too swamped to do it but I would be totally for phpunit-clever-and-smart to be part of phpunit.

[janvt]

Would love to get involved in this, have dreamed of similar things (and failed at bash-scripting them) in the past!

[sebastianbergmann]

Duplicate of #46.