Exit code changed since 5.7.7

[greg0ire]

All the builds in our org projects are failing because we keep using getMock() until we drop phpunit 4, but run them with phpunit 5. Not very clever, I know, but since yesterday, the previously harmless warnings triggered by getMock() make the build fail because it seems the exit code is no longer 0. This looks like a BC-break, are you going to revert it or is it deliberate?

[michaelehrlein]

Same problem in our build chain, downgraded to 5.7.6 this morning...

[remicollet]

Same thing, seems critical.

[sebastianbergmann]

The only change in PHPUnit 5.7.7 that I can think of that would be causing this is #2349 / f4cf4d1.

I believe that exiting with a return code that signals "not successful" is the right thing to do when there are warnings. I will revert PHPUnit 5.7 to not signal "not successful" when there are warnings. But I will leave this change in PHPUnit 6.0.

[sebastianbergmann]

I can reproduce this issue:

<?php
class Test extends PHPUnit_Framework_TestCase
{
    public function testOne()
    {
        throw new PHPUnit_Framework_Warning;
    }
}

PHPUnit 5.7.6

$ phpunit Test
PHPUnit 5.7.6 by Sebastian Bergmann and contributors.

W                                                                   1 / 1 (100%)

Time: 39 ms, Memory: 4.00MB

There was 1 warning:

1) Test::testOne

/home/sb/Test.php:6

WARNINGS!
Tests: 1, Assertions: 0, Warnings: 1.

$ echo $?                  
0

PHPUnit 5.7.7

$ phpunit Test
PHPUnit 5.7.7 by Sebastian Bergmann and contributors.

W                                                                   1 / 1 (100%)

Time: 33 ms, Memory: 4.00MB

There was 1 warning:

1) Test::testOne

/home/sb/Test.php:6

WARNINGS!
Tests: 1, Assertions: 0, Warnings: 1.

$ echo $?
1