Fails and warnings while running unit tests

[bnymn]

After fresh installation of Magento 2.2 via composer, some of the unit tests are failing.

Preconditions

1. Magento 2.2
2. PHP 7.0
3. MySQL 5.7.19, for Linux (x86_64)

Steps to reproduce

1. Install Magento with composer.
2. Run this command vendor/bin/phpunit -c dev/tests/unit/phpunit.xml.dist.
3. You get the error.

Expected result

1. To pass all the unit tests

Actual result

1. There are some PHP Warnings as following

PHP Warning:  Invalid argument supplied for foreach() in /var/www/magento2.2/vendor/magento/module-dhl/Model/Carrier.php on line 707
PHP Stack trace:
PHP   1. {main}() /var/www/magento2.2/vendor/phpunit/phpunit/phpunit:0
PHP   2. PHPUnit\TextUI\Command::main() /var/www/magento2.2/vendor/phpunit/phpunit/phpunit:53
PHP   3. PHPUnit\TextUI\Command->run() /var/www/magento2.2/vendor/phpunit/phpunit/src/TextUI/Command.php:141
PHP   4. PHPUnit\TextUI\TestRunner->doRun() /var/www/magento2.2/vendor/phpunit/phpunit/src/TextUI/Command.php:212
PHP   5. PHPUnit\Framework\TestSuite->run() /var/www/magento2.2/vendor/phpunit/phpunit/src/TextUI/TestRunner.php:537
PHP   6. PHPUnit\Framework\TestSuite->run() /var/www/magento2.2/vendor/phpunit/phpunit/src/Framework/TestSuite.php:744
PHP   7. PHPUnit\Framework\TestCase->run() /var/www/magento2.2/vendor/phpunit/phpunit/src/Framework/TestSuite.php:744
PHP   8. PHPUnit\Framework\TestResult->run() /var/www/magento2.2/vendor/phpunit/phpunit/src/Framework/TestCase.php:883
PHP   9. PHPUnit\Framework\TestCase->runBare() /var/www/magento2.2/vendor/phpunit/phpunit/src/Framework/TestResult.php:695
PHP  10. PHPUnit\Framework\TestCase->runTest() /var/www/magento2.2/vendor/phpunit/phpunit/src/Framework/TestCase.php:928
PHP  11. ReflectionMethod->invokeArgs() /var/www/magento2.2/vendor/phpunit/phpunit/src/Framework/TestCase.php:1069
PHP  12. Magento\Dhl\Test\Unit\Model\CarrierTest->testCollectRatesFail() /var/www/magento2.2/vendor/phpunit/phpunit/src/Framework/TestCase.php:1069
PHP  13. Magento\Dhl\Model\Carrier->collectRates() /var/www/magento2.2/vendor/magento/module-dhl/Test/Unit/Model/CarrierTest.php:370
PHP  14. Magento\Dhl\Model\Carrier->_getQuotes() /var/www/magento2.2/vendor/magento/module-dhl/Model/Carrier.php:351
PHP  15. Magento\Dhl\Model\Carrier->_buildQuotesRequestXml() /var/www/magento2.2/vendor/magento/module-dhl/Model/Carrier.php:928
PHP  16. Magento\Dhl\Model\Carrier->_makePieces() /var/www/magento2.2/vendor/magento/module-dhl/Model/Carrier.php:1015
PHP  17. Magento\Dhl\Model\Carrier->_getAllItems() /var/www/magento2.2/vendor/magento/module-dhl/Model/Carrier.php:797
.
Warning: Invalid argument supplied for foreach() in /var/www/magento2.2/vendor/magento/module-dhl/Model/Carrier.php on line 707

Call Stack:
    0.0002     365240   1. {main}() /var/www/magento2.2/vendor/phpunit/phpunit/phpunit:0
    0.0089    1313352   2. PHPUnit\TextUI\Command::main() /var/www/magento2.2/vendor/phpunit/phpunit/phpunit:53
    0.0089    1313464   3. PHPUnit\TextUI\Command->run() /var/www/magento2.2/vendor/phpunit/phpunit/src/TextUI/Command.php:141
    8.6191  314715520   4. PHPUnit\TextUI\TestRunner->doRun() /var/www/magento2.2/vendor/phpunit/phpunit/src/TextUI/Command.php:212
    8.6299  314876768   5. PHPUnit\Framework\TestSuite->run() /var/www/magento2.2/vendor/phpunit/phpunit/src/TextUI/TestRunner.php:537
   33.9640  641796024   6. PHPUnit\Framework\TestSuite->run() /var/www/magento2.2/vendor/phpunit/phpunit/src/Framework/TestSuite.php:744
   34.0214  643598816   7. PHPUnit\Framework\TestCase->run() /var/www/magento2.2/vendor/phpunit/phpunit/src/Framework/TestSuite.php:744
   34.0214  643598816   8. PHPUnit\Framework\TestResult->run() /var/www/magento2.2/vendor/phpunit/phpunit/src/Framework/TestCase.php:883
   34.0215  643601792   9. PHPUnit\Framework\TestCase->runBare() /var/www/magento2.2/vendor/phpunit/phpunit/src/Framework/TestResult.php:695
   34.0229  643689440  10. PHPUnit\Framework\TestCase->runTest() /var/www/magento2.2/vendor/phpunit/phpunit/src/Framework/TestCase.php:928
   34.0229  643689736  11. ReflectionMethod->invokeArgs() /var/www/magento2.2/vendor/phpunit/phpunit/src/Framework/TestCase.php:1069
   34.0229  643689744  12. Magento\Dhl\Test\Unit\Model\CarrierTest->testCollectRatesFail() /var/www/magento2.2/vendor/phpunit/phpunit/src/Framework/TestCase.php:1069
   34.0230  643690728  13. Magento\Dhl\Model\Carrier->collectRates() /var/www/magento2.2/vendor/magento/module-dhl/Test/Unit/Model/CarrierTest.php:370
   34.0252  643706280  14. Magento\Dhl\Model\Carrier->_getQuotes() /var/www/magento2.2/vendor/magento/module-dhl/Model/Carrier.php:351
   34.0252  643706336  15. Magento\Dhl\Model\Carrier->_buildQuotesRequestXml() /var/www/magento2.2/vendor/magento/module-dhl/Model/Carrier.php:928
   34.0264  643714416  16. Magento\Dhl\Model\Carrier->_makePieces() /var/www/magento2.2/vendor/magento/module-dhl/Model/Carrier.php:1015
   34.0265  643715208  17. Magento\Dhl\Model\Carrier->_getAllItems() /var/www/magento2.2/vendor/magento/module-dhl/Model/Carrier.php:797

2. The second PHP warning

PHP Warning:  filesize(): stat failed for /t/e/test.jpg in /var/www/magento2.2/vendor/magento/module-product-video/Controller/Adminhtml/Product/Gallery/RetrieveImage.php on line 181
PHP Stack trace:
PHP   1. {main}() /var/www/magento2.2/vendor/phpunit/phpunit/phpunit:0
PHP   2. PHPUnit\TextUI\Command::main() /var/www/magento2.2/vendor/phpunit/phpunit/phpunit:53
PHP   3. PHPUnit\TextUI\Command->run() /var/www/magento2.2/vendor/phpunit/phpunit/src/TextUI/Command.php:141
PHP   4. PHPUnit\TextUI\TestRunner->doRun() /var/www/magento2.2/vendor/phpunit/phpunit/src/TextUI/Command.php:212
PHP   5. PHPUnit\Framework\TestSuite->run() /var/www/magento2.2/vendor/phpunit/phpunit/src/TextUI/TestRunner.php:537
PHP   6. PHPUnit\Framework\TestSuite->run() /var/www/magento2.2/vendor/phpunit/phpunit/src/Framework/TestSuite.php:744
PHP   7. PHPUnit\Framework\TestCase->run() /var/www/magento2.2/vendor/phpunit/phpunit/src/Framework/TestSuite.php:744
PHP   8. PHPUnit\Framework\TestResult->run() /var/www/magento2.2/vendor/phpunit/phpunit/src/Framework/TestCase.php:883
PHP   9. PHPUnit\Framework\TestCase->runBare() /var/www/magento2.2/vendor/phpunit/phpunit/src/Framework/TestResult.php:695
PHP  10. PHPUnit\Framework\TestCase->runTest() /var/www/magento2.2/vendor/phpunit/phpunit/src/Framework/TestCase.php:928
PHP  11. ReflectionMethod->invokeArgs() /var/www/magento2.2/vendor/phpunit/phpunit/src/Framework/TestCase.php:1069
PHP  12. Magento\ProductVideo\Test\Unit\Controller\Adminhtml\Product\Gallery\RetrieveImageTest->testExecute() /var/www/magento2.2/vendor/phpunit/phpunit/src/Framework/TestCase.php:1069
PHP  13. Magento\ProductVideo\Controller\Adminhtml\Product\Gallery\RetrieveImage->execute() /var/www/magento2.2/vendor/magento/module-product-video/Test/Unit/Controller/Adminhtml/Product/Gallery/RetrieveImageTest.php:152
PHP  14. Magento\ProductVideo\Controller\Adminhtml\Product\Gallery\RetrieveImage->appendResultSaveRemoteImage() /var/www/magento2.2/vendor/magento/module-product-video/Controller/Adminhtml/Product/Gallery/RetrieveImage.php:120
PHP  15. filesize() /var/www/magento2.2/vendor/magento/module-product-video/Controller/Adminhtml/Product/Gallery/RetrieveImage.php:181
F
Warning: filesize(): stat failed for /t/e/test.jpg in /var/www/magento2.2/vendor/magento/module-product-video/Controller/Adminhtml/Product/Gallery/RetrieveImage.php on line 181

Call Stack:
    0.0002     365240   1. {main}() /var/www/magento2.2/vendor/phpunit/phpunit/phpunit:0
    0.0089    1313352   2. PHPUnit\TextUI\Command::main() /var/www/magento2.2/vendor/phpunit/phpunit/phpunit:53
    0.0089    1313464   3. PHPUnit\TextUI\Command->run() /var/www/magento2.2/vendor/phpunit/phpunit/src/TextUI/Command.php:141
    8.6191  314715520   4. PHPUnit\TextUI\TestRunner->doRun() /var/www/magento2.2/vendor/phpunit/phpunit/src/TextUI/Command.php:212
    8.6299  314876768   5. PHPUnit\Framework\TestSuite->run() /var/www/magento2.2/vendor/phpunit/phpunit/src/TextUI/TestRunner.php:537
   42.2900  752315928   6. PHPUnit\Framework\TestSuite->run() /var/www/magento2.2/vendor/phpunit/phpunit/src/Framework/TestSuite.php:744
   42.2916  752316552   7. PHPUnit\Framework\TestCase->run() /var/www/magento2.2/vendor/phpunit/phpunit/src/Framework/TestSuite.php:744
   42.2916  752316552   8. PHPUnit\Framework\TestResult->run() /var/www/magento2.2/vendor/phpunit/phpunit/src/Framework/TestCase.php:883
   42.2917  752319528   9. PHPUnit\Framework\TestCase->runBare() /var/www/magento2.2/vendor/phpunit/phpunit/src/Framework/TestResult.php:695
   42.3047  752789864  10. PHPUnit\Framework\TestCase->runTest() /var/www/magento2.2/vendor/phpunit/phpunit/src/Framework/TestCase.php:928
   42.3047  752790160  11. ReflectionMethod->invokeArgs() /var/www/magento2.2/vendor/phpunit/phpunit/src/Framework/TestCase.php:1069
   42.3047  752790168  12. Magento\ProductVideo\Test\Unit\Controller\Adminhtml\Product\Gallery\RetrieveImageTest->testExecute() /var/www/magento2.2/vendor/phpunit/phpunit/src/Framework/TestCase.php:1069
   42.3049  752797632  13. Magento\ProductVideo\Controller\Adminhtml\Product\Gallery\RetrieveImage->execute() /var/www/magento2.2/vendor/magento/module-product-video/Test/Unit/Controller/Adminhtml/Product/Gallery/RetrieveImageTest.php:152
   42.3056  752804240  14. Magento\ProductVideo\Controller\Adminhtml\Product\Gallery\RetrieveImage->appendResultSaveRemoteImage() /var/www/magento2.2/vendor/magento/module-product-video/Controller/Adminhtml/Product/Gallery/RetrieveImage.php:120
   42.3058  752806392  15. filesize() /var/www/magento2.2/vendor/magento/module-product-video/Controller/Adminhtml/Product/Gallery/RetrieveImage.php:181

3. Failed test

There was 1 failure:

1) Magento\ProductVideo\Test\Unit\Controller\Adminhtml\Product\Gallery\RetrieveImageTest::testExecute
Expectation failed for method name is equal to <string:getDirectoryWrite> when invoked 1 time(s).
Method was expected to be called 1 times, actually called 0 times.

[barbazul]

I haven't dug too deep into the code but it looks like when the filesystem mock is set up, getDirectoryWrite is expected to by called once https://github.com/magento/magento2/blob/2.2-develop/app/code/Magento/ProductVideo/Test/Unit/Controller/Adminhtml/Product/Gallery/RetrieveImageTest.php#L149

However, that method is only called if there is an exception: https://github.com/magento/magento2/blob/2.2-develop/app/code/Magento/ProductVideo/Controller/Adminhtml/Product/Gallery/RetrieveImage.php#L123

None of the mocks being set up is told to throw an exception (unlike the testExecuteInvalid... tests).

So it looks like the failure itself is being caused by wrongly setting up the mock to expect a call to getDirectoryWrite()

I am really surprised this error does not show up in Travis https://travis-ci.org/magento/magento2/jobs/307118009

[Mobecls]

This issue is not appearing on freshly installed 2.2.4-dev

[matiasnombarasco]

I can confirm that same issue in 2.2.0

[careys7]

Same issue in 2.2.2

[pascal08]

This is the only failing test on 2.2.2 for me.

[clemblanco]

Same issue with 2.2.3