Lazy loaded array-like-objects data providers?

[Wes0617]

Good morning and happy new year.

I've had this issue in the past weeks, that is too big data providers and too many data sets (especially objects are super heavy). PHPUnit has to load all the things and keep them in memory (i think) just to know the total count of data sets.

Here #836 (comment) @sebastianbergmann wrote

In a perfect world we would break B/C and require data providers to be objects that implement an interface that extends Countable.

Why not just allow array-like objects and leave them alone without converting them to arrays first?

function myDataProvider(){
    return new class implements IteratorAggregate, Countable{
         // data is created lazily while iterating
        function getIterator(){
            /*... */
            yield "name" => [$arg1, $arg2, $arg3];
            /*... */ 
        }
        // doesn't require the whole data to be actually counted
        function count(){ return 10000; }
    };
}

[Jean85]

I don't think this is needed. Also it seems a bit complex to read.

I've recently seen a different working approach for this issue, and it seemed really readable too to me: https://github.com/symfony/symfony/blob/3f964689421ac1490f146c71a0811a892bffa684/src/Symfony/Component/Security/Core/Tests/Authorization/DebugAccessDecisionManagerTest.php#L35

    /**
     * @dataProvider provideObjectsAndLogs
     */
    public function testDecideLog($expectedLog, $object)
    {
        // ...
    }

    public function provideObjectsAndLogs()
    {
        yield array($one, $two);
        yield array($one, null);
        // ...
    }

[Wes0617]

I think you missed the point. PHPUnit will convert that iterator to array regardless, just to retrieve its count(). I'm not interested in knowing the total count of tests if it happens at the expense of speed.

[Jean85]

I'm sorry but I can't see how your solution should improve speed; you are loading a lot of code in memory in both case, right?.

Also, I suggest to you to do some benchmarking first, since this change would go into PHP 7 only (master is now for PHPUnit 6) where array are highly optimized.

[Wes0617]

I'm sorry but I can't see how your solution should improve speed; you are loading a lot of code in memory in both case, right?.

No. If iterators are left unconverted, data sets are picked and loaded in memory one at a time:

yield [new Foo(111)];
yield [new Foo(222)]; // this Foo is not constructed until phpunit does $dataProvider->next()

What PHPUnit does currently is loading all data sets from all data providers in the whole test suite before the first test is even started. And that, just to retrieve the total count of data sets, I believe.

[sebastianbergmann]

Your assumption is wrong. The conversion does not happen just for count() but to create test objects for each data set.

[Wes0617]

Which test objects? I'm sorry I know nothing about PHPUnit's internals. What I'm suggesting is simply that such objects could be constructed only upon actual need rather than being created all at once at the startup.

[sebastianbergmann]

With the current architecture of the test runner that is not possible.