symfony/finder incompatibility

[bytestream]

Problem/Motivation

My CI jobs are occasionally failing with the below error

Expected behaviour

No fatal error

Actual behaviour

Fatal error: Uncaught Error: Cannot call constructor in /builds/dist/vendor/symfony/finder/Comparator/NumberComparator.php:76
Stack trace:
#0 phar:///usr/bin/composer/vendor/symfony/finder/Finder.php(230): Symfony\Component\Finder\Comparator\NumberComparator->__construct('<= 3')
#1 /builds/dist/vendor/dealerdirect/phpcodesniffer-composer-installer/src/Plugin.php(449): Symfony\Component\Finder\Finder->depth('<= 3')
#2 /builds/dist/vendor/dealerdirect/phpcodesniffer-composer-installer/src/Plugin.php(194): Dealerdirect\Composer\Plugin\Installers\PHPCodeSniffer\Plugin->updateInstalledPaths()
#3 [internal function]: Dealerdirect\Composer\Plugin\Installers\PHPCodeSniffer\Plugin->onDependenciesChangedEvent(Object(Composer\Script\Event))
#4 phar:///usr/bin/composer/src/Composer/EventDispatcher/EventDispatcher.php(191): call_user_func(Array, Object(Composer\Script\Event))
#5 phar:///usr/bin/composer/src/Composer/EventDispatcher/EventDispatcher.php(118): Composer\EventDisp in /builds/dist/vendor/symfony/finder/Comparator/NumberComparator.php on line 76

Steps to reproduce

Unclear, happens randomly. Composer version is fixed in a Docker image.

Proposed changes

The issue seems to be with the auto loader and a mismatch in symfony/finder versions. I depend on v5.4.0 while composer.phar uses v2.8 (I think). The Cannot call constructor error is because v5.4.0 calls parent::__construct() and a __construct() doesn't exist in the parent class (due to the version mismatch).

Environment

Question	Answer
OS	Debian 11 (bullseye)
PHP version	7.4.27
Composer version	2.2.1
PHP_CodeSniffer version	3.6.2
Dealerdirect PHPCS plugin version	0.7.1
Install type	curl -L -o /usr/bin/composer https://getcomposer.org/composer-2.phar && chmod +x /usr/bin/composer

Tested against master branch?

• I have verified the issue still exists in the master branch.

[jrfnl]

@bytestream Thanks for reporting this. Unfortunately, without a clear way to reproduce this issue, there is not much we can do.

[bytestream]

I think we determined in composer/composer#10413 that it's a composer issue.

[jrfnl]

@bytestream Thanks for the update!

[franck-grenier]

Hello,
having the same (rare) issue on a Laravel 8 app and Composer 2.1.6.

Fixed by removing slevomat/coding-standard dependency which became useless after moving our phpcs to PSR12.

[jrfnl]

I've written up a test scenario to see if this in any reliable way could be reproduced, but no luck.

Test: 9eda3f1
Run results: https://github.com/PHPCSStandards/composer-installer/actions/runs/2418906450

@Potherca Shall I clean up the test and pull it anyway ? Or leave this as "can't reproduce" ?

[Potherca]

Looking at composer/composer#10413 I don't think this is part of "our" side of the equation, so I'd vote for "leave as is".

[jrfnl]

Looking at composer/composer#10413 I don't think this is part of "our" side of the equation, so I'd vote for "leave as is".

Yes and no.

Yes, this is primarily a Composer native problem.

My analysis of what's happening, is as follows:

1. The Composer phar is build with the highest supported version of the Symfony/Finder package still supporting the minimum PHP version which needs to be supported for that version of Composer.
   This means the following in practice:

   Composer phar	Includes Finder
   1.x	2.x
   2.0	2.x
   2.3	5.x

2. During the Composer run, Composer will load whatever Finder files it needs from the version included with the PHAR file.

3. At some point during the Composer run - but before the plugin is run, the autoload file is dumped and included and prepended to the autoload stack (this prepend is the important bit!).

4. If the package for which the autoload was generated has its own dependency on Symfony/Finder, the vendor/autoload will include Symfony/Finder and most likely at a different (higher) version than the version included in the Composer PHAR.

5. The plugin subsequently uses Symfony/Finder and hits a class which Composer itself has not used yet. This class will now be autoloaded via the vendor/autoload file instead of from the Composer PHAR file, leading to a version mismatch between the classes previously loaded (2.x) and the class now being loaded (3.x, 4.x, 5.x, 6.x).

6. And as the Symfony/Finder package naturally has had changes along the way, the 2.x Finder class is calling the 5.x Finder class "in the wrong way" leading to the fatal error.

So, yes, this is primarily a Composer problem as the Finder package included in the Composer PHAR is not prefixed using something like PHP_Scoper (which could prevent this issue).

But no...

The "no" answer is that we could possibly use a try {} catch {} in one or more select places to prevent the fatal error and throw an informative error message instead.

But to do so, I'd need to be able to reproduce the issue consistently to ensure the try {} catch {}(-es) are placed in the right place and actually prevent the fatal.

@Potherca Does that make sense ?

Anyway, that was the reason for me to try and reproduce the issue.

Either way, this write up will hopefully help the next person running into the issue understand what's going on 😄

[Potherca]

The "no" answer is that we could possibly use a try {} catch {} in one or more select places to prevent the fatal error and throw an informative error message instead.

We might need to take a look at other/more places a try/catch might be needed to guard against external code. However, I think if such things are added, they should be tested at the unit level, as that would also make it trivial to reproduce since the exception would then be thrown from a mock instead of the actual code.

Unless / Until this problem becomes more prevalent, I think it should not receive much priority (other than us thinking about the try/catch in general for hardening this project regarding external code).

Either way, this write up will hopefully help the next person running into the issue understand what's going on

Amen Indubitably. 🙏

[danniehansen]

We're seeing the same thing after installing dealerdirect/phpcodesniffer-composer-installer to get https://github.com/FloeDesignTechnologies/phpcs-security-audit working.

We're currently using the latest stable composer version in CI & are randomly seeing this issue on our CI jobs. Any suggestion workaround and/or fix one can apply to resolve this?

[Potherca]

@danniehansen My apologies for the late reply, I watch comments on issues less strictly than new issue.

Regarding FloeDesignTechnologies/phpcs-security-audit, as this issue lies largely beyond the code of this project more information would be needed to make any suggestions (like which php, composer, codesniffer and plugin versions are used and potentially which other dependencies were used.)

[limonazzo]

I update friendsofphp/php-cs-fixer to the last version (3.10) and errors disappear.
😐

[danniehansen]

@Potherca Sorry for the late reply as well. Appreciate you looking into this. Here is the information that I have at hand:

PHP version: 8.0

composer.json:

Click to expand composer.json details

{
    "name": "....",
    "type": "project",
    "description": ".....",
    "keywords": [],
    "license": "MIT",
    "repositories": [
        {
            "type": "git",
            "url": "https://github.com/danniehansen/FuelSDK-PHP.git"
        }
    ],
    "require": {
        "php": "^8.0",
        "ext-curl": "*",
        "ext-dom": "*",
        "ext-ftp": "*",
        "ext-gd": "*",
        "ext-intl": "*",
        "ext-json": "*",
        "ext-openssl": "*",
        "ext-pdo": "*",
        "ext-redis": "*",
        "ext-simplexml": "*",
        "ext-soap": "*",
        "ext-zip": "*",
        "24slides/laravel-saml2": "^2.0.10",
        "aws/aws-sdk-php-laravel": "^3.6.0",
        "bacon/bacon-qr-code": "^2.0.6",
        "box/spout": "^v3.3.0",
        "bref/bref": "^1.5.6",
        "bref/extra-php-extensions": "^0.11.30",
        "bref/laravel-bridge": "^1.2.0",
        "campaignmonitor/createsend-php": "^v6.1.2",
        "doctrine/dbal": "^2.13.7",
        "drewm/mailchimp-api": "^v2.5.4",
        "emailplatform/api_parser": "^1.2.4",
        "fideloper/proxy": "^4.4.1",
        "firebase/php-jwt": "^v5.5.1",
        "fruitcake/laravel-cors": "^v2.2.0",
        "genealabs/laravel-model-caching": "^0.11.0",
        "google/apiclient": "^v2.12.1",
        "guzzlehttp/guzzle": "^7.4.5",
        "heyloyalty/hl-phpclient": "^1.6.3",
        "imtigger/laravel-job-status": "^1.2.0",
        "intervention/image": "^2.7.1",
        "laravel/framework": "^v8.83.3",
        "laravel/passport": "^v10.3.2",
        "laravel/tinker": "^v2.7.0",
        "laravel/ui": "^v3.4.5",
        "lavary/laravel-menu": "^v1.8.3",
        "league/flysystem-aws-s3-v3": "^1.0.29",
        "league/flysystem-cached-adapter": "^1.1.0",
        "league/oauth2-client": "^2.6.1",
        "maxmind-db/reader": "^v1.11.0",
        "owen-it/laravel-auditing": "^v12.2.1",
        "phpoffice/phpspreadsheet": "^1.22.0",
        "phpseclib/phpseclib": "^2.0.36",
        "picqer/php-barcode-generator": "^v2.2.1",
        "pragmarx/google2fa": "^8.0.0",
        "rennokki/laravel-sns-events": "^6.4.0",
        "salesforce-mc/fuel-sdk-php": "dev-hotfix/php-7.4",
        "sentry/sentry-laravel": "^2.5.3",
        "shiftonelabs/laravel-sqs-fifo-queue": "^2.0.1",
        "snowplow/snowplow-tracker": "^0.4.0",
        "sonata-project/google-authenticator": "^2.3.1",
        "symfony/yaml": "^v5.4.3",
        "whichbrowser/parser": "^v2.1.2"
    },
    "require-dev": {
        "brainmaestro/composer-git-hooks": "^v2.8.5",
        "dealerdirect/phpcodesniffer-composer-installer": "^0.7.2",
        "facade/ignition": "^2.17.5",
        "itsgoingd/clockwork": "^v5.1.5",
        "knuckleswtf/scribe": "3.18.0",
        "micheh/phpcs-gitlab": "^1.1",
        "mockery/mockery": "^1.5.0",
        "nunomaduro/collision": "^v5.11.0",
        "nunomaduro/larastan": "^v0.7.15",
        "orchestra/testbench": "^v6.24.1",
        "pheromone/phpcs-security-audit": "^2.0",
        "phpstan/phpstan": "^0.12.99",
        "phpunit/phpunit": "^9.5.18",
        "roave/security-advisories": "dev-master",
        "spatie/laravel-ray": "^1.29.4",
        "squizlabs/php_codesniffer": "^3.6.2"
    },
    "config": {
        "preferred-install": "dist",
        "sort-packages": true,
        "allow-plugins": {
            "dealerdirect/phpcodesniffer-composer-installer": true
        }
    },
    "extra": {
        "laravel": {
            "dont-discover": []
        },
        "hooks": {
            "pre-push": [
                "git-hooks/pre-push.sh"
            ]
        }
    },
    "autoload": {
        "psr-4": {
            "App\\": "app/",
            "Database\\Factories\\": "database/factories/",
            "Database\\Seeders\\": "database/seeders/"
        },
        "files": [
            "app/Utilities/Includes/Utilities.php"
        ]
    },
    "autoload-dev": {
        "psr-4": {
            "Tests\\": "tests/"
        }
    },
    "minimum-stability": "dev",
    "prefer-stable": true,
    "scripts": {
        "post-install-cmd": "git-hooks/add.sh",
        "post-update-cmd": "git-hooks/add.sh",
        "post-autoload-dump": [
            "Illuminate\\Foundation\\ComposerScripts::postAutoloadDump",
            "@php artisan package:discover --ansi"
        ],
        "post-root-package-install": [
            "@php -r \"file_exists('.env') || copy('.env.example', '.env');\""
        ],
        "post-create-project-cmd": [
            "@php artisan key:generate --ansi"
        ]
    }
}

Versions resolved from selectors related.

• squizlabs/php_codesniffer: 3.7.1
• pheromone/phpcs-security-audit: 2.0.1
• micheh/phpcs-gitlab: 1.1.0
• dealerdirect/phpcodesniffer-composer-installer: 0.7.2

We're seeing the issue quite frequently in our jobs. Stopping test suites, builds & releases.

[mreduar]

The same thing happens to me. We are using bitbucket pipelines and I think the problem occurs when there are two or more pipelines running at the same time.

[jrfnl]

I'm reopening the issue for further investigation, though I'm not holding out much hope that we'll be able to find a stable solution. All the same, between the combined brainpower in this thread who knows what we can come up with ;-)

[Potherca]

I'm reopening the issue for further investigation

yeah, I'd love to be able to reproduce this. I've been chewing on an idea that boils down to brute forcing a reproduction scenario: automatically creating all variations of a known "broken" composer.json's dependencies and then triggering random builds on those variations until one of them breaks in the expected fashion...

[mreduar]

Here you can see the composer.json which sometimes causes the error.
https://www.klgrth.io/paste/xfmvb/raw
If you need any other information that I can provide I will be attentive.

[dannieleadfamly]

One thing I've found that may help reproduce this is when resources on the machine running the CI job are exhausted. After we switched from raw EC2 instances running our CI jobs with 1 job per machine to Kubernetes with 2 EC2 instances running concurrent jobs - then we started seeing it almost every time.

So I have a sinking feeling that it may be more easily reproducible in an environment that is under stress. Could maybe be possible to provoke through a docker container restricted to a few CPU/RAM resources?

[danniehansen]

For anyone else that become stuck on this issue & need a solution here and now. We decided just to script our way out of it using the following bash script:

echo "Installing dependencies..."
while ! composer install --prefer-dist
do
  ((c++)) && ((c==10)) && break
  echo "Installing dependencies retry in 1 second..."
  sleep 1
done

In our case it automatically resolves itself after attempting the second time. Leading us to think it's a matter of system constraints at that point in time provoking it. We've had 0 issues after using the above bash in our pipelines since adding it.

[Potherca]

Just to reiterate:

• We do not have a documented reproducible scenario yet
• It might be possible to create such a documented scenario based on the provided comment
• Regardless, it we do want to add try/catch statements to the code
• We have not yet defined where those try-catch statements should go

So... The most actionable item is opening en MR with try-catch around possible failure scenarios (i.e. third-party calls) and/or adding an outer / catch-all try-catch.