RCA of the 3.16.0 release

[johnbillion]

I'm performing a root cause analysis of why the 3.16.0 release was broken so I can hopefully prevent it happening again. I'll update this issue as I go.

Problem

Version 3.16.0 was released on April 22nd. Users who updated to this version immediately reported a fatal error that caused their sites to go down.

Stack trace from a WP-CLI request:

PHP Fatal error: Uncaught Error: Failed opening required 'wp-content/plugins/query-monitor/vendor/composer/../symfony/deprecation-contracts/function.php' (include_path='.:/usr/share/php') in wp-content/plugins/query-monitor/vendor/composer/autoload_real.php:42
Stack trace:
0 wp-content/plugins/query-monitor/vendor/composer/autoload_real.php(46): {closure}()
1 wp-content/plugins/query-monitor/vendor/autoload.php(25): ComposerAutoloaderInitad25d53a23099da4b0886b4f6754360e::getLoader()
2 wp-content/plugins/query-monitor/query-monitor.php(56): require_once('…')
3 wp-settings.php(517): include_once('…')
4 phar:///usr/local/bin/wp/vendor/wp-cli/wp-cli/php/WP_CLI/Runner.php(1374): require('…')
5 phar:///usr/local/bin/wp/vendor/wp-cli/wp-cli/php/WP_CLI/Runner.php(1293): WP_CLI\Runner->load_wordpress()
6 phar:///usr/local/bin/wp/vendor/wp-cli/wp-cli/php/WP_CLI/Bootstrap/LaunchRunner.php(28): WP_CLI\Runner->start()
7 phar:///usr/local/bin/wp/vendor/wp-cli/wp-cli/php/bootstrap.php(83): WP_CLI\Bootstrap\LaunchRunner->process()
8 phar:///usr/local/bin/wp/vendor/wp-cli/wp-cli/php/wp-cli.php(32): WP_CLI\bootstrap()
9 phar:///usr/local/bin/wp/php/boot-phar.php(20): include('…')
10 /usr/local/bin/wp(4): include('…')
11 {main}
thrown in wp-content/plugins/query-monitor/vendor/composer/autoload_real.php on line 42

References

• https://wordpress.org/support/topic/last-update-6-min-ago-dependency-fail/
• Problem w/ the auto-loader in 3.16.0 #861

Notes

• The tag and release zip on GitHub for 3.16 are as expected
• Here are the changes between 3.15.0 and 3.16.0
• Here are the changes for plugin-infrastructure between the two releases
• The problem is with the deployment to wordpress.org

Root cause

The plugin at version 3.16.0 as shipped to wordpress.org triggered a fatal error

Why?

The vendor/composer/autoload_real.php was attempting to load a vendor/symfony/deprecation-contracts/function.php PHP file that does not (and should not) exist

Why?

The vendor/composer/autoload_real.php file contains unexpected calls to require for a bunch of files that do not exist in the build

• Compare version 3.15 to version 3.16
• Main change is the addition of:

  $filesToLoad = \Composer\Autoload\ComposerStaticInitad25d53a23099da4b0886b4f6754360e::$files;
  $requireFile = \Closure::bind(static function ($fileIdentifier, $file) {
      if (empty($GLOBALS['__composer_autoload_files'][$fileIdentifier])) {
          $GLOBALS['__composer_autoload_files'][$fileIdentifier] = true;
  
          require $file;
      }
  }, null, null);
  foreach ($filesToLoad as $fileIdentifier => $file) {
      $requireFile($fileIdentifier, $file);
  }

When composer dump-autoload --no-dev is run, the $filesToLoad code isn't present. It's only present when the autoloader is created with dev dependencies.

Why?

At the point where the Composer autoload files were last generated within the deploy-tag.yml or reusable-deploy-tag.yml workflow, Composer thought those files were needed

Why?

composer install is called within the reusable-deploy-tag.yml workflow, which appears to be the cause of the regenerated autoloader.

Why?

The above is the cause of the regenerated autoloader but I've been unable to determine why this didn't affect version 3.15 which was released after the introduction of this call to composer install. The logs for GitHub Actions workflow runs are only retained for 90 days.

Additional issues

• The adjustment to the setClassMapAuthoritative line (which should happen here) was also not present in the shipped release. See fatal error on update from 1.14 to 1.15 wp-crontrol#91 for the history.

Counter measures

• Need to increase my confidence in the reliability of the build that happens in the GitHub Actions workflows
• Need to run the automated acceptance tests against a version of the plugin which is built in exactly the same manner as when the plugin is built for the deployment to wordpress.org
• Need to be able to dry-run the build and deployment workflows and inspect the output

[Kilbourne]

Not yet tested but could the problem be there isn't

          composer run build-vendor
          git add -f vendor/autoload.php
          git add -f vendor/composer/*

in reusable-deploy-tag.yml ?

Because it seems that workflow installs also dev deps, which pollute the autoloader with not required files

[johnbillion]

I've identified the root cause as being the call to composer install which was added in johnbillion/plugin-infrastructure@13b30ba but I wasn't able to determine why 3.15 wasn't affected. This change was made prior to Query Monitor 3.15 being released. Alas the GitHub Actions run logs are only retained for 90 days so I've exhausted everything that I can investigate.

Improvements that have been made as a result:

• https://github.com/johnbillion/plugin-infrastructure/compare/cf00ac7..2119fb1
• https://github.com/johnbillion/query-monitor/compare/1d1fd13..dd1e988