Magento 2 Recipe: Cache prefix generated different cache prefixes when deploying to multiple hosts

[dverkade]

Situation:

We have Magento 2 projects which are load balanced between different servers. For instance 2 frontend web servers for visitors and a seperate backend service for the Magento backoffice. All servers point to the same Redis instance which is located on the database server. We're deploying with the configuration:

after: deploy:shared: magento:set_cache_prefix deploy:magento: magento:cleanup_cache_prefix

Actual result:

The set cache prefix is different per host because it will take the alias and release number. So for one server it will be:

production_web35_12_

And on the other server the prefix is set to:

production_web40_9_

Issue:

The issue is that cache will not be cleared correctly. The prefix should be the same for all 3 webservers in order for 1 webserver to clear the cache for the other two as well.

Expected result:

The generated cache key should be the same for all hosts within the same "stage" and release. So if we deploy to "stage" production to 3 webservers, the generated key should be the same for all 3 servers for Magento to work correctly.

[dverkade]

@valguss Hope you have any suggestions how we could resolve this.

[valguss]

For the moment i'd suggest disabling using the cache_prefixes
Is there a way for you to sync up the releases, so they are the same number?

[dverkade]

@valguss, yes, I can get the release in sync quite easily. However I'm stuck with the first / "alias" part of the key.

[valguss]

You could override the task in your deployer.php with something like the following (not ideal I know):

desc('Update cache id_prefix');
task('magento:set_cache_prefix', function () {
    //download current env config
    $tmpConfigFile = tempnam(sys_get_temp_dir(), 'deployer_config');
    download('{{deploy_path}}/shared/' . ENV_CONFIG_FILE_PATH, $tmpConfigFile);
    $envConfigArray = include($tmpConfigFile);
    //set prefix to `alias_releasename_`
    $prefixUpdate = 'abc_' . get('release_name') . '_';

    //check for preload keys and update
    if (isset($envConfigArray['cache']['frontend']['default']['backend_options']['preload_keys'])) {
        $oldPrefix = $envConfigArray['cache']['frontend']['default']['id_prefix'];
        $preloadKeys = $envConfigArray['cache']['frontend']['default']['backend_options']['preload_keys'];
        $newPreloadKeys = [];
        foreach ($preloadKeys as $preloadKey) {
            $newPreloadKeys[] = preg_replace('/^' . $oldPrefix . '/', $prefixUpdate, $preloadKey);
        }
        $envConfigArray['cache']['frontend']['default']['backend_options']['preload_keys'] = $newPreloadKeys;
    }

    //update id_prefix to include release name
    $envConfigArray['cache']['frontend']['default']['id_prefix'] = $prefixUpdate;
    $envConfigArray['cache']['frontend']['page_cache']['id_prefix'] = $prefixUpdate;

    //Generate configuration array as string
    $envConfigStr = '<?php return ' . var_export($envConfigArray, true) . ';';
    file_put_contents($tmpConfigFile, $envConfigStr);
    //upload updated config to server
    upload($tmpConfigFile, '{{deploy_path}}/shared/' . TMP_ENV_CONFIG_FILE_PATH);
    //cleanup tmp file
    unlink($tmpConfigFile);
    //delete the symlink for env.php
    run('rm {{release_or_current_path}}/' . ENV_CONFIG_FILE_PATH);
    //link the env to the tmp version
    run('{{bin/symlink}} {{deploy_path}}/shared/' . TMP_ENV_CONFIG_FILE_PATH . ' {{release_path}}/' . ENV_CONFIG_FILE_PATH);
});

I've just updated

$prefixUpdate = get('alias') . '_' . get('release_name') . '_';

to

$prefixUpdate = 'abc_' . get('release_name') . '_';

[dverkade]

@valguss, thanks. We can do a workaround but would like to see this fixed in the official recipe as well and would like to contribute to a solution.

Best practice is to have the same redis DB per environment. So in theory we could drop the "get('alias')" part of the prefix all together. Or we could use the "stage" of the host, so that the first part of the prefix is the same for all hosts in that stage. Can you just do get('stage') in order to get the host stage?

[valguss]

Using stage sounds like a better solution. The premise I was going for was for our specific setup where staging sites sit on the same server as the live env, so was trying to make the key unique to each environment so that they didn't clash. Will get a PR up shortly to change it though.

Thanks

[dverkade]

Using stage sounds like a better solution. The premise I was going for was for our specific setup where staging sites sit on the same server as the live env, so was trying to make the key unique to each environment so that they didn't clash. Will get a PR up shortly to change it though.

Thanks

Stage sounds good to me. I get your use case and with using "stage" this still works. I do think it's better to use a different Redis database number for a test environment and a live environment when they are both on the same server. Something like this:

            'page_cache' => [
                'backend' => 'Cm_Cache_Backend_Redis',
                'backend_options' => [
                    'server' => '127.0.0.1',
                    'port' => '6379',
                    'database' => '1',  ## HAVE A DIFFERENT DB NUMBER HERE FOR TEST OR PRODUCTION ENVIRONMENTS
                    'compress_data' => '0'
                ]
            ]

[valguss]

Now updated to check if stage is available and fall back to alias