Clouddriver 5.83.0 is unable to execute certain changesets against PostgreSQL

[Badbond]

Issue Summary:

Deploying Clouddriver 5.83.0 with a fresh PostgreSQL database fails due disallowed field usage for PostgreSQL in Liquibase changesets.

Cloud Provider(s):

Local (but generic, so should affect all environments).

Environment:

Clouddriver (5.83.0 and 5.84.0) with a clean PostgreSQL 16.2 database.
Orca (8.48.0) seems to start fine with a PostgreSQL database.
We are not (yet) running other Spinnaker services with PostgreSQL storages.

Feature Area:

PostgreSQL persistence for Clouddriver.

Description:

We have a testing setup for various internal plugins, where we attach the plugins in the Docker containers for various Spinnaker services. We noticed that as of Clouddriver 5.83.0 (5.82.2 was still fine), we are unable to run this test setup (which always starts with a clean state), as it can not execute all Liquibase changesets against our PostgreSQL 16.2 database.

It seems that as of Clouddriver 5.83.0 matches matches the timing of the Liquibase upgrade from 3.10.3 to 4.24.0: see spinnaker/kork#1117. As mentioned in the aforelinked pull request, for Orca some changes were made to make the changesets compatible with PostgreSQL by omitting afterColumn fields. It seems from the logs that this was not applied to Clouddriver's changesets.

Steps to Reproduce:

With the following Clouddriver SQL and Redis configuration:

sql:
  enabled: true
  read-only: false
  cache:
    enabled: true
  scheduler:
    enabled: false
  taskRepository:
    enabled: true
  unknown-agent-cleanup-agent:
    enabled: true
  connectionPools:
    default:
      default: true
      dialect: Postgres
      jdbcUrl: jdbc:postgresql://postgres:5432/clouddriver
      user: clouddriver_service
      password: <password>
  migration:
    user: clouddriver_service
    password: <password>
    jdbcUrl: jdbc:postgresql://postgres:5432/clouddriver
redis:
  enabled: true
  cache:
    enabled: false
  scheduler:
    enabled: true
  taskRepository:
    enabled: false

By running Clouddriver 5.83.0 against a clean database, the instance fails to start due to disallowed fields in the changesets.

Additional Details:

Clouddriver 5.83.0 logs

2024-04-11 11:04:23.111 ERROR 1 --- [           main] o.s.boot.SpringApplication               : Application run failed

org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'dockerRegistryImageLookupController': Unsatisfied dependency expressed through field 'cacheView'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'cacheView' defined in class path resource [com/netflix/spinnaker/clouddriver/cache/CacheConfig.class]: Unsatisfied dependency expressed through method 'cacheView' parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'catsModule' defined in class path resource [com/netflix/spinnaker/config/SqlCacheConfiguration.class]: Unsatisfied dependency expressed through method 'catsModule' parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'sqlAgentProvider' defined in class path resource [com/netflix/spinnaker/config/SqlCacheConfiguration.class]: Unsatisfied dependency expressed through method 'sqlAgentProvider' parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'sqlTableMetricsAgent' defined in class path resource [com/netflix/spinnaker/config/SqlCacheConfiguration.class]: Unsatisfied dependency expressed through method 'sqlTableMetricsAgent' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'liquibase' defined in class path resource [com/netflix/spinnaker/kork/sql/config/DefaultSqlConfiguration.class]: Invocation of init method failed; nested exception is liquibase.exception.CommandExecutionException: liquibase.exception.ValidationFailedException: Validation Failed:
     2 changes have validation failures
          addAfterColumn is not allowed on postgresql, db/changelog/20181120-cats.yml::application-index::afeldman
          addAfterColumn is not allowed on postgresql, db/changelog/20190913-task-sagaids.yml::add-task-sagaids-column::robzienert

[kirangodishala]

@Badbond - It seems to be the case that these two files missed the correction. I will work the fix.

[dbyron-sf]

clouddriver 5.83.0 was first present in spinnaker 1.33, so backporting the fix to both the 1.33 and 1.34 branches.