From 616c9c98b5813638381ca75724982edeeca76c0d Mon Sep 17 00:00:00 2001 From: fnaranjo Date: Tue, 25 Oct 2022 10:24:23 +0200 Subject: [PATCH] 183164582 Test mysql using containers (#717) * Experiment running system-tests for mysql inside containers [#183164582] Signed-off-by: Fernando Naranjo * Build database-backup-restore and fix echo commands [#183164582] Also replace some additional commands performed on BOSH VM * Ensure we invoke `bash -c` for every `go.Command` [#183164582] Running os commands using goexec is not identical to running those commands in a shell such as bash. Some examples of known things to require a bash shell: - Pipes - Redirects Signed-off-by: Fernando Naranjo * Create workflow to run MySQL tests on PRs [#183164582] Signed-off-by: Fernando Naranjo * Generate TLS certificates and shared them across containers [#183164582] Mysql service is the one in charge of creating the certificates. To make these generated certificates available to the container running the tests we use a shared volume named `mysql-certs` which we mount in both containers. * Support testing several MYSQL versions using GHAction matrix [#183164582] For this we need to pass the MYSQL version we want to test as a Docker build arg * Remove from matrix failing unsupported versions of MYSQL [#183164582] There are chances that the versions removed could also work with our current release but they need some packages not included by default in the Dockerimage. Doing the extra effort to make the pass is irrelevant to the BOSH release itself, since the filesystem is not identical to what will be included in the VM. We might need to investigate how to reduce this filesystem mismatch if possible. Signed-off-by: Fernando Naranjo * WIP try to improve some docs-type things [#183164582] Signed-off-by: Gareth Smith * Remove unneded folder intialization steps [#183164582] They seem to have inconsistent behaviour on different operating systems * Restore folder initialization and add it to system-mysql Dockerfile too [#183164582] * Print database logs in case of test failures [#183164582] * Reorganize VOLUME declarations to overcome a possible bug in Docker [#183164582] https://github.com/docker/compose/issues/3270#issuecomment-330169838 * Replace chmod with chown to better express the intent of the operation [#183164582] * Revert "Replace chmod with chown to better express the intent of the operation" This reverts commit e102885f8577d003573c31a67cde784545926d80. It seems `chmod root:root` counteracts `chmod mysql:mysql` causing a race condition between the two containers. It seems we are better off using `chmod 777` here. * Improve debugging experience by mounting the whole repo instead of a subfolder [#183164582] By doing this any error message in the logs would point to the exact path where the problem occurred instead of a docker-mounted path in an arbitrary location. This change pursues reducing the knowledge burden introduced by the use of Docker by trying to make the Docker integration as thin and transparent as possible. Ideally, not requiring any Docker knowledge to be able to work with the codebase. * Improve unit-tests debug experience by mounting full repo instead of subfolder [#183164582] By doing this any error message in the logs would point to the exact path where the problem occurred instead of a docker-mounted path in an arbitrary location. This change pursues reducing the knowledge burden introduced by the use of Docker by trying to make the Docker integration as thin and transparent as possible. Ideally, not requiring any Docker knowledge to be able to work with the codebase. * Refactor system-db-tests to use a similar approach to unit-tests [#183164582] Some goals of this refactor were: - Don't have an explicit ENTRYPOINT in Dockerfile, specify it in docker-compose - Move most of the boilerplate required by the tests to the tests folder - Remove hardcoded paths in boilerplate environment variables required by tests - Make scripts/run-system-db-tests-mysqsl more self-container and self-documenting - Flatten the path to start adding tests for other databases such as Postgres - Flatten the path to add some tests for testing the version detection logic * Simplify TestRunner.Dockerfile and `backup`/`restore` invocations [#183164582] Original tests made use of two auxiliary scripts `backup`/`restore` to set the required environment variables every time they were used. Since these variables are static and never change we added these vars to the `run-system-db-tests-mysql.bash` script itself and invoke the `database-backup-restore` binary directly instead from the tests. * Reduce certs handling in `Dockerfile` and `docker-compose.yml` [#183164582] By aglutinating the three previous variables each one pointing to a different certificate into a single variable pointing to a folder we should contain the three files. These files are expected to always be named the same so there isn't a need to specify the full path for each of them individually. * Fix errors introduced in latest commit [#183164582] Some of the environment variables which previously we thought were not needed explicitly by the tests were in fact required for mysql to successfully establish a connection using TLS certificates. * Add MySQL5.7 binary as it is used to determine version of db [#183164582] By looking at the source-code we determined that mysql5.7 binary is used to connect with the mysql database and ask for its version. Only later, the corresponding binary will be choosed and used. Signed-off-by: Fernando Naranjo * Add tests for different versions of MariaDB [#183164582] In this case we didn't copy the binary from the mariadb Docker image because it had unsatisfied dependencies (such as libssl3) * Trigger PR workflows * Show database logs on error to facilitate debugging [#183164582] * Workaround permission issues when specifying Volume in Dockerfile [#183164582] * Try triggering tests again [#183164582] * Use chmod 777 both in TestsRunner and BackingDB Dockerfiles [#183164582] * Fix mysql5.7-debian tests and make mysql tests more precise [#183164582] Since bbr-sdk-release always uses 5.7 at least once (to query the db version) we always need to specify MYSQL_CLIENT_5_7_PATH. Since we are using GitHub matrixes to tests several MySQL versions using the same scripts, it seems difficult to use the MYSQL_VERSION variable to download only the required binaries since we will need some conditional logic to populate the right ENV variables. For that reason this commit simply downloads the latest MySQL 8.0 and MySQL 5.7 binaries and populates the corresponding ENV variables every time, without caring which specific MYSQL_VERSION we are currently testing. * Add Postgres system tests [#183164582] * Fix typo in Postgres system tests workflow [#183164582] * Test compilation against diff stemcells and improve system tests [#183164582] To test compilation against diff stemcells we use multistage Docker containers to download bosh-lite warden tarballs and using the rootfs included in the tarball to create a container FROM scratch. Inside that dockerized stemcell we run the release packaging scripts which allows testing compilation against different stemcell using GHActions matrix strategies. Now that we are able to compile the release directly in GHActions, we can improve system tests by leveraging the binaries built on-the-fly from the blobs. This approach has huge benefits compared to the previous one: - Tests are more true to reality - Bosh blobs are fully tested (instead of testing just the GO code) - Compilation is being tested against different stemcells * Use `backup` and `restore` scripts from the /var/vcap/jobs folder [#183164582] Also, compiled database-backup-restorer using its packaging script instead of manually running golang during the `run-system-db-tests` scripts and refactored tests to use the binary located in /var/vcap/packages * Reduce verbosity of the logs [#183164582] * Add ubuntu-xenial tests except for MySQL 8.0 which is unsupported [#183164582] * Run postgres_tls and postgres_mutual_tls test suites in containers [#183164582] Signed-off-by: Fernando Naranjo * Fix Postgres TLS and MUTUAL_TLS test suites [#183164582] * Ensure system-db-postgres-backing-db database gets cleaned up after each job [#183164582] * Fix ENABLE_TLS var not being passed to docker-compose run commands [#183164582] All tests were running the basic Postgres tests instead of TLS and MUTUAL * Run dockerize-release as a separate step from the tests [#183164582] This change will help us differentiate how much time is spent in the actual tests versus compiling the release itself. This should serve as a first step towards optimizing builds * Prevent running Postgres TLS and MTLS tests for 9.4-alpine [#183164582] postgres:9.4-alpine doesn't have openssl package installed by default which causes the database to error before start and prevents tests from running. Testing Postgres 9.4 without TLS/MTLS should be enough for now to ensure this version is supported. Patching postgres Dockerfile to add openssl is possible but might no be needed or desirable since Postgres 9.4 reached EOGS in February 2020 * Disable 9.4-alpine for Mutual TLS not normal tests [#183164582] * Move dockerfiles from .github/actions to ci/ [#183164582] Signed-off-by: Diego Lemos * Restore tests and guard new BOSH-less functionality under a flag [#183164582] * Redirect stderr to stdout when running with flag RUN_TESTS_WITHOUT_BOSH [#183164582] The tests expected error messages to be available in stdout. Probably because when running the commands by SSHing into a BOSH VM the errors are retreived from Stdout Signed-off-by: Fernando Naranjo Signed-off-by: Gareth Smith Signed-off-by: Diego Lemos Co-authored-by: Gareth Smith Co-authored-by: Diego Lemos Co-authored-by: Cryogenics CI Bot --- .github/workflows/system-db-tests.yml | 106 +++++++++++++++ ci/dockerfiles/dockerize-release/Dockerfile | 97 ++++++++++++++ .../run-golang-unit-tests/Dockerfile | 3 +- ci/dockerfiles/run-system-db-tests/Dockerfile | 11 ++ .../run-system-db-tests/mariadb/Dockerfile | 9 ++ .../run-system-db-tests/mariadb/enable_tls.sh | 42 ++++++ .../run-system-db-tests/mysql/Dockerfile | 10 ++ .../mysql/enable_mysql_tls.sh | 17 +++ .../run-system-db-tests/postgres/Dockerfile | 9 ++ .../postgres/enable_tls.sh | 51 ++++++++ .../sdk-template-unit-tests/Dockerfile | 0 docker-compose.yml | 121 ++++++++++++++++-- .../scripts/run-system-db-tests-mariadb.bash | 30 +++++ .../scripts/run-system-db-tests-mysql.bash | 29 +++++ .../scripts/run-system-db-tests-postgres.bash | 78 +++++++++++ .../system_tests/mysql/mysql_suite_test.go | 10 +- .../postgresql/postgresql_system_test.go | 10 +- .../postgresql_mutual_tls_test.go | 10 +- .../postgresql_tls/postgresql_tls_test.go | 10 +- .../system_tests/utils/job_instance.go | 21 +-- 20 files changed, 638 insertions(+), 36 deletions(-) create mode 100644 .github/workflows/system-db-tests.yml create mode 100644 ci/dockerfiles/dockerize-release/Dockerfile rename {.github/actions => ci/dockerfiles}/run-golang-unit-tests/Dockerfile (52%) create mode 100644 ci/dockerfiles/run-system-db-tests/Dockerfile create mode 100644 ci/dockerfiles/run-system-db-tests/mariadb/Dockerfile create mode 100755 ci/dockerfiles/run-system-db-tests/mariadb/enable_tls.sh create mode 100644 ci/dockerfiles/run-system-db-tests/mysql/Dockerfile create mode 100755 ci/dockerfiles/run-system-db-tests/mysql/enable_mysql_tls.sh create mode 100644 ci/dockerfiles/run-system-db-tests/postgres/Dockerfile create mode 100755 ci/dockerfiles/run-system-db-tests/postgres/enable_tls.sh rename {.github/actions => ci/dockerfiles}/sdk-template-unit-tests/Dockerfile (100%) create mode 100755 src/database-backup-restore/scripts/run-system-db-tests-mariadb.bash create mode 100755 src/database-backup-restore/scripts/run-system-db-tests-mysql.bash create mode 100755 src/database-backup-restore/scripts/run-system-db-tests-postgres.bash diff --git a/.github/workflows/system-db-tests.yml b/.github/workflows/system-db-tests.yml new file mode 100644 index 000000000..dbe4b9ec7 --- /dev/null +++ b/.github/workflows/system-db-tests.yml @@ -0,0 +1,106 @@ +on: + pull_request: + branches: + - main + +jobs: + system-db-mariadb: + name: System DB Tests for MariaDB + runs-on: ubuntu-latest + strategy: + matrix: + stemcell-name: [ubuntu-bionic, ubuntu-jammy, ubuntu-xenial] + mariadb-version: [10.9-jammy, 10.7-focal, 10.5-focal, 10.2-bionic] + + steps: + - uses: actions/checkout@v3 + with: + path: repo + + - name: Dockerize BOSH Release (Compilation Test) + run: | + STEMCELL_NAME="${{ matrix.stemcell-name }}" \ + docker-compose -f repo/docker-compose.yml run dockerize-release + + - name: Run System Tests for MariaDB + run: | + STEMCELL_NAME="${{ matrix.stemcell-name }}" \ + MARIADB_VERSION="${{ matrix.mariadb-version }}" \ + MARIADB_PASSWORD="$(head /dev/urandom | md5sum | cut -f1 -d" ")" \ + docker-compose -f repo/docker-compose.yml run system-db-mariadb || (docker-compose -f repo/docker-compose.yml logs system-db-mariadb-backing-db && exit 1) + + system-db-mysql: + name: System DB Tests for MySQL + runs-on: ubuntu-latest + strategy: + matrix: + stemcell-name: [ubuntu-bionic, ubuntu-jammy, ubuntu-xenial] + mysql-version: [8.0-debian, 8.0-oracle, 5.7-debian] + exclude: + - stemcell-name: ubuntu-xenial + mysql-version: 8.0-debian + - stemcell-name: ubuntu-xenial + mysql-version: 8.0-oracle + + steps: + - uses: actions/checkout@v3 + with: + path: repo + + - name: Dockerize BOSH Release (Compilation Test) + run: | + STEMCELL_NAME="${{ matrix.stemcell-name }}" \ + docker-compose -f repo/docker-compose.yml run dockerize-release + + - name: Run System Tests for MySQL + run: | + STEMCELL_NAME="${{ matrix.stemcell-name }}" \ + MYSQL_VERSION="${{ matrix.mysql-version }}" \ + MYSQL_PASSWORD="$(head /dev/urandom | md5sum | cut -f1 -d" ")" \ + docker-compose -f repo/docker-compose.yml run system-db-mysql || (docker-compose -f repo/docker-compose.yml logs system-db-mysql-backing-db && exit 1) + + system-db-postgres: + name: System DB Tests for Postgres + runs-on: ubuntu-latest + strategy: + matrix: + stemcell-name: [ubuntu-bionic, ubuntu-jammy, ubuntu-xenial] + postgres-version: [13-bullseye, 11-bullseye, 10-bullseye, 9.6-bullseye, 9.4-alpine] + + steps: + - uses: actions/checkout@v3 + with: + path: repo + + - name: Dockerize BOSH Release (Compilation Test) + run: | + STEMCELL_NAME="${{ matrix.stemcell-name }}" \ + docker-compose -f repo/docker-compose.yml run dockerize-release + + - name: Run System Tests for Postgres + run: | + STEMCELL_NAME="${{ matrix.stemcell-name }}" \ + POSTGRES_VERSION="${{ matrix.postgres-version }}" \ + POSTGRES_PASSWORD="$(head /dev/urandom | md5sum | cut -f1 -d" ")" \ + docker-compose -f repo/docker-compose.yml run system-db-postgres || (docker-compose -f repo/docker-compose.yml logs system-db-postgres-backing-db && exit 1) + docker-compose -f repo/docker-compose.yml down -v --remove-orphans --rmi local + + - name: Run System Tests for Postgres with TLS + if: ${{ matrix.postgres-version != '9.4-alpine' }} + run: | + ENABLE_TLS="yes" \ + STEMCELL_NAME="${{ matrix.stemcell-name }}" \ + POSTGRES_VERSION="${{ matrix.postgres-version }}" \ + POSTGRES_PASSWORD="$(head /dev/urandom | md5sum | cut -f1 -d" ")" \ + docker-compose -f repo/docker-compose.yml run system-db-postgres || (docker-compose -f repo/docker-compose.yml logs system-db-postgres-backing-db && exit 1) + docker-compose -f repo/docker-compose.yml down -v --remove-orphans --rmi local + + - name: Run System Tests for Postgres with MUTUAL_TLS + if: ${{ matrix.postgres-version != '9.4-alpine' }} + run: | + ENABLE_TLS="mutual" \ + STEMCELL_NAME="${{ matrix.stemcell-name }}" \ + POSTGRES_VERSION="${{ matrix.postgres-version }}" \ + POSTGRES_PASSWORD="$(head /dev/urandom | md5sum | cut -f1 -d" ")" \ + docker-compose -f repo/docker-compose.yml run system-db-postgres || (docker-compose -f repo/docker-compose.yml logs system-db-postgres-backing-db && exit 1) + docker-compose -f repo/docker-compose.yml down -v --remove-orphans --rmi local diff --git a/ci/dockerfiles/dockerize-release/Dockerfile b/ci/dockerfiles/dockerize-release/Dockerfile new file mode 100644 index 000000000..3bb35719b --- /dev/null +++ b/ci/dockerfiles/dockerize-release/Dockerfile @@ -0,0 +1,97 @@ +######################################################################### +# Find, download and uncompress latest version of $STEMCELL_NAME from Bosh.io +######################################################################### +FROM alpine as uncompressed-stemcell +ARG STEMCELL_NAME +RUN apk add curl --no-cache +RUN STEMCELL_URL="$(curl -L https://bosh.io/stemcells | grep -io "https:\/\/.*warden-boshlite-${STEMCELL_NAME}-go_agent.tgz")" \ + && curl -o /stemcell.tgz -L ${STEMCELL_URL} +RUN mkdir -p /wrapper && tar xf /stemcell.tgz -C /wrapper +RUN mkdir -p /stemcell && tar xf /wrapper/image -C /stemcell + +######################################################################### +# Run bosh create-release and uncompress tarball which contains blobs and vendored packages +######################################################################### +FROM alpine as uncompressed-release +RUN apk add bash curl git ca-certificates --no-cache +RUN curl -o /bin/bosh -L "https://github.com/cloudfoundry/bosh-cli/releases/download/v7.0.1/bosh-cli-7.0.1-linux-amd64" && chmod +x /bin/bosh + +COPY . /release-repo +WORKDIR /release-repo + +RUN git submodule update --init --recursive +RUN bosh create-release --force --tarball /release.tgz +RUN mkdir -p /release && tar -xf /release.tgz -C /release + +WORKDIR /release/packages +RUN bash -c 'for tgz in *.tgz ; do \ + filename=$(basename -- "${tgz}") ; \ + ext="${filename##*.}" ; \ + noext="${filename%.*}" ; \ + \ + mkdir -p "${noext}" ; \ + tar -xf "${tgz}" -C "${noext}" ; \ + rm "${tgz}" ; \ + echo "${tgz} extracted!" ; \ +done' + +WORKDIR /release/jobs +RUN bash -c 'for tgz in *.tgz ; do \ + filename=$(basename -- "${tgz}") ; \ + ext="${filename##*.}" ; \ + noext="${filename%.*}" ; \ + \ + mkdir -p "${noext}" ; \ + tar -xf "${tgz}" -C "${noext}" ; \ + rm "${tgz}" ; \ + echo "${tgz} extracted!" ; \ +done' + +######################################################################### +# Mount alongside stemcell and uncompressed release files and run packaging scripts +######################################################################### +FROM scratch as compiled-release +COPY --from=uncompressed-stemcell /stemcell/ / +COPY --from=uncompressed-release /release /release + +ENV PACKAGES="libpcre2 \ +libopenssl1 \ +database-backup-restorer-mariadb \ +database-backup-restorer-boost \ +database-backup-restorer-mysql-5.6 \ +database-backup-restorer-mysql-5.7 \ +database-backup-restorer-mysql-8.0 \ +database-backup-restorer-postgres-9.4 \ +database-backup-restorer-postgres-9.6 \ +database-backup-restorer-postgres-10 \ +database-backup-restorer-postgres-10 \ +database-backup-restorer-postgres-11 \ +database-backup-restorer-postgres-13 \ +golang-1-linux \ +database-backup-restorer" + +WORKDIR /release/packages +RUN bash -c 'for pkg in ${PACKAGES}; do \ + export BOSH_INSTALL_TARGET="/var/vcap/packages/${pkg}" ; \ + cd "/release/packages/${pkg}" ; \ + mkdir -p "${BOSH_INSTALL_TARGET}" ; \ + chmod +x packaging ; \ + ./packaging ; \ +done' + + +WORKDIR /release/jobs +RUN mkdir -p /var/vcap/jobs/database-backup-restorer/bin \ + && cp database-backup-restorer/templates/backup /var/vcap/jobs/database-backup-restorer/bin/backup \ + && cp database-backup-restorer/templates/restore /var/vcap/jobs/database-backup-restorer/bin/restore \ + && chmod +x /var/vcap/jobs/database-backup-restorer/bin/backup \ + && chmod +x /var/vcap/jobs/database-backup-restorer/bin/restore + +######################################################################### +# Ready to run, clean, filesystem with the stemcell and compiled packages in place +######################################################################### +FROM scratch +COPY --from=uncompressed-stemcell /stemcell/ / +COPY --from=compiled-release /var/vcap/ /var/vcap/ +ENTRYPOINT ["echo", "Successfully dockerized release!"] + diff --git a/.github/actions/run-golang-unit-tests/Dockerfile b/ci/dockerfiles/run-golang-unit-tests/Dockerfile similarity index 52% rename from .github/actions/run-golang-unit-tests/Dockerfile rename to ci/dockerfiles/run-golang-unit-tests/Dockerfile index 0d10c6b16..805df9bb7 100644 --- a/.github/actions/run-golang-unit-tests/Dockerfile +++ b/ci/dockerfiles/run-golang-unit-tests/Dockerfile @@ -1,7 +1,6 @@ FROM golang:1.19.1 -VOLUME /goproject +VOLUME /backup-and-restore-sdk-release RUN go install github.com/onsi/ginkgo/ginkgo@latest -ENTRYPOINT /goproject/scripts/run-unit-tests.bash diff --git a/ci/dockerfiles/run-system-db-tests/Dockerfile b/ci/dockerfiles/run-system-db-tests/Dockerfile new file mode 100644 index 000000000..b2362ad23 --- /dev/null +++ b/ci/dockerfiles/run-system-db-tests/Dockerfile @@ -0,0 +1,11 @@ +FROM scratch +COPY --from=dockerize-release / / + +VOLUME /backup-and-restore-sdk-release + +RUN mkdir -p /tls-certs && chmod -R 777 /tls-certs +VOLUME /tls-certs +# https://boxboat.com/2017/01/23/volumes-and-dockerfiles-dont-mix/ + +ENV PATH=/var/vcap/packages/golang-1-linux/bin/:${PATH} +RUN GOBIN=/usr/local/bin/ go install github.com/onsi/ginkgo/ginkgo@latest diff --git a/ci/dockerfiles/run-system-db-tests/mariadb/Dockerfile b/ci/dockerfiles/run-system-db-tests/mariadb/Dockerfile new file mode 100644 index 000000000..1a5312fab --- /dev/null +++ b/ci/dockerfiles/run-system-db-tests/mariadb/Dockerfile @@ -0,0 +1,9 @@ +ARG MARIADB_VERSION +FROM mariadb:$MARIADB_VERSION + +RUN mkdir -p /tls-certs && chmod -R 777 /tls-certs +VOLUME /tls-certs +# https://boxboat.com/2017/01/23/volumes-and-dockerfiles-dont-mix/ + +RUN mkdir -p /etc/mysql/mariadb.conf.d/ && chown mysql: /etc/mysql/mariadb.conf.d/ +ADD enable_tls.sh /docker-entrypoint-initdb.d/ diff --git a/ci/dockerfiles/run-system-db-tests/mariadb/enable_tls.sh b/ci/dockerfiles/run-system-db-tests/mariadb/enable_tls.sh new file mode 100755 index 000000000..5fde33f9f --- /dev/null +++ b/ci/dockerfiles/run-system-db-tests/mariadb/enable_tls.sh @@ -0,0 +1,42 @@ +#!/usr/bin/env bash + +set -euo pipefail + +# https://portal2portal.blogspot.com/2021/09/openssl-get-your-subject-right.html +# Whatever method you use to generate the certificate and key files, the Common Name +# value used for the server and client certificates/keys must each differ from the +# Common Name value used for the CA certificate. +# Otherwise, the certificate and key files will not work for servers compiled using OpenSSL. + +openssl genrsa 2048 > /tls-certs/ca-key.pem +SUBJ="/C=DK/CN=CertificateAuthority/ST=SomeLand/L=SomeLocality/S=SomeProvince/O=SomeOrganization" +openssl req -new -x509 -nodes -days 3650 -key /tls-certs/ca-key.pem -subj "$SUBJ" > /tls-certs/ca-cert.pem + +SUBJ="/C=IE/CN=ServerCert/ST=SomeOtherLand/L=SomeOtherLocality/S=SomeOtherProvince/O=SomeOtherOrganization" +openssl req -newkey rsa:2048 -days 3650 -nodes -keyout /tls-certs/server-key.pem -subj "$SUBJ" > /tls-certs/server-req.pem +openssl x509 -req -in /tls-certs/server-req.pem -days 3650 -CA /tls-certs/ca-cert.pem -CAkey /tls-certs/ca-key.pem -set_serial 01 > /tls-certs/server-cert.pem +openssl rsa -in /tls-certs/server-key.pem -out /tls-certs/server-key.pem + +SUBJ="/C=CB/CN=ClientCert/ST=EvenOtherLand/L=EvenOtherLocality/S=EvenOtherProvince/O=EvenOtherOrganization" +openssl req -newkey rsa:2048 -days 3650 -nodes -keyout /tls-certs/client-key.pem -subj "$SUBJ" > /tls-certs/client-req.pem +openssl x509 -req -in /tls-certs/client-req.pem -days 3650 -CA /tls-certs/ca-cert.pem -CAkey /tls-certs/ca-key.pem -set_serial 01 > /tls-certs/client-cert.pem +openssl rsa -in /tls-certs/client-key.pem -out /tls-certs/client-key.pem + +# https://mariadb.com/kb/en/mariadb-ssl-connection-issues/ +# The core of the issue, you've used exactly the same information both for the client and the server certificate and OpenSSL doesn't like that +openssl verify -CAfile /tls-certs/ca-cert.pem /tls-certs/server-cert.pem /tls-certs/client-cert.pem + +mkdir -p /etc/mysql/mariadb.conf.d/ +cat << EOF > /etc/mysql/mariadb.conf.d/tls.cnf +[mysqld] +ssl-ca=/tls-certs/ca-cert.pem +ssl-cert=/tls-certs/server-cert.pem +ssl-key=/tls-certs/server-key.pem +bind-address= * + +[client] +ssl-cert=/tls-certs/client-cert.pem +ssl-key=/tls-certs/client-key.pem + +EOF + diff --git a/ci/dockerfiles/run-system-db-tests/mysql/Dockerfile b/ci/dockerfiles/run-system-db-tests/mysql/Dockerfile new file mode 100644 index 000000000..c17c0641c --- /dev/null +++ b/ci/dockerfiles/run-system-db-tests/mysql/Dockerfile @@ -0,0 +1,10 @@ +ARG MYSQL_VERSION +FROM mysql:$MYSQL_VERSION + +RUN mkdir -p /tls-certs && chmod -R 777 /tls-certs +VOLUME /tls-certs +# https://boxboat.com/2017/01/23/volumes-and-dockerfiles-dont-mix/ + +RUN mkdir -p /etc/mysql/mysql.conf.d/ && chown mysql: /etc/mysql/mysql.conf.d/ +ADD enable_mysql_tls.sh /docker-entrypoint-initdb.d/ + diff --git a/ci/dockerfiles/run-system-db-tests/mysql/enable_mysql_tls.sh b/ci/dockerfiles/run-system-db-tests/mysql/enable_mysql_tls.sh new file mode 100755 index 000000000..ea3ef06c1 --- /dev/null +++ b/ci/dockerfiles/run-system-db-tests/mysql/enable_mysql_tls.sh @@ -0,0 +1,17 @@ +#!/usr/bin/env bash + +set -euo pipefail + +mysql_ssl_rsa_setup +cp /var/lib/mysql/ca.pem /tls-certs/ +cp /var/lib/mysql/server-cert.pem /tls-certs/ +cp /var/lib/mysql/server-key.pem /tls-certs/ + +mkdir -p /etc/mysql/mysql.conf.d/ +cat << EOF > /etc/mysql/mysql.conf.d/ssl.cnf +[mysqld] +ssl-ca=/var/lib/mysql/ca.pem +ssl-cert=/var/lib/mysql/server-cert.pem +ssl-key=/var/lib/mysql/server-key.pem +EOF + diff --git a/ci/dockerfiles/run-system-db-tests/postgres/Dockerfile b/ci/dockerfiles/run-system-db-tests/postgres/Dockerfile new file mode 100644 index 000000000..9f2d8f9e6 --- /dev/null +++ b/ci/dockerfiles/run-system-db-tests/postgres/Dockerfile @@ -0,0 +1,9 @@ +ARG POSTGRES_VERSION +FROM postgres:$POSTGRES_VERSION + +RUN mkdir -p /tls-certs && chmod -R 777 /tls-certs +VOLUME /tls-certs +# https://boxboat.com/2017/01/23/volumes-and-dockerfiles-dont-mix/ + +ADD enable_tls.sh /docker-entrypoint-initdb.d/ +RUN touch /docker-entrypoint-initdb.d/enable-tls.sql && chmod 777 /docker-entrypoint-initdb.d/enable-tls.sql diff --git a/ci/dockerfiles/run-system-db-tests/postgres/enable_tls.sh b/ci/dockerfiles/run-system-db-tests/postgres/enable_tls.sh new file mode 100755 index 000000000..e325c3707 --- /dev/null +++ b/ci/dockerfiles/run-system-db-tests/postgres/enable_tls.sh @@ -0,0 +1,51 @@ +#!/usr/bin/env bash + +set -euo pipefail + +ENABLE_TLS="${ENABLE_TLS:-no}" + +if [[ "${ENABLE_TLS}" == "yes" || "${ENABLE_TLS}" = "mutual" ]]; then +# https://portal2portal.blogspot.com/2021/09/openssl-get-your-subject-right.html +# Whatever method you use to generate the certificate and key files, the Common Name +# value used for the server and client certificates/keys must each differ from the +# Common Name value used for the CA certificate. +# Otherwise, the certificate and key files will not work for servers compiled using OpenSSL. + +openssl genrsa 2048 > /tls-certs/ca-key.pem +SUBJ="/C=DK/CN=CertificateAuthority/ST=SomeLand/L=SomeLocality/S=SomeProvince/O=SomeOrganization" +openssl req -new -x509 -nodes -days 3650 -key /tls-certs/ca-key.pem -subj "$SUBJ" > /tls-certs/ca-cert.pem + +# Notice that we are using `system-db-postgres-backing-db` +# which is the same name we give to the Postgres backing db service +# this is needed for the plsql Mutual TLS to work correctly +SUBJ="/C=IE/CN=system-db-postgres-backing-db/ST=SomeOtherLand/L=SomeOtherLocality/S=SomeOtherProvince/O=SomeOtherOrganization" +openssl req -newkey rsa:2048 -days 3650 -nodes -keyout /tls-certs/server-key.pem -subj "$SUBJ" > /tls-certs/server-req.pem +openssl x509 -req -in /tls-certs/server-req.pem -days 3650 -CA /tls-certs/ca-cert.pem -CAkey /tls-certs/ca-key.pem -set_serial 01 > /tls-certs/server-cert.pem +openssl rsa -in /tls-certs/server-key.pem -out /tls-certs/server-key.pem + +SUBJ="/C=CB/CN=ClientCert/ST=EvenOtherLand/L=EvenOtherLocality/S=EvenOtherProvince/O=EvenOtherOrganization" +openssl req -newkey rsa:2048 -days 3650 -nodes -keyout /tls-certs/client-key.pem -subj "$SUBJ" > /tls-certs/client-req.pem +openssl x509 -req -in /tls-certs/client-req.pem -days 3650 -CA /tls-certs/ca-cert.pem -CAkey /tls-certs/ca-key.pem -set_serial 01 > /tls-certs/client-cert.pem +openssl rsa -in /tls-certs/client-key.pem -out /tls-certs/client-key.pem + +# https://mariadb.com/kb/en/mariadb-ssl-connection-issues/ +# The core of the issue, you've used exactly the same information both for the client and the server certificate and OpenSSL doesn't like that +openssl verify -CAfile /tls-certs/ca-cert.pem /tls-certs/server-cert.pem /tls-certs/client-cert.pem + +cat << 'EOF' > /docker-entrypoint-initdb.d/enable-tls.sql +ALTER SYSTEM SET ssl_ca_file TO '/tls-certs/ca-cert.pem'; +ALTER SYSTEM SET ssl_cert_file TO '/tls-certs/server-cert.pem'; +ALTER SYSTEM SET ssl_key_file TO '/tls-certs/server-key.pem'; +ALTER SYSTEM SET ssl TO 'ON'; +EOF +fi + +if [[ "${ENABLE_TLS}" == "yes" ]]; then + cat << 'EOF' > /var/lib/postgresql/data/pg_hba.conf +hostssl all all 0.0.0.0/0 md5 +EOF + elif [[ "${ENABLE_TLS}" == "mutual" ]]; then +cat << 'EOF' > /var/lib/postgresql/data/pg_hba.conf +hostssl all all 0.0.0.0/0 md5 clientcert=1 +EOF +fi diff --git a/.github/actions/sdk-template-unit-tests/Dockerfile b/ci/dockerfiles/sdk-template-unit-tests/Dockerfile similarity index 100% rename from .github/actions/sdk-template-unit-tests/Dockerfile rename to ci/dockerfiles/sdk-template-unit-tests/Dockerfile diff --git a/docker-compose.yml b/docker-compose.yml index ca98318f5..a3baec310 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -1,22 +1,123 @@ version: '3.0' services: + dockerize-release: + image: dockerize-release + build: + context: . + dockerfile: ci/dockerfiles/dockerize-release/Dockerfile + args: + - STEMCELL_NAME=${STEMCELL_NAME} + unit-blobstore-azure: - build: { context: .github/actions/run-golang-unit-tests } - volumes: [./src/azure-blobstore-backup-restore:/goproject] + build: { context: ci/dockerfiles/run-golang-unit-tests } + volumes: [./:/backup-and-restore-sdk-release] + entrypoint: /backup-and-restore-sdk-release/src/azure-blobstore-backup-restore/scripts/run-unit-tests.bash unit-blobstore-gcs: - build: { context: .github/actions/run-golang-unit-tests } - volumes: [./src/gcs-blobstore-backup-restore:/goproject] + build: { context: ci/dockerfiles/run-golang-unit-tests } + volumes: [./:/backup-and-restore-sdk-release] + entrypoint: /backup-and-restore-sdk-release/src/gcs-blobstore-backup-restore/scripts/run-unit-tests.bash unit-blobstore-s3: - build: { context: .github/actions/run-golang-unit-tests } - volumes: [./src/s3-blobstore-backup-restore:/goproject] + build: { context: ci/dockerfiles/run-golang-unit-tests } + volumes: [./:/backup-and-restore-sdk-release] + entrypoint: /backup-and-restore-sdk-release/src/s3-blobstore-backup-restore/scripts/run-unit-tests.bash unit-database: - build: { context: .github/actions/run-golang-unit-tests } - volumes: [./src/database-backup-restore:/goproject] + build: { context: ci/dockerfiles/run-golang-unit-tests } + volumes: [./:/backup-and-restore-sdk-release] + entrypoint: /backup-and-restore-sdk-release/src/database-backup-restore/scripts/run-unit-tests.bash unit-sdk-template: - build: { context: .github/actions/sdk-template-unit-tests } - volumes: [.:/backup-and-restore-sdk-release] + build: { context: ci/dockerfiles/sdk-template-unit-tests } + volumes: [./:/backup-and-restore-sdk-release] + + system-db-mariadb: + depends_on: [dockerize-release, system-db-mariadb-backing-db] + entrypoint: /backup-and-restore-sdk-release/src/database-backup-restore/scripts/run-system-db-tests-mariadb.bash + build: + context: ci/dockerfiles/run-system-db-tests/ + args: + - MARIADB_VERSION=${MARIADB_VERSION} + volumes: + - ./:/backup-and-restore-sdk-release + - mariadb-certs:/tls-certs + environment: + - MYSQL_USERNAME=root + - MYSQL_HOSTNAME=system-db-mariadb-backing-db + - MYSQL_PORT=3306 + - MYSQL_PASSWORD=${MARIADB_PASSWORD} + + system-db-mariadb-backing-db: + build: + context: ci/dockerfiles/run-system-db-tests/mariadb + args: + - MARIADB_VERSION=${MARIADB_VERSION} + volumes: + - mariadb-certs:/tls-certs + environment: + - MARIADB_ROOT_PASSWORD=${MARIADB_PASSWORD} + + system-db-mysql: + depends_on: [dockerize-release, system-db-mysql-backing-db] + entrypoint: /backup-and-restore-sdk-release/src/database-backup-restore/scripts/run-system-db-tests-mysql.bash + build: + context: ci/dockerfiles/run-system-db-tests/ + args: + - MYSQL_VERSION=${MYSQL_VERSION} + volumes: + - ./:/backup-and-restore-sdk-release + - mysql-certs:/tls-certs + environment: + - MYSQL_USERNAME=root + - MYSQL_HOSTNAME=system-db-mysql-backing-db + - MYSQL_PORT=3306 + - MYSQL_PASSWORD=${MYSQL_PASSWORD} + + system-db-mysql-backing-db: + build: + context: ci/dockerfiles/run-system-db-tests/mysql + args: + - MYSQL_VERSION=${MYSQL_VERSION} + volumes: + - mysql-certs:/tls-certs + environment: + - MYSQL_ROOT_PASSWORD=${MYSQL_PASSWORD} + + system-db-postgres: + depends_on: [dockerize-release, system-db-postgres-backing-db] + entrypoint: /backup-and-restore-sdk-release/src/database-backup-restore/scripts/run-system-db-tests-postgres.bash + build: + context: ci/dockerfiles/run-system-db-tests/ + args: + - POSTGRES_VERSION=${POSTGRES_VERSION} + volumes: + - ./:/backup-and-restore-sdk-release + - postgres-certs:/tls-certs + environment: + - ENABLE_TLS=${ENABLE_TLS} + - POSTGRES_USERNAME=postgres + - POSTGRES_HOSTNAME=system-db-postgres-backing-db + - POSTGRES_PORT=5432 + - POSTGRES_PASSWORD=${POSTGRES_PASSWORD} + + system-db-postgres-backing-db: + build: + context: ci/dockerfiles/run-system-db-tests/postgres + args: + - POSTGRES_VERSION=${POSTGRES_VERSION} + volumes: + - postgres-certs:/tls-certs + environment: + - ENABLE_TLS=${ENABLE_TLS} + - POSTGRES_PASSWORD=${POSTGRES_PASSWORD} + + +volumes: +# See https://docs.docker.com/compose/compose-file/#volumes - Specially: +### To reuse a volume across multiple services, +### a named volume MUST be declared in the top-level volumes key. + mysql-certs: + mariadb-certs: + postgres-certs: diff --git a/src/database-backup-restore/scripts/run-system-db-tests-mariadb.bash b/src/database-backup-restore/scripts/run-system-db-tests-mariadb.bash new file mode 100755 index 000000000..b0ecf2a2a --- /dev/null +++ b/src/database-backup-restore/scripts/run-system-db-tests-mariadb.bash @@ -0,0 +1,30 @@ +#!/usr/bin/env bash + +set -euo pipefail + +SRC_DIR="$(cd "$( dirname "$0" )/.." && pwd)" + +pushd "$SRC_DIR" + MYSQL_BINARY="/var/vcap/packages/database-backup-restorer-mysql-5.7/bin/mysql" + + for i in {1..5}; do + # Wait for the database to be ready + ${MYSQL_BINARY} -u ${MYSQL_USERNAME} -p${MYSQL_PASSWORD} -h ${MYSQL_HOSTNAME} -P ${MYSQL_PORT} -e 'SELECT "successfully connected to mysql"' && break || sleep 15 + done + + export MYSQL_CA_CERT_PATH="/tls-certs/ca-cert.pem" + export MYSQL_CLIENT_CERT_PATH="/tls-certs/client-cert.pem" + export MYSQL_CLIENT_KEY_PATH="/tls-certs/client-key.pem" + + export MYSQL_CA_CERT="$( cat "${MYSQL_CA_CERT_PATH}" )" + export MYSQL_CLIENT_CERT="$( cat "${MYSQL_CLIENT_CERT_PATH}" )" + export MYSQL_CLIENT_KEY="$( cat "${MYSQL_CLIENT_KEY_PATH}" )" + + export TEST_TLS=true + export TEST_TLS_MUTUAL_TLS=false + export TEST_TLS_VERIFY_IDENTITY=false + export TEST_SSL_USER_REQUIRES_SSL=true + + export RUN_TESTS_WITHOUT_BOSH=true + ginkgo -mod vendor -r -v "system_tests/mysql" -trace +popd diff --git a/src/database-backup-restore/scripts/run-system-db-tests-mysql.bash b/src/database-backup-restore/scripts/run-system-db-tests-mysql.bash new file mode 100755 index 000000000..dfccfcbc0 --- /dev/null +++ b/src/database-backup-restore/scripts/run-system-db-tests-mysql.bash @@ -0,0 +1,29 @@ +#!/usr/bin/env bash + +set -euo pipefail + +SRC_DIR="$(cd "$( dirname "$0" )/.." && pwd)" + +pushd "$SRC_DIR" + MYSQL_BINARY="/var/vcap/packages/database-backup-restorer-mysql-5.7/bin/mysql" + + for i in {1..5}; do + # Wait for the database to be ready + ${MYSQL_BINARY} -u ${MYSQL_USERNAME} -p${MYSQL_PASSWORD} -h ${MYSQL_HOSTNAME} -P ${MYSQL_PORT} -e 'SELECT "successfully connected to mysql"' && break || sleep 15 + done + + export MYSQL_CA_CERT_PATH="/tls-certs/ca.pem" + export MYSQL_CLIENT_CERT_PATH="/tls-certs/server-cert.pem" + export MYSQL_CLIENT_KEY_PATH="/tls-certs/server-key.pem" + + export MYSQL_CA_CERT="$( cat "${MYSQL_CA_CERT_PATH}" )" + export MYSQL_CLIENT_CERT="$( cat "${MYSQL_CLIENT_CERT_PATH}" )" + export MYSQL_CLIENT_KEY="$( cat "${MYSQL_CLIENT_KEY_PATH}" )" + + export TEST_TLS=true + export TEST_TLS_VERIFY_IDENTITY=false + export TEST_SSL_USER_REQUIRES_SSL=true + + export RUN_TESTS_WITHOUT_BOSH=true + ginkgo -mod vendor -r -v "system_tests/mysql" -trace +popd diff --git a/src/database-backup-restore/scripts/run-system-db-tests-postgres.bash b/src/database-backup-restore/scripts/run-system-db-tests-postgres.bash new file mode 100755 index 000000000..417086ec3 --- /dev/null +++ b/src/database-backup-restore/scripts/run-system-db-tests-postgres.bash @@ -0,0 +1,78 @@ +#!/usr/bin/env bash + +set -euo pipefail + +ENABLE_TLS="${ENABLE_TLS:-no}" + +SRC_DIR="$(cd "$( dirname "$0" )/.." && pwd)" + +pushd "$SRC_DIR" + PG_BINARY="/var/vcap/packages/database-backup-restorer-postgres-13/bin/psql" + + if [[ "$ENABLE_TLS" == "no" ]]; then + + for i in {1..5}; do + # Wait for the database to be ready + PGPASSWORD=${POSTGRES_PASSWORD} ${PG_BINARY} -U ${POSTGRES_USERNAME} -h ${POSTGRES_HOSTNAME} -p ${POSTGRES_PORT} -c "SELECT CAST('successfully connected' AS text) AS healthcheck" && break || sleep 15 + done + + + export TEST_TLS=true + export TEST_TLS_VERIFY_IDENTITY=false + export TEST_SSL_USER_REQUIRES_SSL=true + + export RUN_TESTS_WITHOUT_BOSH=true + ginkgo -mod vendor -r -v "system_tests/postgresql" -trace + + elif [[ "$ENABLE_TLS" == "yes" ]]; then + + export TEST_TLS=true + export TEST_TLS_VERIFY_IDENTITY=false + export TEST_SSL_USER_REQUIRES_SSL=false + + export POSTGRES_CA_CERT_PATH="/tls-certs/ca-cert.pem" + export POSTGRES_CLIENT_CERT_PATH="/tls-certs/client-cert.pem" + export POSTGRES_CLIENT_KEY_PATH="/tls-certs/client-key.pem" + + for i in {1..5}; do + # Wait for the database to be ready + PGPASSWORD=${POSTGRES_PASSWORD} ${PG_BINARY} -U ${POSTGRES_USERNAME} -h ${POSTGRES_HOSTNAME} -p ${POSTGRES_PORT} -c "SELECT CAST('successfully connected' AS text) AS healthcheck" && break || sleep 15 + done + + export POSTGRES_CA_CERT="$( cat "${POSTGRES_CA_CERT_PATH}" )" + export POSTGRES_CLIENT_CERT="$( cat "${POSTGRES_CLIENT_CERT_PATH}" )" + export POSTGRES_CLIENT_KEY="$( cat "${POSTGRES_CLIENT_KEY_PATH}" )" + + export RUN_TESTS_WITHOUT_BOSH=true + ginkgo -mod vendor -r -v "system_tests/postgresql_tls" -trace + + elif [[ "${ENABLE_TLS}" == "mutual" ]]; then + + export TEST_TLS=true + export TEST_TLS_VERIFY_IDENTITY=false + export TEST_SSL_USER_REQUIRES_SSL=false + + export POSTGRES_CA_CERT_PATH="/tls-certs/ca-cert.pem" + export POSTGRES_CLIENT_CERT_PATH="/tls-certs/client-cert.pem" + export POSTGRES_CLIENT_KEY_PATH="/tls-certs/client-key.pem" + + for i in {1..5}; do + # Wait for the database to be ready + PGSSLMODE="verify-full" \ + PGREQUIRESSL=1 \ + PGPASSWORD="${POSTGRES_PASSWORD}" \ + PGSSLROOTCERT="${POSTGRES_CA_CERT_PATH}" \ + PGSSLKEY="${POSTGRES_CLIENT_KEY_PATH}" \ + PGSSLCERT="${POSTGRES_CLIENT_CERT_PATH}" \ + ${PG_BINARY} -U ${POSTGRES_USERNAME} -h ${POSTGRES_HOSTNAME} -p ${POSTGRES_PORT} -c "SELECT CAST('successfully connected' AS text) AS healthcheck" && break || sleep 15 + done + + export POSTGRES_CA_CERT="$( cat "${POSTGRES_CA_CERT_PATH}" )" + export POSTGRES_CLIENT_CERT="$( cat "${POSTGRES_CLIENT_CERT_PATH}" )" + export POSTGRES_CLIENT_KEY="$( cat "${POSTGRES_CLIENT_KEY_PATH}" )" + + export RUN_TESTS_WITHOUT_BOSH=true + ginkgo -mod vendor -r -v "system_tests/postgresql_mutual_tls" -trace + + fi +popd diff --git a/src/database-backup-restore/system_tests/mysql/mysql_suite_test.go b/src/database-backup-restore/system_tests/mysql/mysql_suite_test.go index 3ea98e7a5..c6e7877cc 100644 --- a/src/database-backup-restore/system_tests/mysql/mysql_suite_test.go +++ b/src/database-backup-restore/system_tests/mysql/mysql_suite_test.go @@ -38,10 +38,12 @@ func TestMysql(t *testing.T) { var _ = Describe("mysql", func() { BeforeSuite(func() { - brJob = JobInstance{ - Deployment: MustHaveEnv("SDK_DEPLOYMENT"), - Instance: MustHaveEnv("SDK_INSTANCE_GROUP"), - InstanceIndex: "0", + if os.Getenv("RUN_TESTS_WITHOUT_BOSH") != "true" { + brJob = JobInstance{ + Deployment: MustHaveEnv("SDK_DEPLOYMENT"), + Instance: MustHaveEnv("SDK_INSTANCE_GROUP"), + InstanceIndex: "0", + } } mysqlHostName = MustHaveEnv("MYSQL_HOSTNAME") diff --git a/src/database-backup-restore/system_tests/postgresql/postgresql_system_test.go b/src/database-backup-restore/system_tests/postgresql/postgresql_system_test.go index eaf53f14c..a96f5b9c5 100644 --- a/src/database-backup-restore/system_tests/postgresql/postgresql_system_test.go +++ b/src/database-backup-restore/system_tests/postgresql/postgresql_system_test.go @@ -48,10 +48,12 @@ var _ = Describe("postgres", func() { ) BeforeSuite(func() { - brJob = JobInstance{ - Deployment: MustHaveEnv("SDK_DEPLOYMENT"), - Instance: MustHaveEnv("SDK_INSTANCE_GROUP"), - InstanceIndex: "0", + if os.Getenv("RUN_TESTS_WITHOUT_BOSH") != "true" { + brJob = JobInstance{ + Deployment: MustHaveEnv("SDK_DEPLOYMENT"), + Instance: MustHaveEnv("SDK_INSTANCE_GROUP"), + InstanceIndex: "0", + } } postgresHostName = MustHaveEnv("POSTGRES_HOSTNAME") diff --git a/src/database-backup-restore/system_tests/postgresql_mutual_tls/postgresql_mutual_tls_test.go b/src/database-backup-restore/system_tests/postgresql_mutual_tls/postgresql_mutual_tls_test.go index e2141fd04..2e83d982f 100644 --- a/src/database-backup-restore/system_tests/postgresql_mutual_tls/postgresql_mutual_tls_test.go +++ b/src/database-backup-restore/system_tests/postgresql_mutual_tls/postgresql_mutual_tls_test.go @@ -37,10 +37,12 @@ var _ = Describe("postgres with mutual tls", func() { ) BeforeSuite(func() { - brJob = JobInstance{ - Deployment: MustHaveEnv("SDK_DEPLOYMENT"), - Instance: MustHaveEnv("SDK_INSTANCE_GROUP"), - InstanceIndex: "0", + if os.Getenv("RUN_TESTS_WITHOUT_BOSH") != "true" { + brJob = JobInstance{ + Deployment: MustHaveEnv("SDK_DEPLOYMENT"), + Instance: MustHaveEnv("SDK_INSTANCE_GROUP"), + InstanceIndex: "0", + } } postgresHostName = MustHaveEnv("POSTGRES_HOSTNAME") diff --git a/src/database-backup-restore/system_tests/postgresql_tls/postgresql_tls_test.go b/src/database-backup-restore/system_tests/postgresql_tls/postgresql_tls_test.go index dd2b6ff19..481fc4b00 100644 --- a/src/database-backup-restore/system_tests/postgresql_tls/postgresql_tls_test.go +++ b/src/database-backup-restore/system_tests/postgresql_tls/postgresql_tls_test.go @@ -32,10 +32,12 @@ var _ = Describe("postgres with tls", func() { ) BeforeSuite(func() { - brJob = JobInstance{ - Deployment: MustHaveEnv("SDK_DEPLOYMENT"), - Instance: MustHaveEnv("SDK_INSTANCE_GROUP"), - InstanceIndex: "0", + if os.Getenv("RUN_TESTS_WITHOUT_BOSH") != "true" { + brJob = JobInstance{ + Deployment: MustHaveEnv("SDK_DEPLOYMENT"), + Instance: MustHaveEnv("SDK_INSTANCE_GROUP"), + InstanceIndex: "0", + } } postgresHostName = MustHaveEnv("POSTGRES_HOSTNAME") diff --git a/src/database-backup-restore/system_tests/utils/job_instance.go b/src/database-backup-restore/system_tests/utils/job_instance.go index 8f1be4569..838a44e38 100644 --- a/src/database-backup-restore/system_tests/utils/job_instance.go +++ b/src/database-backup-restore/system_tests/utils/job_instance.go @@ -19,6 +19,7 @@ package utils import ( "encoding/json" "fmt" + "os" . "github.com/onsi/ginkgo" . "github.com/onsi/gomega" @@ -59,14 +60,18 @@ func (jobInstance *JobInstance) RunOnVMAndSucceed(command string) *gexec.Session } func (jobInstance *JobInstance) RunOnInstance(cmd ...string) *gexec.Session { - return RunCommand( - join( - BoshCommand(), - forDeployment(jobInstance.Deployment), - getSSHCommand(jobInstance.Instance, jobInstance.InstanceIndex), - ), - join(cmd...), - ) + if os.Getenv("RUN_TESTS_WITHOUT_BOSH") == "true" { + return RunCommandWithStream(nil, nil, "bash", "-c", join(join(cmd...), "2>&1")) + } else { + return RunCommand( + join( + BoshCommand(), + forDeployment(jobInstance.Deployment), + getSSHCommand(jobInstance.Instance, jobInstance.InstanceIndex), + ), + join(cmd...), + ) + } } func (jobInstance *JobInstance) GetIPOfInstance() string {