Using environment variables for connection string to MySQL db (running in docker)?

[dev-samples]

Is it possible to read e.g. hostname, user and password for a connection to e.g. and external MySQL db?

I have created this data-sources.json file

{
	"folders": {},
	"connections": {
		"mysql8-18f4fad411e-67d520a52b28e036": {
			"provider": "mysql",
			"driver": "mysql8",
			"name": "MySQL@${mysql-host.name}:3306",
			"configuration": {
				"host":"${mysql-host.name}",
				"port": "33306",
				"url": "jdbc:mysql://${mysql-host.name}:3306/",
				"configurationType": "MANUAL",
				"type": "dev",
				"closeIdleConnection": true,
				"auth-model": "native"
			}
		}
	},
	"connection-types": {
		"dev": {
			"name": "Development",
			"color": "255,255,255",
			"description": "Regular development database",
			"auto-commit": true,
			"confirm-execute": false,
			"confirm-data-change": false,
			"smart-commit": false,
			"smart-commit-recover": false,
			"auto-close-transactions": true,
			"close-transactions-period": 1800,
			"auto-close-connections": true,
			"close-connections-period": 14400
		}
	}
}

but when I pass that to my docker run command it just gets ignored:

  docker run \
  --rm -it \
  -e CB_SERVER_NAME='cloudbeaver'\
  -e CB_ADMIN_NAME='administrator' \
  -e CB_ADMIN_PASSWORD='S0mePazzworD' \
  -e mysql-host.name='test-mysql' \
  -p 8080:8978 \
  --name dbbeaver \
  dbbeaver

I found some info here:
https://github.com/bobbyiliev/dbeaver/wiki/Admin-Manage-Connections#using-environment-variables

but its not really clear how to do this - e.g. if there are only a set of fields that support this?

[EvgeniaBzzz]

Hi @dev-samples
Yes, it is possible.
But using some environment variables is disabled by default.
You can enable it in .cloudbeaver.runtime

systemVariablesResolvingEnabled: "true"

We will update wiki documentation, thank you for bringing this to our attention! 🦫

[EvgeniaBzzz]

This article could be helpful for you Server configuration

[dev-samples]

@EvgeniaBzzz first thanks for getting back so quickly! ❤️

I did take a look at:
https://github.com/dbeaver/cloudbeaver/wiki/Server-configuration

but in the sample provided in the server.database scope I assume thats for the built-in db used by the dbeaver webapp and not for the external db connections (data-sources)?

At the bottom of that page I can the go to this link for data-sources:
https://github.com/dbeaver/cloudbeaver/wiki/Configuring-server-datasources

that describes the data-sources.json. This file I have tried to add my external MySQL connection to and it actually works when I start up a fresh container. But its the values (hostname etc.) in there that I want to set based on environments variables I can't seem to get that to work.

I have:

FROM dbeaver/cloudbeaver:24.0.4
RUN mkdir /opt/cloudbeaver/workspace/.data
COPY .cloudbeaver.runtime.conf /opt/cloudbeaver/workspace/.data/.cloudbeaver.runtime.conf
COPY data-sources.json /opt/cloudbeaver/workspace/GlobalConfiguration/.dbeaver/data-sources.json

Where I have updated .cloudbeaver.runtime.conf

app.systemVariablesResolvingEnabled: "${CLOUDBEAVER_SYSTEM_VARIABLES_RESOLVING_ENABLED:false}",

to:

app.systemVariablesResolvingEnabled: "${CLOUDBEAVER_SYSTEM_VARIABLES_RESOLVING_ENABLED:true}",

and then the docker run and data-sources.json as described in:
#2591 (comment)

What am I missing?

[EvgeniaBzzz]

Your config looks correct.
Did you set systemVariablesResolvingEnabled to true?
Also keep in mind that cloudbeaver.runtime.conf has a higher priority than cloudbeaver.conf.

[dev-samples]

Yes and when I start the app I do see the following values:

/opt/cloudbeaver/workspace/.data/.cloudbeaver.runtime.conf

"systemVariablesResolvingEnabled": true,

In:

root@4f5d975fcd14:/opt/cloudbeaver/conf# cat /opt/cloudbeaver/workspace/.data/.cloudbeaver.runtime.conf
{
  "server": {
    "serverName": "cloudbeaver",
    "expireSessionAfterPeriod": 604800000,
    "productSettings": {
      "core.localization.localization": "en",
      "core.theming.theme": "light",
      "plugin.log-viewer.disabled": false,
      "plugin.log-viewer.logBatchSize": 1000.0,
      "plugin.log-viewer.maxFailedRequests": 3.0,
      "plugin.log-viewer.maxLogRecords": 2000.0,
      "plugin.log-viewer.refreshTimeout": 3000.0,
      "plugin.sql-editor.autoSave": true,
      "plugin.sql-editor.disabled": false,
      "plugin.sql-editor.maxFileSize": 10240.0,
      "sql.proposals.insert.table.alias": "PLAIN"
    },
    "database": {
      "driver": "h2_embedded_v2",
      "url": "jdbc:h2:/opt/cloudbeaver/workspace/.data/cb.h2.dat",
      "initialDataConfiguration": "conf/initial-data.conf",
      "backupEnabled": "${CLOUDBEAVER_DB_BACKUP_ENABLED:true}",
      "pool": {
        "minIdleConnections": 4.0,
        "maxIdleConnections": 10.0,
        "maxConnections": 100.0,
        "validationQuery": "SELECT 1"
      }
    },
    "sm": {
      "passwordPolicy": {
        "minLength": "${CLOUDBEAVER_POLICY_MIN_LENGTH:8.0}",
        "minNumberCount": "${CLOUDBEAVER_POLICY_MIN_NUMBER_COUNT:1.0}",
        "minSymbolCount": "${CLOUDBEAVER_POLICY_MIN_SYMBOL_COUNT:0.0}",
        "requireMixedCase": "${CLOUDBEAVER_POLICY_REQUIRE_MIXED_CASE:true}"
      }
    }
  },
  "app": {
    "anonymousAccessEnabled": true,
    "supportsCustomConnections": false,
    "publicCredentialsSaveEnabled": true,
    "adminCredentialsSaveEnabled": true,
    "enableReverseProxyAuth": false,
    "forwardProxy": false,
    "linkExternalCredentialsWithUser": true,
    "redirectOnFederatedAuth": false,
    "resourceManagerEnabled": true,
    "showReadOnlyConnectionInfo": false,
    "grantConnectionsAccessToAnonymousTeam": false,
    "systemVariablesResolvingEnabled": true,
    "resourceQuotas": {
      "dataExportFileSizeLimit": 1.0E7,
      "resourceManagerFileSizeLimit": 500000.0,
      "sqlMaxRunningQueries": 100.0,
      "sqlResultSetRowsLimit": 100000.0,
      "sqlResultSetMemoryLimit": 2000000.0,
      "sqlTextPreviewMaxLength": 4096.0,
      "sqlBinaryPreviewMaxLength": 261120.0
    },
    "defaultNavigatorSettings": {},
    "enabledFeatures": [],
    "enabledAuthProviders": [
      "local"
    ],
    "enabledDrivers": [],
    "disabledDrivers": [
      "sqlite:sqlite_jdbc",
      "h2:h2_embedded",
      "h2:h2_embedded_v2",
      "clickhouse:yandex_clickhouse",
      "generic:duckdb_jdbc"
    ]
  }
}

As (expected?) I see the default values in:

/opt/cloudbeaver/conf/cloudbeaver.conf

root@4f5d975fcd14:/opt/cloudbeaver/conf# cat /opt/cloudbeaver/conf/cloudbeaver.conf
{
    server: {
        serverPort: 8978,

        workspaceLocation: "workspace",
        contentRoot: "web",
        driversLocation: "drivers",

        sslConfigurationPath:"${CLOUDBEAVER_SSL_CONF_PATH:workspace/.data/ssl-config.xml}",

        rootURI: "/",
        serviceURI: "/api/",

        productSettings: {
            # Global properties
            core.theming.theme: 'light',
            core.localization.localization: 'en',
            plugin.sql-editor.autoSave: true,
            plugin.sql-editor.disabled: false,
            # max size of the file that can be uploaded to the editor (in kilobytes)
            plugin.sql-editor.maxFileSize: 10240,
            plugin.log-viewer.disabled: false,
            plugin.log-viewer.logBatchSize: 1000,
            plugin.log-viewer.maxFailedRequests: 3,
            plugin.log-viewer.maxLogRecords: 2000,
            plugin.log-viewer.refreshTimeout: 3000,
            sql.proposals.insert.table.alias: PLAIN
        },

        expireSessionAfterPeriod: 1800000,

        develMode: false,

        enableSecurityManager: false,

        sm: {
            enableBruteForceProtection: "${CLOUDBEAVER_BRUTE_FORCE_PROTECTION_ENABLED:true}",
            maxFailedLogin: "${CLOUDBEAVER_MAX_FAILED_LOGINS:10}",
            minimumLoginTimeout: "${CLOUDBEAVER_MINIMUM_LOGIN_TIMEOUT:1}",
            blockLoginPeriod: "${CLOUDBEAVER_BLOCK_PERIOD:300}",
            passwordPolicy: {
                minLength: "${CLOUDBEAVER_POLICY_MIN_LENGTH:8}",
                requireMixedCase: "${CLOUDBEAVER_POLICY_REQUIRE_MIXED_CASE:true}",
                minNumberCount: "${CLOUDBEAVER_POLICY_MIN_NUMBER_COUNT:1}",
                minSymbolCount: "${CLOUDBEAVER_POLICY_MIN_SYMBOL_COUNT:0}"
            }
        },

        database: {
            driver: "h2_embedded_v2",
            url: "jdbc:h2:${workspace}/.data/cb.h2v2.dat",
            initialDataConfiguration: "conf/initial-data.conf",
            pool: {
                minIdleConnections: 4,
                maxIdleConnections: 10,
                maxConnections: 100,
                validationQuery: "SELECT 1"
            },
            backupEnabled: "${CLOUDBEAVER_DB_BACKUP_ENABLED:true}"
        }

    },
    app: {
        anonymousAccessEnabled: true,
        anonymousUserRole: "user",
        defaultUserTeam: "user",
        grantConnectionsAccessToAnonymousTeam: false,
        supportsCustomConnections: false,
        showReadOnlyConnectionInfo: false,
        systemVariablesResolvingEnabled: "${CLOUDBEAVER_SYSTEM_VARIABLES_RESOLVING_ENABLED:false}",

        forwardProxy: false,

        publicCredentialsSaveEnabled: true,
        adminCredentialsSaveEnabled: true,

        resourceManagerEnabled: true,

        resourceQuotas: {
            dataExportFileSizeLimit: 10000000,
            resourceManagerFileSizeLimit: 500000,
            sqlMaxRunningQueries: 100,
            sqlResultSetRowsLimit: 100000,
            sqlResultSetMemoryLimit: 2000000,
            sqlTextPreviewMaxLength: 4096,
            sqlBinaryPreviewMaxLength: 261120
        },
        enabledAuthProviders: [
            "local"
        ],

        disabledDrivers: [
            "sqlite:sqlite_jdbc",
            "h2:h2_embedded",
            "h2:h2_embedded_v2",
            "clickhouse:yandex_clickhouse"
        ]

    }

}

[EvgeniaBzzz]

Hi @dev-samples
cloudbeaver.conf affects all workspaces, cloudbeaver.runtime.conf affects only current workspace. Configs will be the same on the initial run and then you could adjust cloudbeaver.runtime.conf to your needs. So that's expected that you see default values in cloudbeaver.conf.

What do you see when you press Test button for your connection?

You will not see any changes in UI for this variable, it's expected result. But you should connect successfully if host value in docker run is valid.