Skip to content

Config JSON

thoniTUB edited this page May 16, 2024 · 14 revisions

Config JSON

This is an automatically created documentation. It is not 100% accurate since the generator does not handle every edge case.

Instead of a list ConQuery also always accepts a single element.

The config.json is required for every type of execution. Its root element is a ConqueryConfig object.


Base AuthorizationConfig

An AuthorizationConfig defines the initial users that are created on application startup and other permission related options.

Different types of AuthorizationConfig can be used by setting type to one of the following values:

DEFAULT

Details

Java Type: com.bakdata.conquery.models.config.auth.DefaultAuthorizationConfig

Supported Fields:

Field Type Default Example Description
initialRoles list of ProtoRole
initialUsers list of ProtoUser []
overviewScope list of String ["datasets","admin","*"]

DEVELOPMENT

Details

Java Type: com.bakdata.conquery.models.config.auth.DevelopmentAuthorizationConfig

No fields can be set for this type.


Base AuthenticationRealmFactory

An AuthenticationConfig is used to define how specific realms for authentication are configured.

Different types of AuthenticationRealmFactory can be used by setting type to one of the following values:

DEVELOPMENT

Default configuration for the auth system. Sets up all other default components. This configuration causes that every request is handled as invoked by the super user.

Details

Java Type: com.bakdata.conquery.models.auth.develop.DevAuthConfig

No fields can be set for this type.

JWT_PKCE_REALM

Details

Java Type: com.bakdata.conquery.models.config.auth.JwtPkceVerifyingRealmFactory

Supported Fields:

Field Type Default Example Description
additionalTokenChecks list of String []
alternativeIdClaims list of String []
client String null
idpConfiguration IdpConfiguration null
tokenLeeway @jakarta.validation.constraints.Min(0) int 60
wellKnownEndpoint URI null

LOCAL_AUTHENTICATION

Details

Java Type: com.bakdata.conquery.models.config.auth.LocalAuthenticationConfig

Supported Fields:

Field Type Default Example Description
directory File "./storage"
jwtDuration @MinDuration(value=1, unit=TimeUnit.MINUTES) Duration "12 hours"
passwordStoreConfig XodusConfig Configuration for the password store. An encryption for the store itself might be set here.
storeName String "authenticationStore" The name of the folder the store lives in.

OIDC_AUTHORIZATION_CODE_FLOW

Factory for a simple realm that just forwards tokens to the IDP for verification.

Details

Java Type: com.bakdata.conquery.models.config.auth.OIDCAuthorizationCodeFlowRealmFactory

Supported Fields:

Field Type Default Example Description
client IntrospectionDelegatingRealmFactory null

OIDC_RESOURCE_OWNER_PASSWORD_CREDENTIAL_AUTHENTICATION

Realm that supports the Open ID Connect Resource-Owner-Password-Credential-Flow with a Keycloak IdP.

Details

Java Type: com.bakdata.conquery.models.config.auth.OIDCResourceOwnerPasswordCredentialRealmFactory

Supported Fields:

Field Type Default Example Description
client IntrospectionDelegatingRealmFactory null

Base PluginConfig

A PluginConfig is used to define settings for Conquery plugins.

Different types of PluginConfig can be used by setting type to one of the following values:

FORM_BACKEND

{@link PluginConfig} for an external form backend. The external form backend must implement the OpenAPI spec for external form backend.

Details

Java Type: com.bakdata.conquery.models.config.FormBackendConfig

Supported Fields:

Field Type Default Example Description
authentication @Valid @NotNull AuthenticationClientFilterProvider null
baseURI @NotNull URI null
cancelTaskPath String
conqueryApiUrl @NotNull URL null
formConfigPath String "form-config"
healthCheckPath String "health"
id String null
postFormPath String "task"
statusTemplatePath String
versionPath String "version"

MANUAL

Configuration of manual links for {@link org.checkerframework.checker.signature.qual.InternalForm} and overriding of other {@link com.bakdata.conquery.apiv1.forms.Form}s.

Details

Java Type: com.bakdata.conquery.models.config.ManualConfig

Supported Fields:

Field Type Default Example Description
forms map from String to @ManualURI URI Maps a {@link CPSType} of a {@link com.bakdata.conquery.apiv1.forms.Form} to a manual URL. E.g.:

Other Types

Type APIConfig

Details

Java Type: com.bakdata.conquery.models.config.APIConfig

Supported Fields:

Field Type Default Example Description
allowCORSRequests boolean false
caching boolean true

Type CSVConfig

Holds the necessary information to configure CSV parsers and writers.

Details

Java Type: com.bakdata.conquery.models.config.CSVConfig

Supported Fields:

Field Type Default Example Description
comment char "\u0000"
delimeter char ","
encoding @NotNull Charset "UTF-8"
escape char "\\"
lineSeparator String "\n"
maxColumns int 1000000
parseHeaders boolean true
quote char "\""
skipHeader boolean false

Type ClusterConfig

Details

Java Type: com.bakdata.conquery.models.config.ClusterConfig

Supported Fields:

Field Type Default Example Description
backpressure @jakarta.validation.constraints.Min(0) int 1500 Amount of backpressure before jobs can volunteer to block to send messages to their shards.

Mostly {@link com.bakdata.conquery.models.jobs.ImportJob} is interested in this. Note that an average import should create more than #Entities / {@linkplain #entityBucketSize} jobs (via {@link com.bakdata.conquery.models.jobs.CalculateCBlocksJob}) in short succession, which will cause it to sleep. This field helps alleviate memory pressure on the Shards by slowing down the Manager, should it be sending too fast.

entityBucketSize @jakarta.validation.constraints.Min(1) int 1000
idleTimeOut Duration "5 minutes"
managerURL @Valid @NotNull InetAddress "localhost"
mina MinaConfig
port @io.dropwizard.validation.PortRange int 16170

Type ConqueryConfig

Details

Java Type: com.bakdata.conquery.models.config.ConqueryConfig

Supported Fields:

Field Type Default Example Description
api APIConfig
authentication @Valid @NotNull AuthenticationConfig
authenticationRealms list of AuthenticationRealmFactory
authorizationRealms @Valid @NotNull AuthorizationConfig
cluster ClusterConfig
csv CSVConfig
debugMode boolean or null null
failOnError boolean false
frontend FrontendConfig
idColumns @Valid @NotNull IdColumnConfig
index @NotNull @Valid IndexConfig
jerseyClient @Valid @NotNull JerseyClientConfiguration
locale LocaleConfig
metricsConfig ConqueryMetricsConfig
plugins list of PluginConfig []
preprocessor PreprocessingConfig
queries QueryConfig
resultProviders list of ResultRendererProvider
sqlConnectorConfig @Valid @NotNull SqlConnectorConfig
standalone StandaloneConfig
storage @Valid @NotNull StoreFactory

Type CurrencyConfig

Details

Java Type: com.bakdata.conquery.models.config.FrontendConfig$CurrencyConfig

Supported Fields:

Field Type Default Example Description
decimalScale int 2
decimalSeparator String ","
thousandSeparator String "."
unit String "€"

Type FrontendConfig

Details

Java Type: com.bakdata.conquery.models.config.FrontendConfig

Supported Fields:

Field Type Default Example Description
alwaysAllowCreateValue boolean false If true, users are always allowed to add custom values into SelectFilter input fields.
contactEmail String null
currency CurrencyConfig
manualUrl URL null The url that points a manual. This is also used by the {@link FormScanner} as the base url for forms that specify a relative url. Internally {@link URI#resolve(URI)} is used to concatenate this base url and the manual url from the form. An final slash ('{@code /}') on the base url has the following effect:
observationPeriodYears @jakarta.validation.constraints.Min(0) int 6 Years to include in entity preview.
visualisationPercentiles Range<Integer>
visualisationsHistogramLimit @jakarta.validation.constraints.Min(0) int 10 Limit to number of histogram entries. Note, that zero and out of bounds values are tracked in separate bins, so you can have three additional bins.

Type LocaleConfig

Details

Java Type: com.bakdata.conquery.models.config.LocaleConfig

Supported Fields:

Field Type Default Example Description
dateFormatMapping map from Locale to String Mappings from user provided locale to date format which is used in the generation of result tables. The formats are also available for parsing dates using the {@link DateReader}. However, the locale is neglected there and the formats are tried until one that fits is found.
frontend @NotNull Locale "und"
listFormats list of ListFormat List formats that are available for parsing inputs and (the first one) for rendering result tables. Spaces at the ends of the separator are only relevant for the output of results. For the input (parsing) the separator string can be surrounded by an arbitrary number of spaces.
localeRangeStartEndSeparators map from Locale to String Mappings from user provided locale to date range format which is used in the generation of result tables. The formats are also available for parsing dates ranges using the {@link DateReader}. However, the locale is neglected there and the formats are tried until one that fits is found.
parsingDateFormats @NotNull Set<String> ["yyyyMMdd"] Additional date formats that are available only for parsing.
parsingRangeStartEndSeparators Set<String> ["/"] Additional date range formats that are available only for parsing.

Type MinaConfig

Details

Java Type: com.bakdata.conquery.models.config.MinaConfig

Supported Fields:

Field Type Default Example Description
bothIdleTime int 300 The delay in seconds before we notify a session that it has been idle on read and write. Default to infinite
maxReadBufferSize int 500000000 The maximum size of the buffer used to read incoming data
minReadBufferSize int 64 The minimum size of the buffer used to read incoming data
readBufferSize int 8192 The default size of the buffer used to read incoming data
readerIdleTime int 300 The delay in seconds before we notify a session that it has been idle on read. Default to infinite
throughputCalculationInterval int 3
useReadOperation boolean false A flag set to true when we allow the application to do a session.read(). Default to false
writeTimeout int 0 The delay to wait for a write operation to complete before bailing out
writerIdleTime int 300 The delay in seconds before we notify a session that it has been idle on write. Default to infinite

Type PreprocessingConfig

Details

Java Type: com.bakdata.conquery.models.config.PreprocessingConfig

Supported Fields:

Field Type Default Example Description
faultyLineThreshold @jakarta.validation.constraints.Min(0) @jakarta.validation.constraints.Max(1) double 0.01
maximumPrintedErrors @jakarta.validation.constraints.Min(0) int 10
nThreads @jakarta.validation.constraints.Min(1) int
parsers ParserConfig

Type QueryConfig

Details

Java Type: com.bakdata.conquery.models.config.QueryConfig

Supported Fields:

Field Type Default Example Description
executionPool ThreadPoolDefinition
oldQueriesTime Duration "30 days"
secondaryIdSubPlanRetention int 15 Limits how many subQuery-Plans should be cached between executions: This number limits how many sub-plans are cached per core so that outliers do not cause massive memory overhead. TODO Implement global limit of active secondaryId sub plans

Type StandaloneConfig

Details

Java Type: com.bakdata.conquery.models.config.StandaloneConfig

Supported Fields:

Field Type Default Example Description
numberOfShardNodes int 2

Type XodusConfig

Details

Java Type: com.bakdata.conquery.models.config.XodusConfig

Supported Fields:

Field Type Default Example Description
cipherBasicIV long or null null
cipherId String null
cipherKey String null
envCloseForcedly boolean or null null
envGatherStatistics boolean or null null
envIsReadonly boolean or null null
envMaxParallelReadonlyTxns int or null null
envMaxParallelTxns int or null null
envMonitorTxnsCheckFreq int or null null
envMonitorTxnsTimeout @NotNull @Valid Duration "10 minutes"
envReadonlyEmptyStores boolean or null null
envStoreGetCacheSize int or null null
envTxnDowngradeAfterFlush boolean or null null
envTxnReplayMaxCount int or null null
envTxnReplayTimeout long or null null
fullFileReadonly boolean or null null
gcEnabled boolean or null null
gcFileMinAge int or null null
gcFilesDeletionDelay int or null null
gcFilesInterval int or null null
gcMinUtilization int or null null
gcRenameFiles boolean or null null
gcRunPeriod int or null null
gcStartIn int or null null
gcTransactionAcquireTimeout int or null null
gcTransactionTimeout int or null null
gcUseExclusiveTransaction boolean or null null
gcUtilizationFromFile String null
gcUtilizationFromScratch boolean or null null
logCacheFreePhysicalMemoryThreshold long or null null
logCacheNonBlocking boolean or null null
logCacheOpenFilesCount int or null null
logCachePageSize DataSize null
logCacheShared boolean or null null
logCacheUseNio boolean or null null
logCleanDirectoryExpected boolean or null null
logClearInvalid boolean or null null
logDurableWrite boolean or null null
logFileSize @MaxDataSize(value=1, unit=DataSizeUnit.GIGABYTES) DataSize "400 megabytes"
logLockId String null
logLockTimeout Duration "1 second"
logSyncPeriod long or null null
managementEnabled boolean or null null
managementOperationsRestricted boolean or null null
memoryUsage DataSize null
memoryUsagePercentage int or null null
treeMaxPageSize int or null null

Type XodusStoreFactory

Details

Java Type: com.bakdata.conquery.models.config.XodusStoreFactory

Supported Fields:

Field Type Default Example Description
bufferPerWorker @jakarta.validation.constraints.Min(1) int 20 How many slots of buffering to use before the IO thread is put to sleep.
directory Path "file://./storage"
loadEnvironmentWithMissingStores boolean false If set, an environment will not be loaded if it misses a required store. If not set, the environment is loaded and the application needs to create the store. This is useful if a new version introduces a new store, but will also alter the environment upon reading.
readerWorkers @jakarta.validation.constraints.Min(1) int 4 Number of threads reading from XoduStore.
removeUnreadableFromStore boolean false Flag for the {@link SerializingStore} whether to delete values from the underlying store, that cannot be mapped to an object anymore.
unreadableDataDumpDirectory File null When set, all values that could not be deserialized from the persistent store, are dump into individual files.
useWeakDictionaryCaching boolean false
validateOnWrite boolean false
weakCacheDuration @NotNull Duration "48 hours"
xodus XodusConfig