Skip to content

Import JSONs

thoniTUB edited this page May 6, 2024 · 2 revisions

Import JSONs

This is an automatically created documentation. It is not 100% accurate since the generator does not handle every edge case.

Instead of a list ConQuery also always accepts a single element.

Each *.import.json has to contain exactly one ImportDescriptor.


Base OutputDescription

Different types of OutputDescription can be used by setting operation to one of the following values:

COMPOUND_DATE_RANGE

Output creating delegating store of start and end-Column neighbours.

This output will still parse and validate the data to ensure that some assertions are held (ie.: only open when allowOpen is set, and start <= end).

Details

Java Type: com.bakdata.conquery.models.preproc.outputs.CompoundDateRangeOutput

Supported Fields:

Field Type Default Example Description
allowOpen boolean false
endColumn String null
startColumn String null
name String null
required boolean false

COPY

Parse column as type.

Details

Java Type: com.bakdata.conquery.models.preproc.outputs.CopyOutput

Supported Fields:

Field Type Default Example Description
inputColumn String null
inputType one of STRING, INTEGER, BOOLEAN, REAL, DECIMAL, MONEY, DATE, DATE_RANGE null
name String null
required boolean false

DATE_RANGE

Parse input columns as {@link CDateRange}. Input values must be {@link com.bakdata.conquery.models.common.CDate} based ints.

Details

Java Type: com.bakdata.conquery.models.preproc.outputs.DateRangeOutput

Supported Fields:

Field Type Default Example Description
allowOpen boolean false Parse null values as open date-range if true.
endColumn String null
startColumn String null
name String null
required boolean false

EPOCH

Parse input column as {@link com.bakdata.conquery.models.common.CDate} based int.

Details

Java Type: com.bakdata.conquery.models.preproc.outputs.EpochOutput

Supported Fields:

Field Type Default Example Description
inputColumn String null
name String null
required boolean false

EPOCH_DATE_RANGE

Parse input columns as {@link CDateRange}. Input values must be {@link com.bakdata.conquery.models.common.CDate} based ints.

Details

Java Type: com.bakdata.conquery.models.preproc.outputs.EpochDateRangeOutput

Supported Fields:

Field Type Default Example Description
allowOpen boolean false Parse null values as open date-range if true.
endColumn String null
startColumn String null
name String null
required boolean false

LINE

Outputs the current line in the file.

Details

Java Type: com.bakdata.conquery.models.preproc.outputs.LineOutput

Supported Fields:

Field Type Default Example Description
name String null
required boolean true

NULL

Output a null value.

Details

Java Type: com.bakdata.conquery.models.preproc.outputs.NullOutput

Supported Fields:

Field Type Default Example Description
inputType one of STRING, INTEGER, BOOLEAN, REAL, DECIMAL, MONEY, DATE, DATE_RANGE null
name String null
required boolean false

Other Types

Type TableImportDescriptor

Combines potentially multiple input files to be loaded into a single table. Describing their respective transformation. All Inputs must produce the same types of outputs. For further detail see {@link TableInputDescriptor}, and {@link Preprocessor}. This file describes an import.json used as description for the preprocess command.

Details

Java Type: com.bakdata.conquery.models.preproc.TableImportDescriptor

Supported Fields:

Field Type Default Example Description
label String null "someLabel" shown in the frontend
name String null
inputs list of TableInputDescriptor null A single source input.
table String null Target table to load the import to.

Type TableInputDescriptor

An input describes transformations on a single CSV file to be loaded into the table described in {@link TableImportDescriptor}.

It requires a primary Output and at least one normal output.

Input data can be filtered using the field filter, which is evaluated as a groovy script on every row.

Details

Java Type: com.bakdata.conquery.models.preproc.TableInputDescriptor

Supported Fields:

Field Type Default Example Description
filter String null
output list of @Valid @NotEmpty OutputDescription null
primary @NotNull @Valid OutputDescription Output producing the primary column. This should be the primary key across all tables. Default is COPY("pid", STRING)
sourceFile String null