Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLink]Fixed handle of null values and add some usage document #471

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

ChenYunHey
Copy link
Contributor

LakesoulConvert hold a hashMap which the key is mongoDB filed's name and the value is filed's schema,the hashMap will put all the fields in,if the field's value is null,the hashmap wiil set the filed's schema null .When build a Struct,if the bsonDocument dose not exist the hashmap hold's filed,then struct will supplement the filed.

another,I fixed the SyncDatabase.Users can control the input of optional parameters of flink jdbc.

Signed-off-by: ChenYunHey <1908166778@qq.com>
Signed-off-by: ChenYunHey <1908166778@qq.com>
Signed-off-by: ChenYunHey <1908166778@qq.com>
@ChenYunHey
Copy link
Contributor Author

my mongoDB document like this:
[
{ _id: 1, name: null },
{ _id: 2, name: 'Bob' },
{ _id: 3, name: null },
{ _id: 4, name: null },
{ _id: 12, name: null },
{ _id: 13, age: null },
{ _id: 14, age: 33 },
{ _id: 16, age: null },
{ _id: 20, name: null },
{ _id: 21, name: 'ss' },
{ _id: 22, name: 'ss' },
{ _id: 24, name: null }
]
the parquet file when write
+-------+------------+
| _id | rowKinds |
|-------+------------|
| 20 | insert |
+-------+------------+
+-------+-------+------------+
| _id | age | rowKinds |
|-------+-------+------------|
| 4 | | insert |
| 14 | 33 | insert |
| 16 | | insert |
+-------+-------+------------+

+-------+--------+-------+------------+
| _id | name | age | rowKinds |
|-------+--------+-------+------------|
| 1 | | | insert |
| 2 | Bob | | insert |
| 3 | | | insert |
| 12 | | | insert |
| 21 | ss | | insert |
| 22 | ss | | insert |
| 24 | | | insert |
+-------+--------+-------+------------+

Signed-off-by: ChenYunHey <1908166778@qq.com>
Signed-off-by: ChenYunHey <1908166778@qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant