[#3259] feat(spark-connector): Support atomic create or replace Iceberg table #3260
Conversation
caican00
force-pushed
the
iceberg-stage-ddl
branch
2 times, most recently
from
3388498
to
388df1f
Compare
…lace Iceberg table
caican00
force-pushed
the
iceberg-stage-ddl
branch
from
388df1f
to
6bde9ff
Compare
|
Hi @FANNG1 could you help review this PR? Thank you. |
ok |
| @@ -416,4 +489,11 @@ private SparkMetadataColumnInfo[] getIcebergMetadataColumns() { | |||
| new SparkMetadataColumnInfo("_deleted", DataTypes.BooleanType, false) | |||
| }; | |||
| } | |||
|
|
|||
| private List<SparkTableInfo.SparkColumnInfo> getSimpleTableColumnWithoutComments() { | |||
There was a problem hiding this comment.
please add comment why add this method and when to use
| String tableName = "test_atomic_create_table_as_select"; | ||
| dropTableIfExists(tableName); | ||
|
|
||
| CatalogPlugin sparkIcebergCatalog = |
There was a problem hiding this comment.
is it necessary to add this check?
| @@ -96,6 +103,28 @@ public UnboundFunction loadFunction(Identifier ident) throws NoSuchFunctionExcep | |||
| return ((SparkCatalog) sparkCatalog).loadFunction(ident); | |||
| } | |||
|
|
|||
| @SuppressWarnings("deprecation") | |||
| @Override | |||
| public StagedTable stageCreate( | |||
There was a problem hiding this comment.
Gravitino doesn't support stage create for now, this will bypass the gravitino when create&replace table as select, I prefer to delay support of stage create for now. WDYT? cc @jerryshao
There was a problem hiding this comment.
Gravitino doesn't support stage create for now, this will bypass the gravitino when
create&replace table as select, I prefer to delay support of stage create for now. WDYT? cc @jerryshao
@FANNG1 But rest apis don't support RTAS. Is the purpose an audit?
What changes were proposed in this pull request?
Support atomic create or replace Iceberg table.
Why are the changes needed?
Support atomic create or replace Iceberg table.
Fix: #3259
Does this PR introduce any user-facing change?
No.
How was this patch tested?
New ITs.