Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Subtask] support row-level operations to iceberg Table #2543

Closed
caican00 opened this issue Mar 15, 2024 · 1 comment · Fixed by #3243
Closed

[Subtask] support row-level operations to iceberg Table #2543

caican00 opened this issue Mar 15, 2024 · 1 comment · Fixed by #3243
Assignees
Labels
0.5.1 Release v0.5.1 0.6.0 Release v0.6.0 subtask Subtasks of umbrella issue

Comments

@caican00
Copy link
Contributor

caican00 commented Mar 15, 2024

Describe the subtask

  1. support merge operation to iceberg Table.
    For commands:
    update catalog.db.table set c1=x, c2=y, ... where xxx

merge into xxx

  1. inject iceberg extension

Parent issue

#1571

@caican00 caican00 added the subtask Subtasks of umbrella issue label Mar 15, 2024
@caican00
Copy link
Contributor Author

will work on it.

@caican00 caican00 changed the title [Subtask] support merge operation to iceberg Table [Subtask] support row-level update operation to iceberg Table Mar 19, 2024
@caican00 caican00 changed the title [Subtask] support row-level update operation to iceberg Table [Subtask] support row-level operation to iceberg Table Mar 21, 2024
@caican00 caican00 changed the title [Subtask] support row-level operation to iceberg Table [Subtask] support row-level operations to iceberg Table Mar 21, 2024
caican00 added a commit to caican00/gravitino that referenced this issue Mar 22, 2024
@jerryshao jerryshao added this to the Gravitino June Release milestone Apr 24, 2024
caican00 added a commit to caican00/gravitino that referenced this issue May 1, 2024
caican00 added a commit to caican00/gravitino that referenced this issue May 1, 2024
caican00 added a commit to caican00/gravitino that referenced this issue May 1, 2024
caican00 added a commit to caican00/gravitino that referenced this issue May 2, 2024
caican00 added a commit to caican00/gravitino that referenced this issue May 5, 2024
FANNG1 pushed a commit that referenced this issue May 13, 2024
…g Table (#3243)

### What changes were proposed in this pull request?

- refactor table implementation, make `SparkIcebergTable` extend Iceberg
`SparkTable`, and `SparkHiveTable` extend Kyuubi `HiveTable`.

- support row-level operations to iceberg Table

```
1. update tableName set c1=v1, c2=v2, ...

2. merge into targetTable t
   using sourceTable s
   on s.key=t.key
   when matched then ...
   when not matched then ...

3. delete from table where xxx
```

### Why are the changes needed?

1. For spark-connector in Iceberg, it explicitly uses `SparkTable` to
identify whether it is an Iceberg table, so the `SparkIcebergTable` must
extend `SparkTable`.

2. support row-level operations to iceberg Table.

Fix: #2543

### Does this PR introduce any user-facing change?
Yes, support update ... , merge into ..., delete from ...

### How was this patch tested?
New ITs.
github-actions bot pushed a commit that referenced this issue May 13, 2024
…g Table (#3243)

### What changes were proposed in this pull request?

- refactor table implementation, make `SparkIcebergTable` extend Iceberg
`SparkTable`, and `SparkHiveTable` extend Kyuubi `HiveTable`.

- support row-level operations to iceberg Table

```
1. update tableName set c1=v1, c2=v2, ...

2. merge into targetTable t
   using sourceTable s
   on s.key=t.key
   when matched then ...
   when not matched then ...

3. delete from table where xxx
```

### Why are the changes needed?

1. For spark-connector in Iceberg, it explicitly uses `SparkTable` to
identify whether it is an Iceberg table, so the `SparkIcebergTable` must
extend `SparkTable`.

2. support row-level operations to iceberg Table.

Fix: #2543

### Does this PR introduce any user-facing change?
Yes, support update ... , merge into ..., delete from ...

### How was this patch tested?
New ITs.
caican00 added a commit to caican00/gravitino that referenced this issue May 14, 2024
… to iceberg Table (datastrato#3243)

### What changes were proposed in this pull request?

- refactor table implementation, make `SparkIcebergTable` extend Iceberg
`SparkTable`, and `SparkHiveTable` extend Kyuubi `HiveTable`.

- support row-level operations to iceberg Table

```
1. update tableName set c1=v1, c2=v2, ...

2. merge into targetTable t
   using sourceTable s
   on s.key=t.key
   when matched then ...
   when not matched then ...

3. delete from table where xxx
```

### Why are the changes needed?

1. For spark-connector in Iceberg, it explicitly uses `SparkTable` to
identify whether it is an Iceberg table, so the `SparkIcebergTable` must
extend `SparkTable`.

2. support row-level operations to iceberg Table.

Fix: datastrato#2543

### Does this PR introduce any user-facing change?
Yes, support update ... , merge into ..., delete from ...

### How was this patch tested?
New ITs.
FANNG1 pushed a commit that referenced this issue May 14, 2024
…g Table (#3382)

### What changes were proposed in this pull request?

- refactor table implementation, make `SparkIcebergTable` extend Iceberg
`SparkTable`, and `SparkHiveTable` extend Kyuubi `HiveTable`.

- support row-level operations to iceberg Table

```
1. update tableName set c1=v1, c2=v2, ...

2. merge into targetTable t
   using sourceTable s
   on s.key=t.key
   when matched then ...
   when not matched then ...

3. delete from table where xxx
```

### Why are the changes needed?

1. For spark-connector in Iceberg, it explicitly uses `SparkTable` to
identify whether it is an Iceberg table, so the `SparkIcebergTable` must
extend `SparkTable`.

2. support row-level operations to iceberg Table.

Fix: #2543

### Does this PR introduce any user-facing change? 
Yes, support update ... , merge into ..., delete from ...

### How was this patch tested?
New ITs.
@FANNG1 FANNG1 added 0.5.1 Release v0.5.1 0.6.0 Release v0.6.0 labels May 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0.5.1 Release v0.5.1 0.6.0 Release v0.6.0 subtask Subtasks of umbrella issue
Projects
None yet
3 participants