Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Improvement] revisit all the dropXXX interfaces to have a consistent behavior #2434

Open
1 task done
FANNG1 opened this issue Mar 5, 2024 · 16 comments · May be fixed by #3222
Open
1 task done

[Improvement] revisit all the dropXXX interfaces to have a consistent behavior #2434

FANNG1 opened this issue Mar 5, 2024 · 16 comments · May be fixed by #3222
Assignees
Labels
improvement Improvements on everything

Comments

@FANNG1
Copy link
Contributor

FANNG1 commented Mar 5, 2024

What would you like to be improved?

we should revisit all the dropXXX interfaces:

a) We should define a clear behavior of return value (false) and exception thrown.
b) We should have a consistent behavior for all the dropXXX interfaces.

How should we improve?

No response

Subtasks

@FANNG1 FANNG1 added the improvement Improvements on everything label Mar 5, 2024
@FANNG1 FANNG1 added this to the Gravitino 0.5.0 milestone Mar 5, 2024
@jerryshao
Copy link
Collaborator

@mchades @FANNG1 @yuqi1129 @diqiu50 let's discuss here.

@FANNG1
Copy link
Contributor Author

FANNG1 commented Mar 19, 2024

if succ, return true, if not exists return false, otherwise, throw corresponding exception, the behavior seems clear.

@jerryshao
Copy link
Collaborator

jerryshao commented Mar 19, 2024

I think we should revisit all the drop related APIs and implementations, and change the behavior if necessary, which includes:

  1. Rest client and server implementations.
  2. The drop implementations in each catalog.

@charliecheng630
Copy link
Contributor

I would love to revisit all the drop related APIs and implementations first, then write the results into a document.
What do you think about this? @mchades @FANNG1 @yuqi1129 @diqiu50 @jerryshao

@jerryshao
Copy link
Collaborator

Great, you can go ahead. Thanks a lot.

@charliecheng630
Copy link
Contributor

Is this a list of all Rest APIs?
https://github.com/datastrato/gravitino/tree/main/docs/open-api

@mchades
Copy link
Contributor

mchades commented Mar 19, 2024

Is this a list of all Rest APIs? https://github.com/datastrato/gravitino/tree/main/docs/open-api

Yep, you can also look into the codes in package com.datastrato.gravitino.server.web.rest

The web also FYI: https://datastrato.ai/docs/0.4.0/api/rest/gravitino-rest-api

@charliecheng630
Copy link
Contributor

@mchades @FANNG1 @yuqi1129 @diqiu50 @jerryshao
The result after revisiting the drop APIs is following. Please correct me if there are any omissions.
Drop Related APIs

@mchades
I didn't find the information about dropPartition in the open API. Has it not been made available yet?

@mchades
Copy link
Contributor

mchades commented Mar 20, 2024

Could you please also add the exception response? For example, the response when encountering a RuntimeException.

I didn't find the information about dropPartition in the open API. Has it not been made available yet?

yep, it's WIP #2492

BTW, it's better to make your doc commentable

@charliecheng630
Copy link
Contributor

Could you please also add the exception response? For example, the response when encountering a RuntimeException.

I didn't find the information about dropPartition in the open API. Has it not been made available yet?

yep, it's WIP #2492

BTW, it's better to make your doc commentable

The exception response has been updated, but it seems that the implementation is not aligned.

@jerryshao
Copy link
Collaborator

@charliecheng630 are you still working on this? I found that for most the JDBC catalogs, the current implementation is not aligned the definition, we should improve it.

@charliecheng630
Copy link
Contributor

@jerryshao I haven't started to work yet. I will work on this ASAP.

@jerryshao
Copy link
Collaborator

OK, thanks!

@charliecheng630
Copy link
Contributor

@mchades After tracing code, here are the areas that need to be improve:

com.datastrato.gravitino.catalog.xxx.xxxCatalogOperations#dropXXX
com.datastrato.gravitino.catalog.xxxOperationDispatcher#dropXXX
com.datastrato.gravitino.storage.relational.service.xxxService#deleteXXX

Please correct me if I am wrong.

@jerryshao
Copy link
Collaborator

jerryshao commented Apr 22, 2024

I'm working on the core part to fix the current fix, you can check #3075 and the related PR. I think what you should check is:

  1. all the specific catalog's dropXXX.
  2. CatalogManger/MetalakeManager's dropXXX.
  3. Rest server's dropXXX.
  4. Client API's dropXXX.

@jerryshao
Copy link
Collaborator

I've check all the Operation Dispatcher's dropXXX in #3075.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
improvement Improvements on everything
Projects
None yet
4 participants