Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HIVE-28219: Support drop partitions by names in IMetaStoreClient #5223

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

wecharyu
Copy link
Contributor

What changes were proposed in this pull request?

In thrift api definition, HMS support to drop partitions by both partition names and expressions. But current api in IMetaStoreClient only support drop partitions by expressions, we should add new api to support drop partitions by names.

  // IMetaStoreClient.java
  List<Partition> dropPartitions(String catName, String dbName, String tblName,
                                 RequestPartsSpec partsSpec, PartitionDropOptions options)
      throws NoSuchObjectException, MetaException, TException;

Why are the changes needed?

To improve the dropPartitions function.

Does this PR introduce any user-facing change?

Yes, user can use the new api to drop partitions by names.

Is the change a dependency upgrade?

No.

How was this patch tested?

Add unit test:

mvn test -Dtest=org.apache.hadoop.hive.ql.metadata.TestHive -pl :hive-exec

@wecharyu
Copy link
Contributor Author

wecharyu commented May 8, 2024

This PR is followed #4905 (comment), @zhangbutao @dengzhhu653 @deniskuzZ pls take a look in your convenience.

Copy link

sonarcloud bot commented May 8, 2024

Quality Gate Passed Quality Gate passed

Issues
20 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

@zhangbutao
Copy link
Contributor

Does drop partitions by names have any advantages over drop partitions by expressions?

My concern is that we already have lots of redundant api and other components(Spark/Trino/Impala) may not need the new added api.

But if the new added api has some advantages over old api, i think we can move on.

I also want to listen to other folks's opinions. :)

@wecharyu
Copy link
Contributor Author

wecharyu commented May 9, 2024

Does drop partitions by names have any advantages over drop partitions by expressions?

drop partitions by names is not aimed to replace drop by expressions, they each serve distinct purpose.
And it's more easy to use and do not need deserialization in server side compared with expressions.

@dengzhhu653
Copy link
Member

Does drop partitions by names have any advantages over drop partitions by expressions?

My concern is that we already have lots of redundant api and other components(Spark/Trino/Impala) may not need the new added api.

But if the new added api has some advantages over old api, i think we can move on.

I also want to listen to other folks's opinions. :)

+1

This change looks like a variant of dropPartitions based on one current implementation, it's not a good choice to declare it as a first class API.

If we really need to add it in the future, I would prefer making it an util method instead in MetaStoreUtils

@wecharyu
Copy link
Contributor Author

This change looks like a variant of dropPartitions based on one current implementation, it's not a good choice to declare it as a first class API.

It's more like a function complement, before this PR, hive client can only drop partitions by exprs, but actually the server side supports drop partitions by names.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants