Improve modularity of presto-hive-common module #22774
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Refactoring presto-hive-common module by creating a separate module for files required by presto-parquet and presto-cache modules to reduce dependency clutter
Description
This change aims to reduce the friction between the Prism connector and the Hive connector by separating out some of the classes from presto-hive-common into a new module called presto-hdfs-core. This will allow for better decoupling between the two connectors and eliminate the need for weird runtime interfaces.
Motivation and Context
The Prism connector is where most of Meta's specific integration points are, and it takes the OSS Hive connector as a runtime dependency to reduce code duplication. However, differences in file formats, metastore, and storage create a lot of friction. The current approach of introducing unclean runtime interfaces to support Prism specific use cases creates strong coupling between the two connectors. By separating out some of the classes from presto-hive-common into presto-hdfs-core, we can eliminate the dependency on presto-hive-common from non-connector modules. This will make it easier to maintain and update the codebase, and reduce the risk of introducing bugs or issues when making changes to the connectors. Additionally, this change will make it easier to add new features or functionality to the Prism connector without affecting the Hive connector.
Test Plan
Contributor checklist
Release Notes