Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Iceberg] Support custom catalog name for the Iceberg RESTCatalog #21813

Open
osscm opened this issue May 3, 2024 · 4 comments
Open

[Iceberg] Support custom catalog name for the Iceberg RESTCatalog #21813

osscm opened this issue May 3, 2024 · 4 comments

Comments

@osscm
Copy link
Contributor

osscm commented May 3, 2024

Looks like currently the catalogName is the Trino's catalog-name (https://github.com/trinodb/trino/blob/master/lib/trino-plugin-toolkit/src/main/java/io/trino/plugin/base/CatalogNameModule.java#L29)

there will be cases where the catalogName used on the REST Server side could be different, specially when users are migrating from HMS to RESTCatalog, than in the current model, we force user to rename the Trino catalog name.

ref: https://github.com/trinodb/trino/blob/master/plugin/trino-iceberg/src/main/java/io/trino/plugin/iceberg/catalog/rest/TrinoRestCatalog.java#L493

@osscm
Copy link
Contributor Author

osscm commented May 3, 2024

cc @danielcweeks if you can please validate my understanding, and does request make sense? Or I am just blabbering :)

@danielcweeks
Copy link
Contributor

@osscm I don't think this is an issue.

The catalog name in Trino (as referenced by sql) will be baed on the <catalog>.properties filename or the name of the registered catalog using CREATE CATALOG <catalog_name> . . . sql syntax.

The name of the warehouse for the REST catalog will either be defined by the uri endpoint itself or can be provided by the iceberg.rest-catalog.warehouse property in the REST catalog configuration. This is up to the backing REST catalog implementation.

These two identifiers are independent (but generally good idea to use the same catalog names everywhere if they represent the same thing).

@osscm
Copy link
Contributor Author

osscm commented May 3, 2024

thanks @danielcweeks for a quick response.

If someone is utilizing a customized hive metastore catalog-name and transitioning to REST, it's recommended to employ it either as the iceberg.rest-catalog.warehouse or as the suffix of the URL? and adhering to the RESTCatalog specifications. Ideally, the RESTCatalog server should support either of of these options?

@osscm
Copy link
Contributor Author

osscm commented May 6, 2024

fyi, related Iceberg issue apache/iceberg#10205

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

2 participants