-
Notifications
You must be signed in to change notification settings - Fork 28k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-48031] Decompose viewSchemaMode config, add SHOW CREATE TABLE support #46652
Conversation
be6d238
to
ffa1fdb
Compare
ffa1fdb
to
da06989
Compare
the last commit only updates the migration guide, no need to retest. merging to master! |
.doc("Set to DISABLE to disable the WITH SCHEMA clause for view DDL and suppress the line in " + | ||
" DESCRIBE EXTENDED. The default, and only other value, is COMPENSATION. Views without " + | ||
" WITH SCHEMA clause are defaulted to WITH SCHEMA COMPENSATION.") | ||
val VIEW_SCHEMA_BINDING_ENABLED = buildConf("spark.sql.legacy.viewSchemaBindingMode") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is a bit weird that the boolean config name ends with Mode
.booleanConf | ||
.createWithDefault(true) | ||
|
||
val VIEW_SCHEMA_COMPENSATION = buildConf("spark.sql.legacy.viewSchemaCompensation") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The COMPENSATION behavior should not be legacy.
What changes were proposed in this pull request?
We separate enablement of WITH SCHEMA ... clause from the change in default from SCHEMA BINDING to SCHEMA COMPENSATION.
This allows user to upgrade in two steps:
Why are the changes needed?
It allows customers to upgrade more safely.
Does this PR introduce any user-facing change?
Yes
How was this patch tested?
Added more tests
Was this patch authored or co-authored using generative AI tooling?
No