Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Improvement]: For Mixed-Hive tables, default refresh time for Hive partitions should be configured #2391

Open
3 tasks done
shendanfengg opened this issue Nov 30, 2023 · 1 comment · May be fixed by #2392
Open
3 tasks done

[Improvement]: For Mixed-Hive tables, default refresh time for Hive partitions should be configured #2391

shendanfengg opened this issue Nov 30, 2023 · 1 comment · May be fixed by #2392
Labels
type:improvement

Comments

@shendanfengg
Copy link
Contributor

Search before asking

  • I have searched in the issues and found no similar issues.

What would you like to be improved?

image
As shown in the figure, our default configuration is -1, which may result in Full Self-Optimizing not being executed for some Mixed Hive table history partitions.

How should we improve?

Set a default value for this configuration

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Subtasks

No response

Code of Conduct
@shendanfengg shendanfengg linked a pull request Nov 30, 2023 that will close this issue
Open
3 tasks
@majin1102
Copy link
Contributor

@shendanfengg I saw you trying to change the default value of base.hive.refresh-interval in order to refresh the data in Hive for all Mixed-Hive format tables every half an hour by default.

This configuration was initially disabled because we couldn't ensure that all users of Mixed-Hive format tables have this requirement. Can you adjust the default configuration under Catalog to enable this feature for all your Mixed-Hive tables instead of directly changing the default value of this configuration? I still can't confirm that all users of Mixed-Hive format tables have this requirement.

Hi, danfeng, please check the comment in releated PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type:improvement
Projects
None yet
Milestone
No milestone
2 participants
@majin1102 @shendanfengg