You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
sometimes it'll failover because compact deletion will conflict with the partition expiration.
Let's say there is fileA which need to be marked as delete in a partition expiration task, and writer suppose the file still exists and pick it in a compaction task. Then compaction task will be finished with committing it as delete and will result in deleting a file which is not recorded by the base manifest.
What doesn't meet your expectations?
This caused a flink job failover
Anything else?
No response
Are you willing to submit a PR?
I'm willing to submit a PR!
The text was updated successfully, but these errors were encountered:
Search before asking
Paimon version
release-0.7
Compute Engine
flink-1.17
Minimal reproduce step
Let's say there is fileA which need to be marked as
delete
in a partition expiration task, and writer suppose the file still exists and pick it in a compaction task. Then compaction task will be finished with committing it asdelete
and will result in deleting a file which is not recorded by the base manifest.What doesn't meet your expectations?
This caused a flink job failover
Anything else?
No response
Are you willing to submit a PR?
The text was updated successfully, but these errors were encountered: