[AMORO-2365] Fix get target snapshot NPE when commiting #2366
Conversation
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## master #2366 +/- ##
============================================
- Coverage 53.14% 52.77% -0.38%
- Complexity 3785 4374 +589
============================================
Files 488 534 +46
Lines 25641 30384 +4743
Branches 2450 2967 +517
============================================
+ Hits 13628 16035 +2407
- Misses 10953 13059 +2106
- Partials 1060 1290 +230
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
|
When the table is planning:
After planning:
When the table is planning: Restart the AMS: When the table is planning I set |
ams/server/src/main/java/com/netease/arctic/server/optimizing/OptimizingQueue.java
Outdated
Show resolved
Hide resolved
ams/server/src/main/java/com/netease/arctic/server/optimizing/OptimizingProcess.java
Outdated
Show resolved
Hide resolved
ams/server/src/main/java/com/netease/arctic/server/optimizing/OptimizingQueue.java
Outdated
Show resolved
Hide resolved
ams/server/src/main/java/com/netease/arctic/server/optimizing/OptimizingQueue.java
Outdated
Show resolved
Hide resolved
ams/server/src/main/java/com/netease/arctic/server/optimizing/OptimizingQueue.java
Outdated
Show resolved
Hide resolved
…-npe # Conflicts: # ams/server/src/main/java/com/netease/arctic/server/optimizing/OptimizingQueue.java # ams/server/src/main/java/com/netease/arctic/server/optimizing/maintainer/IcebergTableMaintainer.java # ams/server/src/main/java/com/netease/arctic/server/table/executor/TableRuntimeRefreshExecutor.java
Fixed. |
| private volatile String failedReason; | ||
| private long endTime = ArcticServiceConstants.INVALID_TIME; | ||
| private Map<String, Long> fromSequence = Maps.newHashMap(); | ||
| private Map<String, Long> toSequence = Maps.newHashMap(); | ||
| private final OptimizingPlanner planner; |
There was a problem hiding this comment.
I think the main issue here is that the planner, as a field of the optimizing process, is not being persisted, which leads to the process having to be closed after restarting AMS.
My suggestion is not to add this field planner, but instead to construct a new planner when calling planTasks() and pass it as a parameter, and the targetSnapshotId and targetChangeSnapshotId will be set when planning tasks.
Why are the changes needed?
Close #2365.
Brief change log
TableOptimizingProcesswhen the table is planning and update it after planned.How was this patch tested?
Add some test cases that check the changes thoroughly including negative and positive cases if possible
Add screenshots for manual tests if appropriate
Run test locally before making a pull request
Documentation