-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
issue#1503: memtable scan optimization #1504
base: master
Are you sure you want to change the base?
Conversation
please fill the pull request form, thanks |
e68e0f3
to
3ceb0b1
Compare
@@ -170,6 +171,27 @@ class ObMemtableScanIterator : public ObIMemtableScanIterator | |||
uint8_t iter_flag_; | |||
}; | |||
|
|||
class ObMemtableBlockScanIterator : public ObMemtableScanIterator |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe inherit the ObIMemtableScanIterator class here?
const common::ObIArray<blocksstable::ObStorageDatum> &default_datums = filter->get_default_datums(); | ||
const blocksstable::ObColDescIArray &cols_desc = read_info_->get_columns_desc(); | ||
|
||
for (int64_t row_idx = pd_filter_info.start_; OB_SUCC(ret) && row_idx < pd_filter_info.end_; ++row_idx) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If pd_filter_info.start_ is always 0, perhaps we could use pd_filter_info.row_count_ to iterate all rows?
Use row_count appears to be clearer and more straightforward to comprehend for me.
if (filter->is_logic_and_node()) { | ||
if (OB_FAIL(result->bit_and(*child_result))) { | ||
LOG_WARN("Failed to merge result bitmap", K(ret), KP(child_result)); | ||
} else if (result->is_all_false()) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add unittest case to test these situations :
- skip partitial rows in next child filter node
- skip all next child nodes when all_false or all_true
src/storage/memtable/ob_memtable.cpp
Outdated
// TRANS_LOG(INFO, "Memtable scan iterator choice : normal"); | ||
// } | ||
if (tenant_config.is_valid()) { | ||
if (tenant_config->memtable_scan_filter_pushdown) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are two suggestions:
- The code here is a little redundant.
- We can use _pushdown_storage_level config
e073cec
to
f43d6d6
Compare
2ad003d
to
15ad5d1
Compare
15ad5d1
to
aa21c19
Compare
Task Description
close #1503
Solution Description
(1) First, enable single edge table scan for memtable, which is the basis for later optimization.
(2) Second, acquire a batch of rows instead of only one.
(3) Third, use bitmap result to calculate the whole batch.
Passed Regressions
Upgrade Compatibility
Other Information
Release Note