[EXPERIMENT]: Subscriptions: Fuse BSATN for all table row ops in one allocation #1200
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of Changes
Making a combined combined "ticket plus code experiment" that demonstrates the issue.
Here in this PR, instead of creating one
Vec<u8>
of BSATN per row, we put all the BSATN of all rows in a singleVec<u8>
.(In a real PR we'd need to add the number of elements as well and other changes, but this is just a perf demonstration.)
In a sample benchmark run on my i7-7700K, 64GB RAM, this result in the following improvements compared to HEAD~1:
For full-scan, this is about a 3.5x improvement to runtime (was about 96ms before).
For full-join, this is about a 1.3x improvement (full-join does more additional interesting work, explaining the difference).
Here is a picture of the flamegraph before:
And here is one after:
The big
malloc
bar is gone and instead,TableCursor
+ andto_bsatn_extend
are taking up the remaining time.API and ABI breaking changes
If this is an API or ABI breaking change, please apply the
corresponding GitHub label.
Expected complexity level and risk
How complicated do you think these changes are? Grade on a scale from 1 to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex change.
This complexity rating applies not only to the complexity apparent in the diff,
but also to its interactions with existing and future code.
If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning ways.
Testing
Describe any testing you've done, and any testing you'd like your reviewers to do,
so that you're confident that all the changes work as expected!