Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[EXPERIMENT]: Subscriptions: Fuse BSATN for all table row ops in one allocation #1200

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

Centril
Copy link
Contributor

@Centril Centril commented May 3, 2024

Description of Changes

Making a combined combined "ticket plus code experiment" that demonstrates the issue.
Here in this PR, instead of creating one Vec<u8> of BSATN per row, we put all the BSATN of all rows in a single Vec<u8>.
(In a real PR we'd need to add the number of elements as well and other changes, but this is just a perf demonstration.)

In a sample benchmark run on my i7-7700K, 64GB RAM, this result in the following improvements compared to HEAD~1:

Benchmarking full-scan: Collecting 100 samples in estimated 5.5034 s (200 iteration
full-scan               time:   [27.010 ms 27.122 ms 27.300 ms]
                        change: [-71.553% -71.319% -71.037%] (p = 0.00 < 0.05)
                        Performance has improved.

Benchmarking full-join: Collecting 100 samples in estimated 5.2570 s (25k iteration
full-join               time:   [216.69 µs 218.12 µs 220.19 µs]
                        change: [-25.202% -24.271% -23.234%] (p = 0.00 < 0.05)
                        Performance has improved.

For full-scan, this is about a 3.5x improvement to runtime (was about 96ms before).
For full-join, this is about a 1.3x improvement (full-join does more additional interesting work, explaining the difference).

Here is a picture of the flamegraph before:

image

And here is one after:

image

The big malloc bar is gone and instead, TableCursor + and to_bsatn_extend are taking up the remaining time.

API and ABI breaking changes

If this is an API or ABI breaking change, please apply the
corresponding GitHub label.

Expected complexity level and risk

How complicated do you think these changes are? Grade on a scale from 1 to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex change.

This complexity rating applies not only to the complexity apparent in the diff,
but also to its interactions with existing and future code.

If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning ways.

Testing

Describe any testing you've done, and any testing you'd like your reviewers to do,
so that you're confident that all the changes work as expected!

  • Write a test you've completed here.
  • Write a test you want a reviewer to do here, so they can check it off when they're satisfied.

@Centril
Copy link
Contributor Author

Centril commented May 3, 2024

Blocked by #1077.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant