Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Control Delta allocation to be separate from storage #1895

Draft
wants to merge 12 commits into
base: master
Choose a base branch
from

Conversation

Ignition
Copy link
Contributor

@Ignition Ignition commented Apr 2, 2024

Description

  • Change PropertyValue's map from std::map -> boost::container::flat_map
  • Template PropertyValue to take an allocator
  • Custom delta container to ensure allocations exist separate to storage allocations

related to #1685

Using this benchmark
FOREACH (i in range(1, 3000000) | CREATE (:Label1:Label2:Label3:Label4:Label5:Label6));

RSS is now

IN_MEMORY_TRANSACTIONAL ~575MiB
IN_MEMORY_ANALYTICAL    ~560MiB

[master < Task] PR

  • Provide the full content or a guide for the final git message
    • Explicit control over delta allocations

Documentation checklist

  • Add the documentation label tag
  • Add the bug / feature label tag
  • Add the milestone for which this feature is intended
    • If not known, set for a later milestone
  • Write a release note, including added/changed clauses
    • [Release note text]
  • Link the documentation PR here
    • [Documentation PR link]
  • Tag someone from docs team in the comments

@Ignition Ignition added memory Docs - changelog only Docs - changelog only labels Apr 2, 2024
@Ignition Ignition self-assigned this Apr 2, 2024
@Ignition Ignition added this to the mg-v2.17.0 milestone Apr 3, 2024
@Ignition Ignition marked this pull request as ready for review April 25, 2024 10:13
@Ignition Ignition changed the title WIP: Introduce delta pool memory allocator Introduce delta pool memory allocator Apr 25, 2024
@Ignition Ignition changed the title Introduce delta pool memory allocator Control Delta allocation to be separate from storage Apr 25, 2024
Copy link
Collaborator

@antoniofilipovic antoniofilipovic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really great job, I don't have much comments but few suggestions

src/storage/v2/delta.hpp Show resolved Hide resolved
src/storage/v2/delta.hpp Show resolved Hide resolved
src/storage/v2/delta_container.hpp Outdated Show resolved Hide resolved
src/utils/static_vector.hpp Outdated Show resolved Hide resolved
src/utils/static_vector.hpp Outdated Show resolved Hide resolved
src/utils/static_vector.hpp Outdated Show resolved Hide resolved
src/storage/v2/property_value.hpp Show resolved Hide resolved
src/storage/v2/property_value.hpp Show resolved Hide resolved
src/storage/v2/property_value.hpp Show resolved Hide resolved
src/storage/v2/delta_container.hpp Outdated Show resolved Hide resolved
@Ignition Ignition added CI -build=debug -test=core Run debug build and core tests on push CI -build=debug -test=integration Run debug build and integration tests on push CI -build=release -test=core Run release build and core tests on push CI -build=release -test=e2e Run release build and e2e tests on push CI -build=release -test=benchmark Run release build and benchmark on push labels Apr 30, 2024
@Ignition Ignition force-pushed the delta_pool branch 2 times, most recently from e33893c to 990cd94 Compare April 30, 2024 12:27
Copy link
Collaborator

@antoniofilipovic antoniofilipovic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome job!

Copy link

sonarcloud bot commented May 1, 2024

Quality Gate Passed Quality Gate passed

Issues
39 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.9% Duplication on New Code

See analysis details on SonarCloud

@Ignition Ignition added this pull request to the merge queue May 1, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks May 1, 2024
@Ignition Ignition added this pull request to the merge queue May 1, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks May 1, 2024
@Ignition Ignition marked this pull request as draft May 7, 2024 14:34
@Ignition Ignition modified the milestones: mg-v2.17.0, mg-v2.18.0 May 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI -build=debug -test=core Run debug build and core tests on push CI -build=debug -test=integration Run debug build and integration tests on push CI -build=release -test=benchmark Run release build and benchmark on push CI -build=release -test=core Run release build and core tests on push CI -build=release -test=e2e Run release build and e2e tests on push Docs - changelog only Docs - changelog only memory
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants