Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update mimalloc version to april 2024 #2999

Open
romange opened this issue May 2, 2024 · 5 comments
Open

update mimalloc version to april 2024 #2999

romange opened this issue May 2, 2024 · 5 comments
Assignees
Milestone

Comments

@romange
Copy link
Collaborator

romange commented May 2, 2024

No description provided.

@romange romange added this to the v1.18 milestone May 2, 2024
@adiholden
Copy link
Collaborator

comparing new version of mimalloc 2.1.4 to 2.0.9 shows degradation in unaccounted memory

with 2.0.9
127.0.0.1:6379> DEBUG POPULATE 300000 JSON 100 RAND TYPE JSON ELEMENTS 100
OK
(2.28s)
127.0.0.1:6379> info memory

Memory

used_memory:6679902592
used_memory_human:6.22GiB
used_memory_peak:6679902592
used_memory_peak_human:6.22GiB
fibers_stack_vms:1637600
fibers_count:50
used_memory_rss:6743437312
used_memory_rss_human:6.28GiB
used_memory_peak_rss:6743437312
maxmemory:18635944755
maxmemory_human:17.36GiB
object_used_memory:4800000
type_used_memory_JSON:4800000
table_used_memory:16912128
num_buckets:30720
num_entries:300000
inline_keys:300000
listpack_blobs:0
listpack_bytes:0
small_string_bytes:0
pipeline_cache_bytes:0
dispatch_queue_bytes:0
dispatch_queue_subscriber_bytes:0
dispatch_queue_peak_bytes:0
client_read_buffer_peak_bytes:256
tls_bytes:0
cache_mode:store
maxmemory_policy:noeviction

with 2.1.4
127.0.0.1:6379> DEBUG POPULATE 300000 JSON 100 RAND TYPE JSON ELEMENTS 100
OK
(2.29s)
127.0.0.1:6379> info memory

Memory

used_memory:6366138816
used_memory_human:5.93GiB
used_memory_peak:6366138816
used_memory_peak_human:5.93GiB
fibers_stack_vms:1637600
fibers_count:50
used_memory_rss:7705546752
used_memory_rss_human:7.18GiB
used_memory_peak_rss:7705546752
maxmemory:17508161945
maxmemory_human:16.31GiB
object_used_memory:4800000
type_used_memory_JSON:4800000
table_used_memory:16912128
num_buckets:30720
num_entries:300000
inline_keys:300000
listpack_blobs:0
listpack_bytes:0
small_string_bytes:0
pipeline_cache_bytes:0
dispatch_queue_bytes:0
dispatch_queue_subscriber_bytes:0
dispatch_queue_peak_bytes:0
client_read_buffer_peak_bytes:256
tls_bytes:0
cache_mode:store
maxmemory_policy:noeviction

@romange
Copy link
Collaborator Author

romange commented May 2, 2024

i suggest checking memory malloc-stats as well

@adiholden
Copy link
Collaborator

microsoft/mimalloc#887

@romange
Copy link
Collaborator Author

romange commented May 7, 2024

the issue with bigger block allocation, 18488 160 65120 65120 65120 - old vs
22132 184 65280 65280 62560 - new

@romange
Copy link
Collaborator Author

romange commented May 7, 2024

One issue I found we have problems with mi_heap_malloc_aligned.

  1. 8 byte alignment should be a noop but in practice it can increase the effective size.
  2. with (size % alignment) == 0 we may also have the unjustifiable overhead.

This is not related to theversion update, but was discovered when checking for differences in used memory

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants