Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

com.intel.pmem.llpl.HeapException: Failed to allocate memory of size #13

Open
Reidddddd opened this issue Jul 7, 2022 · 3 comments
Open
Assignees

Comments

@Reidddddd
Copy link

com.intel.pmem.llpl.HeapException: Failed to allocate memory of size 2097152
        at com.intel.pmem.llpl.MemoryAccessor.lambda$new$0(MemoryAccessor.java:65)
        at com.intel.pmem.llpl.MemoryAccessor.<init>(MemoryAccessor.java:77)
        at com.intel.pmem.llpl.AnyMemoryBlock.<init>(AnyMemoryBlock.java:34)
        at com.intel.pmem.llpl.AbstractMemoryBlock.<init>(AbstractMemoryBlock.java:16)
        at com.intel.pmem.llpl.MemoryBlock.<init>(MemoryBlock.java:25)
        at com.intel.pmem.llpl.Heap.allocateMemoryBlock(Heap.java:226)
        at org.apache.hadoop.hbase.regionserver.PersistentMemstoreLAB.allocateBytes(PersistentMemstoreLAB.java:102)
        at org.apache.hadoop.hbase.regionserver.TestPersistentMemstoreLAB.testLABLargeAllocation(TestPersistentMemstoreLAB.java:74)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)

I got this exception when I tested HBase, using LLPL.
The Heap size was 8MB on my server.

Use growing heap: Heap.createHeap(heapPath, heapSize);

Any hints about where went wrong?

@asonje asonje self-assigned this Jul 13, 2022
@asonje
Copy link
Contributor

asonje commented Jul 13, 2022

Hello, it looks like you have run of memory on the persistent heap. if you trying to allocate multiple 2MB memory blocks then you may need to increase the heapSize value. Setting the heapSize to 8MB, for example, means the heap will not grow beyond 8MB. You will however get less than 8MB of allocatable memory. This is a characteristic of the underlying memory allocator

@Reidddddd
Copy link
Author

Reidddddd commented Jul 14, 2022

Hello,

if you trying to allocate multiple 2MB memory blocks

I was trying allocate one 2MB block

means the heap will not grow beyond 8MB.

Emm, learning from the java docs, I was using the growing heap...

More background: I was writing a UT, and one of method was trying to test a large allocation (this issue, 2MB). And after each method, I would free(false) the MemoryBlock.
So, my understanding was for each method, the Heap should contain enough memory (at least enough for 2MB), but as the exception pasted...

@asonje
Copy link
Contributor

asonje commented Jul 19, 2022

i wrote a small test on an 8MB growable heap does the following while (true) { mb = allocateMemoryBlock(2MB, false); mb.free(false); } This will run forever.
perhaps your unit test does not always free the memory block after each test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants