TLSF memory allocator. Less free flash, moar free ram.

[DrZlo13]

What's new

• Heap4 allocator is replaced by tlsf.
• Moar free heap, less free flash.
• Updater: properly read large update file.

Verification

• Run unit_tests.
• In general, everything is affected.
• Update from any previous version to this one compiled with LIB_DEBUG=1 will not work (updater bug, see cab517f)
• Update from this version to this one compiled with LIB_DEBUG=1 will work

Checklist (For Reviewer)

• PR has description of feature/bug or link to Confluence/Jira task
• Description contains actions to verify feature/bugfix
• I've built this code, uploaded it to the device and verified feature/bugfix

[github-actions]

PVS-Studio report for commit 3a4bff82:

• Report

[github-actions]

Compiled f7 firmware for commit cab517fb:

• 📦 Update package
• 📥 DFU file
• ☁️ Web/App updater
• 📊 Size report

[CookiePLMonster]

Could maybe aligned_malloc be removed in favour of aligned_alloc while you're breaking API anyway? Less symbols and coincidentally also better conformance with the C standard, especially now that non-standard aligned_free is gone:
https://en.cppreference.com/w/c/memory/aligned_alloc

Or does definining standard C symbols come with other gotchas?

EDIT:
The order of parameters is swapped between the two, that's nasty 😑

[Willy-JL]

perhaps as suggested in #3574 and on discord contributor chat, could have along malloc() also a malloc_nothrow() that does not crash on size=0 or out of memory? i get wanting to keep strict standards to make sure things dont go wrong due to the nature of embedded systems, but also maybe it should be availableas an option. not the default, but available if necessary

[CookiePLMonster]

perhaps as suggested in #3574 and on discord contributor chat, could have along malloc() also a malloc_nothrow() that does not crash on size=0 or out of memory? i get wanting to keep strict standards to make sure things dont go wrong due to the nature of embedded systems, but also maybe it should be availableas an option. not the default, but available if necessary

Ever since we discussed it, I had a realization. Realistically, this would only work for very temporary allocations, maybe even with the scheduler disabled, for a simple reason - if the OS is so close to OOM so a nothrow allocator fails (gracefully), one of the two can happen:

1. nothrow allocator succeeds, but now we're near OOM, and something else in the firmware is likely going to try allocate memory soon-ish and crash.
2. nothrow allocator doesn't succeed because we're past OOM. The same scenario happens - something else in the firmware is likely going to crash soon, and the nothrow allocator has only bought us some time.

Or am I missing some scenario, or overestimating how many "random" allocations the firmware is doing when doing its thing?

[CookiePLMonster]

Max heap size check is prone to a race though, unless the program checks the heap size and mallocs with the memory manager locked. A nothrow alloc abstracts this detail away from the program.

That said, if the program needs to be memory-layout-aware enough to check for the max heap size, it can also just lock the heap for this operation, especially since the heap lock is recursive, so allocating with a held lock is OK.

EDIT: Nevermind, heap lock/unlock are not public symbols.

[DrZlo13]

@Willy-JL @CookiePLMonster
First of all, I personally don't see how malloc_nothrow can be used at all. Iterate with decreasing size until it succeeds?
Second, I don't understand how it can be used safely. You need to leave something around 20k for the OS.
Third, we prefer not to give a loaded gun to anyone who is not smart enough to make and load it themselves.

(Hint: you can use block_walk and malloc while the kernel is stopped and even inside the critical section. Do not forget to start the kernel back, or exit from critical section to properly shoot your own leg off).

[CookiePLMonster]

(Hint: you can use block_walk and malloc while the kernel is stopped and even inside the critical section. Do not forget to start the kernel back, or exit from critical section to properly shoot your own leg off).

Yeah, that's what I meant by locking the heap, as memmgr_lock and _unlock are effectively that. Suspending/resuming tasks is re-entrant so it's safe to suspend it twice (once from the user code, then again from malloc).

Broken update

[Sameesunkaria]

💛