running into RAM limitations for freertos tasks

[brettdaman]

First of all, thank you for the excellent project. It's a great library with tons of options, well done.

We have a large project that runs on a stm32 with Freertos. We use littlefs to store persistent variables and files. This has worked well for a couple of years until recently we ran into very slow lfs_renames.

Together, with removing lfs_rename we also updated littleFS to the latest version.

This is throwing some issues because some operations require a lot more stack than before. We need to bump each task that stores some variable in littlefs.

How would you structure the littlefs integration if you have for example 30 Freertos tasks that all require storing a unique single value in littlefs? The current solution to store the variable directly from the tasks is not workable, because each task requires a stack increase of +2000 bytes to be able to open the files.

Is there a solution in littlefs that we are missing? Our solution might be to write a caching layer in a task that contains littlefs to a single task.

Does anyone with a similar problem already fixed this?

[geky]

Hi @brettdaman, thanks for creating an issue.

Out of curiosity, what version did you upgrade from? I may be forgetting something, but I don't remember a recent version bump with a significant stack increase. Do you think this was a sort of death-by-a-thousand-cuts situation?

How would you structure the littlefs integration if you have for example 30 Freertos tasks that all require storing a unique single value in littlefs? The current solution to store the variable directly from the tasks is not workable, because each task requires a stack increase of +2000 bytes to be able to open the files.

This does seem challenging, and an unfortunate downside of littlefs's stack-heavy implementation. The tradeoff being better RAM utilization in simpler single-stack systems.

It does sounds like the best option involves a single storage task that other tasks communicate with to get work done. If you block the other tasks, this could work by passing references to buffers to write from/read into. But it may be easier to implement the cross-task communication with application-specific queries.

---

You could imagine a struct-heavy implementation of littlefs that moves more of the on-stack state into the lfs_t struct. But you would still be blocking tasks because littlefs isn't really multi-threaded internally (and ultimately IO would force serialization), and it would be more difficult to make sure RAM is reused effectively across littlefs's internal functions.

That being said, an implementation that synchronizes tasks via the internal lfs_dir_commit sync point could be very interesting...