Allow block_size/block_count to be optional, add erase_count, lfs_fs_stat, and lfs_fs_grow

[geky]

This PR moves block_size/block_count into RAM, allowing more flexibility during lfs_mount.

The main benefit is allowing block_size and block_count to be marked as "unknown", using the newly added erase_count to read block_size and block_count from the superblock during lfs_mount.

Unfortunately this isn't as simple as it sounds. In littlefs, as a part of enabling atomic updates, the superblock can live in either block 0 or block 1. But if we don't know the block_size, we don't really know where block 1 begins. This can put us in a catch-22 situation where we need the block_size in order to read the block_size from the superblock.

The solution proposed here is to search possible block_sizes until we find the superblock, optionally using the size of the disk to narrow down our search. Surprisingly, this adds no runtime if we find a littlefs image, but unfortunately has some complicated performance implications if the disk in unformatted:

1. .block_size = BLOCK_SIZE,
   .block_count = BLOCK_COUNT,
   

   If we know the block_size and block_count, we can easily mount the filesystem, erroring with LFS_ERR_INVAL if the block_size/block_count stored in the superblock are different. Worst case $O(1)$.

2. .block_size = BLOCK_SIZE,
   .block_count = 0, // unknown
   

   If we know the block_size, we can easily mount the filesystem and read the block_count from the superblock. Worst case $O(1)$.

3. .block_size = 0, // unknown
   .block_count = DISK_SIZE / ERASE_SIZE,
   

   If we don't know the block_size, we need to search for it. Two shortcuts: 1. we know the block_size must be a multiple of the erase_size, and 2. if we know the block_count, we can assume the block_size divides the block_count evenly. Worst case $O(d(n))$.

4. .block_size = 0, // unknown
   .block_count = 0, // unknown
   

   If we don't know the block_count, or can't rely on the above assumption, the best we can do is search all multiples of the erase_size. This risks taking a long time on large, unformatted disks. Worst case $O(n)$.

The above function, $d(n)$, is the "divisor function", and it's analysis is a bit difficult. On average, and for all powers-of-two, it runs in $O(\log n)$. But there are so called "highly composite numbers" where it can take at worst $O(\sqrt n)$.

#349 has more discussion about this scheme and alternatives.

I put together a quick benchmark to measure how long lfs_mount takes to fail as the disk grows. We can see the sub-linear runtime if we know the block_count, and linear runtime if both block_size and block_count are unknown. If we know the block_size, lfs_mount reads only ~136 bytes:

Choosing a random device, mt25q, shows a max of ~90MiB/s reads in the datasheet. So for a very rough estimate:

	unformatted mount 1 GiB @ 90 MiB/s
known block_count	~125 us
known block_size	~1.44 us
known bs/bc	~1.44 us
unknown bs/bc	~376 ms

---

With block_size/block_count now variable, this PR also includes a couple relevant new functions:

• lfs_fs_stat

  lfs_fs_stat allows you to access the configuration read from littlefs's superblock during mount, as well as other state such as the number of blocks in use (susceptible to over-counting due to CoW).

  This is supposed to mirror statvfs common in other filesystems.

  struct lfs_fsstat fsstat;
  int err = lfs_fs_stat(lfs, &fsstat);
  if (err) {
      return err;
  }
  
  printf("block_size:  %d\n", fsstat.block_size);
  printf("block_count: %d\n", fsstat.block_count);
  printf("block_usage: %d\n", fsstat.block_usage); // same as lfs_fs_size
  printf("name_max:    %d\n", fsstat.name_max);
  printf("file_max:    %d\n", fsstat.file_max);
  printf("attr_max:    %d\n", fsstat.attr_max);

• lfs_fs_grow

  First implemented by @kaetemi, lfs_fs_grow allows you to change the size of the filesystem on disk to take advantage of additional space if it becomes available:

  cfg.block_count = 1024;
  
  int err = lfs_mount(lfs, cfg);
  if (err) {
      return err;
  }
  
  struct lfs_fsstat fsstat;
  err = lfs_fs_stat(lfs, &fsstat);
  if (err) {
      return err;
  }
  printf("block_count: %d\n", fsstat.block_count); // prints 1024
  
  err = lfs_fs_grow(lfs, 2048);
  if (err) {
      return err;
  }
  
  struct lfs_fsstat fsstat;
  err = lfs_fs_stat(lfs, &fsstat);
  if (err) {
      return err;
  }
  printf("block_count: %d\n", fsstat.block_count); // prints 2048

  If you want to automatically resize littlefs to the full disk size every mount, this can be done by using these two calls as your "mount" operation:

  lfs_size_t required_block_count = DISK_SIZE / BLOCK_SIZE;
  
  // unknown block_count
  cfg.block_count = 0;
  
  // mount and resize if disk has grown
  int err = lfs_mount(lfs, cfg);
  if (err) {
      return err;
  }
  
  err = lfs_fs_grow(lfs, required_block_count );
  if (err) {
      return err;
  }

Relevant discussions:

• Automatic volume recognition #349 - Allow block_size/block_count to be optional
• question: is it possible to "grow" the filesystem? #279 - lfs_fs_grow
• Grow block count to configured value when LFS_M_GROW mount flag is set #702 - @kaetemi's implementation of LFS_M_GROW

Note this PR is currently under discussion in #349.

[e107steved]

I expressed some concern about additional RAM usage in #349 - it could (at least at first glance) be a significant amount in the context of a small to medium processor that doesn't need the RAM-based config block.
What are the benefits (if any!) of littleFS doing the scan with different block sizes when needed, rather than leaving a user to set up the incredibly simple code loop and create a persistent config block in RAM?

[geky]

Don't mind me just copying the CI output here. I don't have a strong opinion yet on if this goes in or not:

Tests passed ✓, Code: 16334 B (+4.0%), Stack: 1344 B (+3.1%), Structs: 812 B (+5.2%)

	Code	Stack	Structs		Coverage
Default	16334 B (+4.0%)	1344 B (+3.1%)	812 B (+5.2%)	Lines	2293/2476 lines (+0.3%)
Readonly	6154 B (+7.4%)	400 B (+13.6%)	812 B (+5.2%)	Branches	1183/1498 branches (-0.4%)
Threadsafe	17178 B (+4.1%)	1344 B (+3.1%)	820 B (+5.1%)		Benchmarks
Migrate	18042 B (+3.8%)	1672 B (+2.0%)	816 B (+5.2%)	Readed	29303789027 B (+0.0%)
mkdir -p comment
Error-asserts	17090 B (+4.7%)	1344 B (+3.7%)	812 B (+5.2%)	Proged	1482456139 B (+0.0%)
				Erased	1568290304 B (+0.0%)

[geky]

Just an FYI for anyone watching this PR, at the moment I'm considering this PR defunct.

lfs_fs_stat has been merged in #838, and with any luck, unknown block-counts will be merged in #866, and lfs_fs_grow in #872.

The only remaining feature is the detection of unknown block-sizes, but after taking some time away from the problem, I'm not sure detecting unknown block-sizes based on the assumption of the total disk size being divisible is very reasonable in practice. And as @e107steved notes above, it's possible to do the naive search for block-size outside of the filesystem, so I will be leaving this feature out for now.

At the moment it looks like block-size will always be a required configuration option.