BUG: periodically saving collection bloats db storage size

[edwinkys]

Description

This bug happens when saving a collection in a periodic manner like the example below.

# Create the DB and collection...
for _ in range(300):
    records = Record.many_random(dimension=512, len=100)
    collection.insert_many(records)
    db.save_collection("storage", collection)

This caused unflushed dirty IO buffers to bloats the database folder to an amount unimaginable 🤯
From 70MB database size to 10GB.

Read the discussion #77 for more context about what is happening.

[edwinkys]

The methods added in #84, Database.flush and Database.async_flush don't solve this issue. It seems that the correct approach to fix this issue is to store the collection in it's own file and only use Sled to store the reference to the file.