Can't open checkpoint after a program crash

[ozataman]

Here is how my database directory looks:

After a program crash due to an unrelated bug, my program can no longer open/load the database, giving me this error:

myapp: db/checkpoints-0000000106.log: hGetBuf: invalid argument (Invalid argument)

It may be of note that:

• I call 'createArchive' right after loading the database for the first time during application boot.
• I call createSnapshot quite often (every minute or so). The program is a continuously running and loopin data collection/computation application.
• While the total data is likely well under 1GB, the workload is very update-heavy with thousands of (some large-ish) updates flying around every minute.
• I've noticed my checkpoints get larger and larger. Aren't they supposed to only contain the data itself without any history?

I'd appreciate any direction as far as handling this situation goes. Unfortunately, I will have to switch to something like redis if we can't be sure of acid-state's reliability under this kind of workload.

[lemmih]

It seems something went wrong with the 106th checkpoint. The size of the state went from ~50MB to >21GB. Perhaps you have an infinite structure in your state?

Try deleting checkpoint 106. You should then be able to load the state and inspect it.

The only data AcidState adds to the checkpoints are a simple checksum. All the rest is completely your data.

[ozataman]

Huh. Interesting. Thanks for the feedback. I did so and the program went back to normal operation. I'll have to keep an eye on this. I definitely don't have infinite data structures in there, but some parts certainly could be growing larger than expected.

Does each checkpoint file guarantee to only keep a single full copy of the database and nothing else? If my checkpoint file is, say 100MB, does that mean I roughly have 100MB worth of data in the state?

[lemmih]

No, multiple checkpoints can be written to the same file. If you run 'createArchive' directly after 'createCheckpoint' then the checkpoint file should only contain a single state snapshot.