Blocks && fully spent transactions pruning #510
Conversation
svyatonik
added
A3-inprogress
db/src/block_chain_db.rs
Outdated
| @@ -397,7 +501,7 @@ impl<T> BlockHeaderProvider for BlockChainDatabase<T> where T: KeyValueDatabase | |||
| } | |||
|
|
|||
| impl<T> BlockProvider for BlockChainDatabase<T> where T: KeyValueDatabase { | |||
| fn block_number(&self, hash: &H256) -> Option<u32> { | |||
| fn block_number(&self, hash: &H256) -> Option<u32> { | |||
| @@ -13,6 +13,7 @@ pub const COL_TRANSACTIONS: u32 = 4; | |||
| pub const COL_TRANSACTIONS_META: u32 = 5; | |||
| pub const COL_BLOCK_NUMBERS: u32 = 6; | |||
| pub const COL_CONFIGURATION: u32 = 7; | |||
| pub const COL_SPENT_TRANSACTIONS: u32 = 8; | |||
There was a problem hiding this comment.
Note to myself (and possibly others reviewing this code): we have COL_COUNT preset to 10 so no db migration needed for this change. 🎉
There was a problem hiding this comment.
Yes. I even had an idea to keep this in-memory only - so if node doesn't restarts while syncing, all fully spent transactions will be pruned. But in the case of restart, we could miss some fully spent transactions => 'garbage' in db. Not sure - what's better in general, but for my use-cases (I'm often restarting the node), the permanent storage is better.
svyatonik
added
A4-gotissues
|
Two issues:
|
svyatonik
added
A0-pleasereview
svyatonik
added
A3-inprogress
|
Seems like this patch is not as effective as it intended to be - probably because number of fully spent transactions is not that big. After synchronizing ~310000 blocks, db size is almost the same. And most of space is still occupied by tx && tx meta columns. I'll try to change db structure by extracting tx outputs to separate columns && reopen PR when results will be ready. |
|
To me in the future: last commit of this branch changes size of db to ~12GB at block 416614 - it is > x10 times smaller than before. Still have to check another option (single tx out column with spent/unspent sign) + adjust config options + check that performance (tools/bencher) is not significantly affected. |
closes #472
So the idea is:
BlockHeaderandBlockTransactionsof ancient blocks (blocks from canon chain with age > 2048);TransactionandTransactionMetaof transaction, when block, spending the last utxo of this transaction became ancient.prune-modecli option can take 3 values now:"none" - no pruning;
"header" - 1 from ^^^;
"full" - 1+2 from ^^^.
I'll leave one of chains (
--btcor--bch) syncing to see the final db size && post it here.