After submitting 4000 transactions, they slowly trickle into transaction_pool. See if this can be improved

[stringhandler]

To replicate:
Submit 4000 transactions
The transaction_submitter ends quickly.
Then repeatedly run

select count(*) as mempool, sum(case when stage = 'AllPrepared' then 1 else 0 end) as all_prepared,
       sum(case when stage = 'LocalPrepared' then 1 else 0 end) as local_prepared,
         sum(case when stage = 'Prepared' then 1 else 0 end) as prepared,
            sum(case when stage = 'New' then 1 else 0 end) as new
from transaction_pool;

Expected:
All 4000 should be there in New

Actual:
Transactions appear in New (count goes up) at a rate of about 5 txs a second

Was tested using 5 vns on dan-testing

Cifko:
I haven't encountered the issue above, but the 4000 transactions was slow. In total (for all 4000 txs)

• 122 seconds was the inserting into the DB
  • 8 second out of it is the creation of the db row,
  • less than one second is hex and json conversion
  • the rest is the tx.commit
• 58 seconds was gossiping

So on average the whole transaction is 0.046 seconds (~22 tx/sec).

[stringhandler]

seems like sqlite is the limit here. We use sqlite for easy debugging. Perhaps it's time to try a more performant db