Default bonsai to fully flat db and code storage by codehash #6894

garyschulte · 2024-04-06T05:02:24Z

PR description

This PR changes the default storage mode of bonsai to use code storage by code hash, and uses flat db by default. This supports the future direction of bonsai, and simplifies the configuration necessary for besu to serve snap protocol (see description on #6640, now just --Xsnapsync-server-enabled is necessary).

changes:

defaults bonsai to using a fully flattened db
defaults bonsai to using code stored by code hash
deprecates snap sync full flat config param
creates --Xbonsai-full-flat-db-enabled parameter
FlatDbStrategyProvider defers config if db is empty and persists a flat db strategy config

tested with:

full sync, correctly creates full flattened db
snap sync, correctly creates full flattend db, correctly heals flat db
existing holesky sync, with FULL flat db and code by CODE_HASH
- starting with config for partial db and code by account hash correctly defers to db config and logs actual
existing 24.3.0 holesky with PARTIAL flat db and code by ACCOUNT_HASH
- starting with configs for full and code by code hash correctly defers to db config and logs actual

Fixed Issue(s)

Thanks for sending a pull request! Have you done the following?

Checked out our contribution guidelines?
Considered documentation and added the doc-change-required label to this PR if updates are required.
Considered the changelog and included an update if required.
For database changes (e.g. KeyValueSegmentIdentifier) considered compatibility and performed forwards and backwards compatibility tests

Locally, you can run these tests to catch failures early:

unit tests: ./gradlew build
acceptance tests: ./gradlew acceptanceTest
integration tests: ./gradlew integrationTest
reference tests: ./gradlew ethereum:referenceTests:referenceTests

garyschulte · 2024-04-22T21:10:18Z

It is worth noting there is a side effect of this PR on worldstate fault heal/recovery. If a db with a FULL flat db strategy encounters an inconsistent worldstate db, as part of fixing the worldstate, the database will be left with a PARTIAL flat db strategy after fixing the worldstate.

We can re-enable flat db after the healing, but this side effect might actually be desirable.

matkt

added small comments

matkt · 2024-04-23T08:14:41Z

besu/src/test/resources/everything_config.toml

@@ -217,6 +217,7 @@ receipt-compaction-enabled=true
 # feature flags
 Xsecp256k1-native-enabled=false
 Xaltbn128-native-enabled=false
+Xsnapsync-server-enabled=true


we should not add the new full flat db flag ?

matkt · 2024-04-23T08:22:33Z

...t/java/org/hyperledger/besu/ethereum/trie/diffbased/bonsai/BonsaiSnapshotIsolationTests.java

@@ -152,6 +152,8 @@ public void assertCloseDisposesOfStateWithoutCommitting() {
      assertThat(res.isSuccessful()).isTrue();
      shouldCloseSnapshot.persist(oneTx.getHeader());

+      // TODO: full flat strategy will not be updated with these values, only the trie
+      //       determine how we want to make this assertion for full flat


I think we can use an updater in order to fix the test

I thought about it and realized it will degrade the performance of the 'normal' use case for frozen/cached worldstates. I think this note in the test superclass should be sufficient to explain why we are using PARTIAL for the test - removed from TODO.

matkt · 2024-04-23T08:23:29Z

...org/hyperledger/besu/ethereum/trie/diffbased/common/storage/flat/FlatDbStrategyProvider.java

    var flatDbMode =
        FlatDbMode.fromVersion(
            composedWorldStateStorage
                .get(TRIE_BRANCH_STORAGE, FLAT_DB_MODE)
                .map(Bytes::wrap)
-                .orElse(FlatDbMode.PARTIAL.getVersion()));
+                .orElseGet(


matkt · 2024-04-23T08:24:03Z

...org/hyperledger/besu/ethereum/trie/diffbased/common/storage/flat/FlatDbStrategyProvider.java

@@ -71,12 +72,36 @@ public void loadFlatDbStrategy(final SegmentedKeyValueStorage composedWorldState

  @VisibleForTesting
  FlatDbMode deriveFlatDbStrategy(final SegmentedKeyValueStorage composedWorldStateStorage) {
+    final FlatDbMode requestedFlatDbMode =
+        dataStorageConfiguration.getUnstable().getBonsaiFullFlatDbEnabled()


is this flag still in unstable ?

it is, but the generated classes have to be rebuilt to see it. Pretty much all of the bonsai configs are in unstable, we should move them all out of unstable when we change the default to bonsai 👍

matkt · 2024-05-02T07:45:45Z

.../src/main/java/org/hyperledger/besu/ethereum/eth/sync/snapsync/SnapWorldStateDownloader.java

@@ -183,16 +183,6 @@ public CompletableFuture<Void> run(
      } else {
        // start from scratch
        worldStateStorageCoordinator.clear();
-        // we have to upgrade to full flat db mode if we are in bonsai mode
-        if (snapSyncConfiguration.isFlatDbHealingEnabled()) {


we don't have to downgradeToPartialFlatDbMode if a user disable the full flat flag ? as the default is full now

that is a good point. I hadn't considered that anybody would want to run in a partial mode. I will double check, but the storage should be created in the correct/requested mode, whether that is partial or full.

We can prevent user from doing that. It would not be logic

Signed-off-by: garyschulte <garyschulte@gmail.com>

…torageOption for --Xbonsai-full-flat-db-enabled Signed-off-by: garyschulte <garyschulte@gmail.com>

Signed-off-by: garyschulte <garyschulte@gmail.com>

garyschulte requested review from jframe and matkt April 6, 2024 05:02

garyschulte force-pushed the feature/default-flat-db-and-codehash-storage branch 2 times, most recently from 7c12480 to d201d64 Compare April 22, 2024 17:57

garyschulte marked this pull request as ready for review April 22, 2024 18:05

matkt reviewed Apr 23, 2024

View reviewed changes

garyschulte force-pushed the feature/default-flat-db-and-codehash-storage branch 3 times, most recently from db32129 to b16d74d Compare May 1, 2024 16:16

matkt reviewed May 2, 2024

View reviewed changes

garyschulte force-pushed the feature/default-flat-db-and-codehash-storage branch from b16d74d to 11a8d25 Compare May 2, 2024 20:59

matkt approved these changes May 3, 2024

View reviewed changes

garyschulte added 9 commits May 3, 2024 16:00

change to full flat db and code stored by code hash by default

185f102

Signed-off-by: garyschulte <garyschulte@gmail.com>

deprecate --Xsnapsync-synchronizer-flat-db-healing-enabled, add DataS…

1093a31

…torageOption for --Xbonsai-full-flat-db-enabled Signed-off-by: garyschulte <garyschulte@gmail.com>

fix flat db and code storage tests to reflect new defaults

9367070

Signed-off-by: garyschulte <garyschulte@gmail.com>

fix bonsai isolation test assertions, workaround

5943456

Signed-off-by: garyschulte <garyschulte@gmail.com>

use arity to solve surprising toggle behavior for picocli boolean

25596cc

Signed-off-by: garyschulte <garyschulte@gmail.com>

add full/partial strategy set info logs

1cf6a64

Signed-off-by: garyschulte <garyschulte@gmail.com>

fix test fixture

5e67090

Signed-off-by: garyschulte <garyschulte@gmail.com>

feedback from Karim's review

120f331

Signed-off-by: garyschulte <garyschulte@gmail.com>

boxed type for flag

f7055ec

Signed-off-by: garyschulte <garyschulte@gmail.com>

garyschulte force-pushed the feature/default-flat-db-and-codehash-storage branch from 11a8d25 to f7055ec Compare May 3, 2024 23:00

add changelog entry

4be991e

Signed-off-by: garyschulte <garyschulte@gmail.com>

garyschulte enabled auto-merge (squash) May 3, 2024 23:24

garyschulte merged commit 0a24acc into hyperledger:main May 3, 2024
42 checks passed

garyschulte deleted the feature/default-flat-db-and-codehash-storage branch May 4, 2024 00:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Default bonsai to fully flat db and code storage by codehash #6894

Default bonsai to fully flat db and code storage by codehash #6894

garyschulte commented Apr 6, 2024 •

edited

garyschulte commented Apr 22, 2024

matkt left a comment

matkt Apr 23, 2024

matkt Apr 23, 2024

garyschulte Apr 29, 2024 •

edited

matkt Apr 23, 2024

matkt Apr 23, 2024

garyschulte Apr 29, 2024 •

edited

matkt May 2, 2024

garyschulte May 2, 2024 •

edited

matkt May 2, 2024

Default bonsai to fully flat db and code storage by codehash #6894

Default bonsai to fully flat db and code storage by codehash #6894

Conversation

garyschulte commented Apr 6, 2024 • edited

PR description

Fixed Issue(s)

Thanks for sending a pull request! Have you done the following?

Locally, you can run these tests to catch failures early:

garyschulte commented Apr 22, 2024

matkt left a comment

Choose a reason for hiding this comment

matkt Apr 23, 2024

Choose a reason for hiding this comment

matkt Apr 23, 2024

Choose a reason for hiding this comment

garyschulte Apr 29, 2024 • edited

Choose a reason for hiding this comment

matkt Apr 23, 2024

Choose a reason for hiding this comment

matkt Apr 23, 2024

Choose a reason for hiding this comment

garyschulte Apr 29, 2024 • edited

Choose a reason for hiding this comment

matkt May 2, 2024

Choose a reason for hiding this comment

garyschulte May 2, 2024 • edited

Choose a reason for hiding this comment

matkt May 2, 2024

Choose a reason for hiding this comment

garyschulte commented Apr 6, 2024 •

edited

garyschulte Apr 29, 2024 •

edited

garyschulte Apr 29, 2024 •

edited

garyschulte May 2, 2024 •

edited