Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve genesis state performance at startup #6977

Draft
wants to merge 20 commits into
base: main
Choose a base branch
from

Conversation

fab-10
Copy link
Contributor

@fab-10 fab-10 commented Apr 22, 2024

PR description

Fixed Issue(s)

Thanks for sending a pull request! Have you done the following?

  • Checked out our contribution guidelines?
  • Considered documentation and added the doc-change-required label to this PR if updates are required.
  • Considered the changelog and included an update if required.
  • For database changes (e.g. KeyValueSegmentIdentifier) considered compatibility and performed forwards and backwards compatibility tests

Locally, you can run these tests to catch failures early:

  • unit tests: ./gradlew build
  • acceptance tests: ./gradlew acceptanceTest
  • integration tests: ./gradlew integrationTest
  • reference tests: ./gradlew ethereum:referenceTests:referenceTests

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
@lyfsn
Copy link
Contributor

lyfsn commented Apr 23, 2024

Hey, I tested the current code; here are some records:

Status Time Memory Allocated
first start (database empty) 14 mins 658 GB
second start (database not empty) 30 s 2.25 GB

Before your refactor, the first start of Besu needed only 7 mins and 186 GB of memory allocated.
The step in there almost completely occupies 645 GB now: https://github.com/fab-10/besu/blob/758adc7d7468e52e17942d3d221116a0ecdda008/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/chain/GenesisState.java#L125

The test record is in the jfr record file; you can load it directly using IntelliJ IDEA: https://github.com/lyfsn/besu-starter/blob/main/g-1.jfr

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
@fab-10 fab-10 force-pushed the refactor-genesis-file-options branch from 08ab687 to 0889ee4 Compare April 24, 2024 16:59
# Conflicts:
#	acceptance-tests/dsl/src/main/java/org/hyperledger/besu/tests/acceptance/dsl/node/ThreadBesuNodeRunner.java
#	besu/src/main/java/org/hyperledger/besu/cli/BesuCommand.java
#	besu/src/main/java/org/hyperledger/besu/cli/config/EthNetworkConfig.java
#	besu/src/main/java/org/hyperledger/besu/controller/BesuController.java
#	besu/src/main/java/org/hyperledger/besu/controller/BesuControllerBuilder.java
#	besu/src/main/java/org/hyperledger/besu/controller/TransitionBesuControllerBuilder.java
#	besu/src/test/java/org/hyperledger/besu/PrivacyReorgTest.java
#	besu/src/test/java/org/hyperledger/besu/RunnerTest.java
#	besu/src/test/java/org/hyperledger/besu/cli/BesuCommandTest.java
#	besu/src/test/java/org/hyperledger/besu/cli/CascadingDefaultProviderTest.java
#	besu/src/test/java/org/hyperledger/besu/cli/config/EthNetworkConfigTest.java
#	besu/src/test/java/org/hyperledger/besu/controller/BesuControllerTest.java
#	config/src/main/java/org/hyperledger/besu/config/GenesisConfigFile.java
#	config/src/test/java/org/hyperledger/besu/config/JsonUtilTest.java
#	consensus/merge/src/test/java/org/hyperledger/besu/consensus/merge/blockcreation/MergeGenesisConfigHelper.java
#	consensus/qbft/src/integration-test/java/org/hyperledger/besu/consensus/qbft/support/TestContextBuilder.java
#	ethereum/core/src/test/java/org/hyperledger/besu/ethereum/trie/diffbased/bonsai/AbstractIsolationTests.java
@fab-10 fab-10 force-pushed the refactor-genesis-file-options branch from 9ca104c to b3a8978 Compare May 2, 2024 15:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants