EVM: bump opcode coverage + add performance tester #3198
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files
Flags with carried forward coverage won't be shown. Click here to find out more. |
|
This looks great but we should merge #3198 before merging this one. It's not obvious to me but are these opcode tests also run by CI? |
acolytec3
reviewed
Dec 18, 2023
| const code = makeLoopCode( | ||
| createBytecode([createOpcodeTest(test.stack, opcodeName, 'none'), POP]) | ||
| ) | ||
| const evm = new EVM({ common, profiler: { enabled: true } }) |
There was a problem hiding this comment.
Suggested change
| const evm = new EVM({ common, profiler: { enabled: true } }) | |
| common.events.removeAllListeners() |
Since we're reusing the same common each time, we should clear listeners because the evm adds a new listener here and I see memory leak errors when I run the profiler script.
…reumjs-monorepo into evm-coverage-performance
…reumjs-monorepo into evm-coverage-performance
|
@jochem-brouwer What is the status of this PR? |
|
I intend to finish this one this month. The goal is to write basic tests for all opcodes and provide a test framework. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The goal of this PR is to:
The goal of this PR is to create both simple opcode tests (compare stack/memory/storage output to expected code), with transparant bytecode generation. Some (all?) of these tests are also used in the profiler report, which tries edge cases (adding maximal numbers together, exponentation to the max, etc.) and reports MGas/S output of those. This can be used to spot slow/fast variants of certain opcodes, or identify which opcodes need some optimization.
Sample profile report output:
To run the tests, run
npm run profiler:report. You need to havetsxinstalled for this, sonpm i -g tsx. Note: the default gas limit for each test is 30M, and currently all opcodes keep looping until this gas limit is reached (this is slow). There is a progress report though so one can track how much time it should take to finish.WIP