New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[seqbench] the-good-parts-mk2
sequence (on Cancun)
#15031
Conversation
…didate - `the-good-parts` variant found using seqbench. - second refinement, based on the erc20.sol contract.
|
File name | IR optimized | Legacy optimized | Legacy |
---|---|---|---|
array/create_memory_array.sol |
9% | ||
array/array_storage_push_empty_length_address.sol |
8% | ||
externalContracts/base64.sol |
8% | -0% | |
array/array_storage_index_boundary_test.sol |
7% | ||
array/delete/bytes_delete_element.sol |
7% | -0% | |
array/copying/bytes_storage_to_storage.sol |
6% | -0% | |
array/copying/cleanup_during_multi_element_per_slot_copy.sol |
6% | ||
array/copying/storage_memory_packed_dyn.sol |
5% | ||
array/byte_array_transitional_2.sol |
5% | ||
array/copying/array_copy_cleanup_uint40.sol |
5% | ||
abiEncoderV2/calldata_array.sol |
3% | ||
array/array_storage_push_pop.sol |
3% | ||
inheritance/inherited_function_calldata_memory_interface.sol |
3% | ||
abiEncoderV2/abi_encode_calldata_slice.sol |
3% | -0% | |
abiEncoderV1/abi_encode_calldata_slice.sol |
3% | -0% | |
array/array_storage_push_empty.sol |
2% | ||
array/array_storage_index_zeroed_test.sol |
2% | ||
isoltestTesting/balance_other_contract.sol |
2% | ||
externalContracts/prbmath_unsigned.sol |
2% | ||
array/dynamic_arrays_in_storage.sol |
2% | ||
externalContracts/FixedFeeRegistrar.sol |
1% | +0% | |
various/erc20.sol |
1% | +0% | |
events/event_indexed_string.sol |
1% | -0% | |
abiEncoderV2/abi_encode_v2_in_function_inherited_in_v1_contract.sol |
1% | -0% | |
array/fixed_arrays_as_return_type.sol |
1% | -0% | |
array/push/byte_array_push_transition.sol |
1% | ||
various/selfdestruct_pre_cancun_multiple_beneficiaries.sol |
1% | ||
various/selfdestruct_post_cancun_multiple_beneficiaries.sol |
1% | ||
userDefinedValueType/erc20.sol |
1% | +0% | |
array/copying/array_of_struct_calldata_to_storage.sol |
1% | ||
array/copying/array_copy_clear_storage.sol |
1% | ||
array/copying/memory_dyn_2d_bytes_to_storage.sol |
1% | -0% | |
constructor/bytes_in_constructors_packer.sol |
+0% | 1% | |
events/event_dynamic_array_storage_v2.sol |
1% | +0% | |
events/event_dynamic_array_storage.sol |
1% | +0% | |
array/array_storage_index_access.sol |
1% | ||
externalContracts/prbmath_signed.sol |
1% | ||
various/value_complex.sol |
1% | ||
various/value_insane.sol |
+0% | ||
salted_create/salted_create_with_value.sol |
+0% | ||
byte_array_to_storage_cleanup.sol |
1% | -0% | |
array/push/array_push.sol |
+0% | ||
structs/memory_structs_nested_load.sol |
+0% | ||
externalContracts/snark.sol |
+0% | ||
array/copying/array_of_struct_memory_to_storage.sol |
+0% | ||
array/pop/byte_array_pop_long_storage_empty.sol |
+0% | ||
array/copying/storage_memory_nested_from_pointer.sol |
+0% | ||
array/copying/storage_memory_nested.sol |
+0% | ||
various/selfdestruct_post_cancun_redeploy.sol |
+0% | ||
various/selfdestruct_pre_cancun_redeploy.sol |
+0% | ||
array/copying/array_copy_storage_storage_different_base.sol |
+0% | ||
array/copying/copy_byte_array_to_storage.sol |
+0% | -0% | |
array/push/push_no_args_bytes.sol |
+0% | ||
structs/struct_memory_to_storage_function_ptr.sol |
+0% | ||
array/copying/array_copy_storage_storage_dynamic_dynamic.sol |
+0% | ||
structs/struct_delete_storage_with_array.sol |
+0% | ||
array/push/nested_bytes_push.sol |
+0% | -0% | |
array/constant_var_as_array_length.sol |
+0% | +0% | |
array/copying/array_of_structs_containing_arrays_calldata_to_storage.sol |
+0% | ||
array/dynamic_multi_array_cleanup.sol |
+0% | ||
structs/struct_copy_via_local.sol |
+0% | ||
externalContracts/deposit_contract.sol |
+0% | +0% | |
array/push/array_push_struct.sol |
+0% | ||
abiEncoderV1/struct/struct_storage_ptr.sol |
+0% | ||
libraries/using_library_mappings_public.sol |
+0% | ||
array/fixed_arrays_in_constructors.sol |
+0% | ||
array/copying/array_copy_different_packing.sol |
+0% | ||
constructor/constructor_arguments_external.sol |
+0% | ||
array/copying/nested_array_element_storage_to_storage.sol |
+0% | ||
viaYul/copy_struct_invalid_ir_bug.sol |
+0% | ||
userDefinedValueType/calldata.sol |
+0% | -0% | |
array/copying/array_copy_storage_to_memory_nested.sol |
+0% | +0% | |
storage/packed_storage_structs_bytes.sol |
+0% | ||
abiEncoderV2/abi_encode_v2.sol |
+0% | +0% | |
array/copying/calldata_array_dynamic_to_storage.sol |
+0% | -0% | |
array/dynamic_array_cleanup.sol |
+0% | ||
inheritance/value_for_constructor.sol |
+0% | ||
array/copying/array_nested_calldata_to_storage.sol |
+0% | ||
immutable/use_scratch.sol |
+0% | ||
constructor/constructor_static_array_argument.sol |
+0% | -0% | |
events/event_dynamic_nested_array_storage_v2.sol |
+0% | +0% | |
array/push/array_push_struct_from_calldata.sol |
+0% | -0% | |
various/destructuring_assignment.sol |
+0% | -0% | |
array/copying/array_copy_target_simple.sol |
+0% | ||
array/copying/array_copy_including_array.sol |
+0% | ||
functionCall/creation_function_call_with_salt.sol |
+0% | ||
array/copying/array_copy_storage_storage_different_base_nested.sol |
+0% | ||
array/copying/storage_memory_nested_struct.sol |
+0% | ||
array/copying/nested_array_of_structs_calldata_to_storage.sol |
+0% | ||
array/copying/elements_of_nested_array_of_structs_calldata_to_storage.sol |
+0% | ||
structs/copy_struct_array_from_storage.sol |
+0% | ||
functionCall/creation_function_call_with_args.sol |
+0% | ||
array/copying/array_nested_memory_to_storage.sol |
+0% | ||
types/mapping/copy_from_mapping_to_mapping.sol |
+0% | +0% | |
array/copying/nested_dynamic_array_element_calldata_to_storage.sol |
+0% | ||
array/fixed_array_cleanup.sol |
+0% | ||
array/copying/nested_array_of_structs_memory_to_storage.sol |
+0% | ||
array/copying/array_copy_storage_storage_struct.sol |
+0% | ||
array/copying/array_copy_storage_storage_static_dynamic.sol |
+0% | ||
abiencodedecode/abi_decode_simple_storage.sol |
+0% | -0% | |
structs/calldata/calldata_struct_with_nested_array_to_storage.sol |
+0% | +0% | |
array/bytes_length_member.sol |
+0% | ||
array/reusing_memory.sol |
+0% | ||
array/pop/array_pop_array_transition.sol |
+0% | ||
array/push/array_push_nested_from_calldata.sol |
+0% | -0% | |
array/copying/nested_array_of_structs_storage_to_storage.sol |
+0% | ||
array/copying/array_of_structs_containing_arrays_memory_to_storage.sol |
+0% | ||
abiEncoderV1/abi_decode_v2_storage.sol |
+0% | +0% | |
array/push/push_no_args_2d.sol |
+0% | ||
array/copying/array_storage_multi_items_per_slot.sol |
+0% | ||
structs/struct_copy.sol |
+0% | ||
abiEncoderV2/calldata_overlapped_dynamic_arrays.sol |
+0% | -0% | |
array/copying/copy_function_internal_storage_array.sol |
+0% | ||
array/copying/array_copy_storage_storage_dyn_dyn.sol |
+0% | ||
various/staticcall_for_view_and_pure.sol |
-0% | ||
array/array_storage_length_access.sol |
-0% | ||
calldata/copy_from_calldata_removes_bytes_data.sol |
-0% | ||
array/copying/array_copy_storage_storage_static_static.sol |
-0% | ||
array/copying/bytes_inside_mappings.sol |
-0% | ||
array/invalid_encoding_for_storage_byte_array.sol |
-0% | -0% | |
array/copying/array_of_function_external_storage_to_storage_dynamic_different_mutability.sol |
-0% | +0% | |
constructor/no_callvalue_check.sol |
-0% | ||
array/copying/copy_byte_array_in_struct_to_storage.sol |
-0% | -0% | |
array/copying/array_copy_nested_array.sol |
-0% | -0% | |
array/copying/elements_of_nested_array_of_structs_memory_to_storage.sol |
-0% | ||
array/array_memory_index_access.sol |
-0% | ||
array/copying/copy_removes_bytes_data.sol |
-0% | ||
various/many_subassemblies.sol |
-0% | ||
immutable/multi_creation.sol |
-0% | ||
constructor/bytes_in_constructors_unpacker.sol |
-0% | +0% | |
array/copying/calldata_array_to_mapping.sol |
-0% | ||
array/copying/array_to_mapping.sol |
-0% | +0% | |
array/copying/array_of_function_external_storage_to_storage_dynamic.sol |
-0% | -0% | |
array/copying/storage_memory_nested_bytes.sol |
-0% | +0% | |
structs/struct_containing_bytes_copy_and_delete.sol |
-0% | -0% | |
array/copying/array_copy_target_simple_2.sol |
-0% | ||
array/copying/function_type_array_to_storage.sol |
-0% | -0% | |
abiEncoderV2/storage_array_encoding.sol |
-0% | -0% | |
structs/copy_substructures_from_mapping.sol |
-0% | -0% | |
array/copying/array_elements_to_mapping.sol |
-0% | +0% | |
various/skip_dynamic_types_for_structs.sol |
-0% | -0% | |
storage/empty_nonempty_empty.sol |
-0% | -0% | |
structs/copy_to_mapping.sol |
-0% | -0% | |
structs/copy_from_mapping.sol |
-0% | -0% | |
array/pop/byte_array_pop_long_storage_empty_garbage_ref.sol |
-0% | ||
array/copying/array_copy_calldata_storage.sol |
-0% | -0% | |
array/pop/array_pop_uint16_transition.sol |
-0% | ||
array/copying/arrays_from_and_to_storage.sol |
-0% | ||
array/pop/array_pop_uint24_transition.sol |
-0% | ||
functionCall/gas_and_value_brace_syntax.sol |
-0% | ||
functionCall/gas_and_value_basic.sol |
-0% | ||
structs/copy_substructures_to_mapping.sol |
-0% | -0% | |
libraries/internal_types_in_library.sol |
-0% | ||
functionCall/external_call_to_nonexisting_debugstrings.sol |
-1% | +0% | |
constructor/arrays_in_constructors.sol |
-1% | +0% | |
structs/structs.sol |
-0% | ||
functionCall/mapping_array_internal_argument.sol |
-0% | ||
array/pop/byte_array_pop_masking_long.sol |
-0% | ||
array/copying/array_copy_target_leftover.sol |
-0% | -0% | |
structs/conversion/recursive_storage_memory.sol |
-0% | ||
externalContracts/strings.sol |
-0% | +0% | |
libraries/using_library_mappings_return.sol |
-1% | ||
various/address_code.sol |
-0% | -0% | |
array/arrays_complex_from_and_to_storage.sol |
-1% | +0% | |
various/create_calldata.sol |
-1% | ||
functionCall/external_call_to_nonexisting.sol |
-1% | 1% | |
inlineAssembly/transient_storage_selfdestruct.sol |
-1% | ||
array/function_array_cross_calls.sol |
-2% | 1% | |
externalContracts/ramanujan_pi.sol |
-6% |
Time benchmark
|
File | Pipeline | Bytecode size | Time | Exit code |
---|---|---|---|---|
verifier.sol |
legacy | 4899 bytes | 0.10 s | 0 |
verifier.sol |
via-ir | 4324 bytes | 0.37 s | 0 |
OptimizorClub.sol |
legacy | 0 bytes | 0.37 s | 1 |
OptimizorClub.sol |
via-ir | 22170 bytes | 1.89 s | 0 |
chains.sol |
legacy | 5866 bytes | 0.12 s | 0 |
chains.sol |
via-ir | 21398 bytes | 10.71 s | 0 |
default sequence for reference
File | Pipeline | Bytecode size | Time | Exit code |
---|---|---|---|---|
verifier.sol |
legacy | 4874 bytes | 0.14 s | 0 |
verifier.sol |
via-ir | 4351 bytes | 0.70 s | 0 |
OptimizorClub.sol |
legacy | 0 bytes | 0.55 s | 1 |
OptimizorClub.sol |
via-ir | 22193 bytes | 3.94 s | 0 |
chains.sol |
legacy | 5845 bytes | 0.19 s | 0 |
chains.sol |
via-ir | 23043 bytes | 21.33 s | 0 |
External test benchmark diffCI run on
|
project | bytecode_size | deployment_gas | method_gas |
---|---|---|---|
brink | 0% |
||
colony | 0% |
||
elementfi | 0% |
||
ens | 0% |
||
euler | |||
gnosis | |||
gp2 | 0% |
||
perpetual-pools | 0% |
+0% |
-0.01% ✅ |
pool-together | 0% |
||
uniswap | 0% |
||
yield_liquidator | 0% |
-0% |
0% |
zeppelin |
ir-optimize-evm+yul
project | bytecode_size | deployment_gas | method_gas |
---|---|---|---|
brink | +0.55% ❌ |
||
colony | +0.13% ❌ |
||
elementfi | -1.75% ✅ |
||
ens | -0.93% ✅ |
-2.96% ✅ |
-0.01% ✅ |
euler | +1.21% ❌ |
||
gnosis | |||
gp2 | +0.5% ❌ |
||
perpetual-pools | +0.07% ❌ |
-0.23% ✅ |
+0.15% ❌ |
pool-together | -1.2% ✅ |
||
uniswap | +1.65% ❌ |
||
yield_liquidator | +0.93% ❌ |
-0.09% ✅ |
+0.04% ❌ |
zeppelin | -0.4% ✅ |
-1.21% ✅ |
-0.04% ✅ |
ir-optimize-evm-only
project | bytecode_size | deployment_gas | method_gas |
---|---|---|---|
brink | 0% |
||
colony | 0% |
||
elementfi | 0% |
||
ens | 0% |
-0% |
0% |
euler | |||
gnosis | |||
gp2 | 0% |
||
perpetual-pools | 0% |
+0% |
-0% |
pool-together | 0% |
||
uniswap | 0% |
||
yield_liquidator | 0% |
+0% |
0% |
zeppelin | 0% |
legacy-no-optimize
project | bytecode_size | deployment_gas | method_gas |
---|---|---|---|
brink | 0% |
||
colony | 0% |
||
elementfi | 0% |
||
ens | 0% |
||
euler | 0% |
||
gnosis | 0% |
||
gp2 | 0% |
||
perpetual-pools | 0% |
-0% |
-0.01% ✅ |
pool-together | 0% |
||
uniswap | 0% |
||
yield_liquidator | 0% |
-0% |
0% |
zeppelin | 0% |
+0% |
+0% |
legacy-optimize-evm+yul
project | bytecode_size | deployment_gas | method_gas |
---|---|---|---|
brink | -0.03% ✅ |
||
colony | +0.13% ❌ |
||
elementfi | +0.1% ❌ |
||
ens | +0.2% ❌ |
-0.01% ✅ |
-0% |
euler | +0.16% ❌ |
||
gnosis | +0.18% ❌ |
||
gp2 | +0.3% ❌ |
||
perpetual-pools | -0.01% ✅ |
+0.01% ❌ |
+0.02% ❌ |
pool-together | +0.04% ❌ |
||
uniswap | +0.04% ❌ |
||
yield_liquidator | +0.06% ❌ |
+0.02% ❌ |
-0.01% ✅ |
zeppelin | +0.17% ❌ |
+0.18% ❌ |
+0.02% ❌ |
legacy-optimize-evm-only
project | bytecode_size | deployment_gas | method_gas |
---|---|---|---|
brink | 0% |
||
colony | 0% |
||
elementfi | 0% |
||
ens | 0% |
-0% |
0% |
euler | 0% |
||
gnosis | 0% |
||
gp2 | 0% |
||
perpetual-pools | 0% |
+0% |
-0% |
pool-together | 0% |
||
uniswap | 0% |
||
yield_liquidator | 0% |
0% |
0% |
zeppelin | 0% |
-0% |
+0.09% ❌ |
!V
= version mismatch
!B
= no value in the "before" version
!A
= no value in the "after" version
!T
= one or both values were not numeric and could not be compared
-0
= very small negative value rounded to zero
+0
= very small positive value rounded to zero
External test compilation time
Table includes also timing of the default sequence ( |
Related to #14406
Depends on #15023 (only for benchmarking)
Do not merge. This is just a test PR to evaluate how the
the-good-parts-mk2
sequence would affect costs in CI.This PR replaces #14928. The original PR contains a lot of analysis that is still relevant, but I want to gather benchmarks on top of 0.8.25 (which made
cancun
the default EVM version) for a fair comparison withthe-good-parts-mk3
(#15030).