Weird balance overrides when forking py-evm

[Pet3ris]

• py-evm Version: 0.3.0a19
• OS: osx
• Python Version (python --version): 3.7
• Environment (output of pip freeze):

Pip freeze output

aniso8601==7.0.0
appdirs==1.4.4
appnope==0.1.0
argon2-cffi==20.1.0
asttokens==2.0.3
async-generator==1.10
attrdict==2.0.1
attrs==20.2.0
Babel==2.8.0
backcall==0.2.0
base58==2.0.1
bitarray==1.2.2
black==20.8b1
blake2b-py==0.1.3
bleach==3.2.1
cached-property==1.5.2
certifi==2020.6.20
cffi==1.14.3
chardet==3.0.4
click==7.1.2
cytoolz==0.11.0
decorator==4.4.2
defusedxml==0.6.0
entrypoints==0.3
eth-abi==2.1.1
eth-account==0.5.4
eth-bloom==1.0.3
eth-hash==0.2.0
eth-keyfile==0.5.1
eth-keys==0.3.3
eth-rlp==0.2.1
eth-tester==0.5.0b2
eth-typing==2.2.2
eth-utils==1.9.5
fastdiff==0.2.0
Flask==1.1.2
Flask-Cors==3.0.9
Flask-GraphQL==2.0.1
future==0.18.2
gevent==20.9.0
graphene==2.1.8
graphql-core==2.3.2
graphql-relay==2.0.1
graphql-server-core==1.2.0
greenlet==0.4.17
gunicorn==20.0.4
hexbytes==0.2.1
idna==2.10
importlib-metadata==2.0.0
iniconfig==1.0.1
ipfshttpclient==0.6.1
ipykernel==5.3.4
ipython==7.18.1
ipython-genutils==0.2.0
isort==5.5.3
itsdangerous==1.1.0
jedi==0.17.2
Jinja2==2.11.2
json5==0.9.5
jsonschema==3.2.0
jupyter-client==6.1.7
jupyter-core==4.6.3
jupyter-server==1.0.0rc16
jupyterlab==2.2.8
jupyterlab-pygments==0.1.1
jupyterlab-server==1.2.0
lru-dict==1.1.6
MarkupSafe==1.1.1
mistune==0.8.4
more-itertools==8.5.0
multiaddr==0.0.9
mypy==0.782
mypy-extensions==0.4.3
nbclassic==0.2.0rc7
nbclient==0.5.0
nbconvert==6.0.5
nbformat==5.0.7
nest-asyncio==1.4.0
netaddr==0.8.0
notebook==6.1.4
packaging==20.4
pandocfilters==1.4.2
parsimonious==0.8.1
parso==0.7.1
pathspec==0.8.0
pexpect==4.8.0
pickleshare==0.7.5
pluggy==0.13.1
prometheus-client==0.8.0
promise==2.3
prompt-toolkit==3.0.7
protobuf==3.13.0
ptyprocess==0.6.0
py==1.9.0
py-ecc==4.1.0
py-evm==0.3.0a19
py-geth==2.4.0
py-solc===3.2.0-fixedstdin
py-solc-x==1.0.0
pycparser==2.20
pycryptodome==3.9.8
pyethash==0.1.27
pyevmasm==0.2.3
Pygments==2.7.1
pyparsing==2.4.7
pyrsistent==0.17.3
pysha3==1.0.2
pytest==6.0.2
pytest-tornasync==0.6.0.post2
python-dateutil==2.8.1
python-dotenv==0.14.0
python-json-logger==2.0.1
pytz==2020.1
pyzmq==19.0.2
regex==2020.7.14
requests==2.24.0
rlp==2.0.0a1
rusty-rlp==0.1.15
Rx==1.6.1
semantic-version==2.8.5
Send2Trash==1.5.0
six==1.15.0
snapshottest==0.5.1
sortedcontainers==2.2.2
termcolor==1.1.0
terminado==0.9.1
testpath==0.4.4
toml==0.10.1
toolz==0.11.1
tornado==6.0.4
traitlets==5.0.4
trie==2.0.0a4
typed-ast==1.4.1
typing-extensions==3.7.4.3
urllib3==1.25.11
varint==1.0.2
vyper==0.2.7
wasmer==0.4.1
wcwidth==0.2.5
web3==5.12.2
webencodings==0.5.1
websockets==8.1
Werkzeug==1.0.1
zipp==3.3.1
zope.event==4.5.0
zope.interface==5.1.2

What is wrong?

I'm running a notebook shown below (exported in md for simplicity).

More about how this works:

• This is a py-evm fork that attempts to override the balance for a single address (the coinbase account)
• Initially the address seems to be overridden, but when we call getBalance on the account, it reverts back to 0, any thoughts on why this may be the case? Note the last get_balance call updated get_balance(0x7E5F4552091A69125d5DfCb7b8C2659029395Bdf) = 0 which comes after setting the balance for that address set_balance(0x7E5F4552091A69125d5DfCb7b8C2659029395Bdf) := 2000000000000000000.

How is the fork itself set up:

• I'm forking the get_balance and set_balance calls on the _account_db
• set balance is very simple, it just marks which addresses have been modified in balances_set, in addition, it ignores the first run (which is the initial balance set for the coinbase account)
• get balance works normally, but if the coinbase address has not been set yet, it returns a preset value (2 * 10^18).

Fork testing notebook

Designed to explore why account balances do not preserve correctly when forking py-evm.

from typing import Any, Iterable, Optional, Type

from eth.abc import BlockAPI, ExecutionContextAPI
from eth.rlp.blocks import BaseBlock
from eth.vm.forks.muir_glacier import MuirGlacierVM
from eth.vm.forks.muir_glacier.blocks import MuirGlacierBlock
from eth.vm.forks.muir_glacier.headers import (
    compute_muir_glacier_difficulty,
    configure_muir_glacier_header,
    create_muir_glacier_header_from_parent,
)
from eth.vm.forks.muir_glacier.state import MuirGlacierState
from eth.vm.state import BaseState
from eth_hash.auto import keccak
from eth_typing import Address, BlockNumber, Hash32
from eth_utils import to_checksum_address

def get_fallback_vm_configuration():
    balances_set = set()
    ran = [0]
    
    class FallbackState(MuirGlacierState):
        def get_balance(self, address: Address) -> int:
            adx = to_checksum_address(address.hex())
            if address in balances_set or adx != "0x7E5F4552091A69125d5DfCb7b8C2659029395Bdf":
                balance = self._account_db.get_balance(address)
                if address in balances_set:
                    print(f"updated get_balance({adx}) = {balance}")
                else:
                    print(f"original get_balance({adx}) = {balance}")
                return balance

            # New idea: commit fallback balance to current chain when retrieving
            if adx != "0x7E5F4552091A69125d5DfCb7b8C2659029395Bdf":
                raise Exception("impossible")
            balance = 2 * 10**18
            self.set_balance(address, balance)
            print(f"override get_balance({adx}) = {balance}")
            return balance

        def set_balance(self, address: Address, balance: int) -> None:
            adx = to_checksum_address(address.hex())
            if ran[0] < 1 and adx == "0x7E5F4552091A69125d5DfCb7b8C2659029395Bdf":
                ran[0] += 1
                print(f"ignored set_balance({adx}) := {balance}")
                return # ignore first run

            print(f"set_balance({adx}) := {balance}")
            balances_set.add(address)
            
            self._account_db.set_balance(address, balance)

    class FallbackVM(MuirGlacierVM):
        """Fallback virtual machine that forks from a given web3 network."""

        # fork name
        fork = "muir-glacier-mainnet-fallback"

        # classes
        block_class: Type[BaseBlock] = MuirGlacierBlock
        _state_class: Type[BaseState] = FallbackState

        # Methods
        create_header_from_parent = staticmethod(create_muir_glacier_header_from_parent)  # type: ignore
        compute_difficulty = staticmethod(compute_muir_glacier_difficulty)  # type: ignore
        configure_header = configure_muir_glacier_header

    no_proof_vms = ((0, FallbackVM),)
    return no_proof_vms

import web3
import eth_tester

vm_configuration = get_fallback_vm_configuration()
backend = eth_tester.PyEVMBackend(vm_configuration=vm_configuration)
w3 = web3.Web3(web3.Web3.EthereumTesterProvider(backend))

ignored set_balance(0x7E5F4552091A69125d5DfCb7b8C2659029395Bdf) := 1000000000000000000000000
set_balance(0x2B5AD5c4795c026514f8317c7a215E218DcCD6cF) := 1000000000000000000000000
set_balance(0x6813Eb9362372EEF6200f3b1dbC3f819671cBA69) := 1000000000000000000000000
set_balance(0x1efF47bc3a10a45D4B230B5d10E37751FE6AA718) := 1000000000000000000000000
set_balance(0xe1AB8145F7E55DC933d51a18c793F901A3A0b276) := 1000000000000000000000000
set_balance(0xE57bFE9F44b819898F47BF37E5AF72a0783e1141) := 1000000000000000000000000
set_balance(0xd41c057fd1c78805AAC12B0A94a405c0461A6FBb) := 1000000000000000000000000
set_balance(0xF1F6619B38A98d6De0800F1DefC0a6399eB6d30C) := 1000000000000000000000000
set_balance(0xF7Edc8FA1eCc32967F827C9043FcAe6ba73afA5c) := 1000000000000000000000000
set_balance(0x4CCeBa2d7D2B4fdcE4304d3e09a1fea9fbEb1528) := 1000000000000000000000000

w3.eth.coinbase

'0x7E5F4552091A69125d5DfCb7b8C2659029395Bdf'

w3.eth.getBalance(w3.eth.coinbase)

set_balance(0x7E5F4552091A69125d5DfCb7b8C2659029395Bdf) := 2000000000000000000
override get_balance(0x7E5F4552091A69125d5DfCb7b8C2659029395Bdf) = 2000000000000000000





2000000000000000000

w3.eth.accounts

['0x7E5F4552091A69125d5DfCb7b8C2659029395Bdf',
 '0x2B5AD5c4795c026514f8317c7a215E218DcCD6cF',
 '0x6813Eb9362372EEF6200f3b1dbC3f819671cBA69',
 '0x1efF47bc3a10a45D4B230B5d10E37751FE6AA718',
 '0xe1AB8145F7E55DC933d51a18c793F901A3A0b276',
 '0xE57bFE9F44b819898F47BF37E5AF72a0783e1141',
 '0xd41c057fd1c78805AAC12B0A94a405c0461A6FBb',
 '0xF1F6619B38A98d6De0800F1DefC0a6399eB6d30C',
 '0xF7Edc8FA1eCc32967F827C9043FcAe6ba73afA5c',
 '0x4CCeBa2d7D2B4fdcE4304d3e09a1fea9fbEb1528']

starting_bal = w3.eth.getBalance(w3.eth.accounts[0])

updated get_balance(0x7E5F4552091A69125d5DfCb7b8C2659029395Bdf) = 0

How can it be fixed

This may not be a bug, but I'd love to hear how to ensure that the balances stay consistent between transactions if I do override them.

[carver]

I haven't dived deep into what's going on with this code yet. This isn't really an answer to your question, but an alternative would be to follow the model of TheDAO fork:

py-evm/eth/vm/forks/homestead/headers.py

Lines 85 to 105 in 239c72c

	# In geth the modification of the state in the DAO fork block is performed
	# before any transactions are applied, so doing it here is the closest we
	# get to that. Another alternative would be to do it in Block.mine(), but
	# there we'd need to manually instantiate the State and update
	# header.state_root after we're done.
	if vm.support_dao_fork and changeset.block_number == vm.get_dao_fork_block_number():
	state = vm.state
	for hex_account in dao_drain_list:
	address = Address(decode_hex(hex_account))
	balance = state.get_balance(address)
	state.delta_balance(dao_refund_contract, balance)
	state.set_balance(address, 0)
	# Persist the changes to the database
	state.persist()
	# Update state_root manually
	changeset.state_root = state.state_root
	header = changeset.commit()

[Pet3ris]

@carver this makes sense and thanks so much for highlighting the example, it's quite elegant. Unfortunately, I'm making a dynamic fork to simulate mainnet calls so I'm following an approach similar to ganache which responds and fills in gaps lazily as they are requested by specific transactions rather than pre-loading state at once. The idea is that without running the transactions in the first place, I don't really know what state to set.

[flux627]

Did you ever figure this out?

[Pet3ris]

Unfortunately I didn't, I parked this until I have more observability infrastructure to be able to debug it better but one thing that helped with related issues was being thorough in incorporating all the API functions. For example for nonces, in addition to set_nonce and get_nonce there is increment_nonce and important to update all of them.

[Pet3ris]

@flux627 are you running into a similar issue?

[flux627]

I'm still in a research phase for tooling- looking to see if it was possible to fork mainnet with this like Ganache, in hopes that this implementation is faster. But, it seems that this tooling isn't really meant for this. Also looking at hevm, but I don't know Haskell and it doesn't have any bindings. Any suggestions for performant mainnet forking tests are welcome.

[carver]

Unfortunately, we aren't currently putting any resources toward new features like this in py-evm (though forking mainnet is definitely a cool one that we've talked about, and would like some day).

Though it's fairly straightforward to think of "forking mainnet" (without having the full state) as a kind of variant of Beam Sync. So you can check out how Beam Sync is implemented in trinity, especially the pausing_vm_decorator and how it overwrites VMState. Note that it overrides all the methods (like increment_nonce) in a similar way.

See https://github.com/ethereum/trinity/blob/eaa3b040ffdf0848b8e00a5f329480662ecc7c11/trinity/sync/beam/importer.py#L190