discussion: x-client fork prep and test patterns

[meowsbits]

🚧 🐠 I would like to see and help work toward describing plans and accountable standards for

A. Test network testing protocol

As cody speaks to, testnet outcomes are only as meaningful as the rigors they are subjected to. We ought to be clear and collaborative about the design and implementation of these "live" testnet protocols. Who will participate in this? Who will take responsibility, and for which aspects? What are standards that we expect for these protocols? ... reproducability? ... regression-readiness?

• Kensington
• Kotti
• Morden

B. Cross client integration tests

Like establishing a shared and meaningful idea of what "not breaking" means above, there are clear limitations to the implications of even successful testnet behavior (for example eth-classic/go-ethereum#67 would not have been caught).

This points to the value of shared cross-client test suites, which can prove out edge cases and other important simulations that may not be possible or practical on a test network. ethereum/tests is an example of this, albeit in a very one-trick-pony kind of way.

---

Since the teams "owning" client development are either independent and/or volunteer-based, and that participation in the network is entirely in the hands of the client operator in the first place, the standards that might be developed here could not be considered with any more earnest than suggested guidelines for client software and operator review, albeit developed with common aims and shared skin in the game.

[meowsbits]

Rel B. Cross client integration tests

At ETC Labs, we've been developing an "extrapolated" test suite based on ethereum/tests that extends relevance for some of these "sub-suites" thru the anticipated Atlantis configuration. So far -- albeit experimentally -- classic geth and multigeth have shown consistent behavior for custom Difficulty and GeneralStateTests and we're working on adoption for Parity as well.

Work on this at multi-geth can be found here: https://github.com/etclabscore/multi-geth/tree/feat/parity-chainspec-tests-fill-atlantis-2, and an example Atlantis-filled difficulty test can be found here: https://gist.github.com/whilei/cf5700910b41207a37d9f7ca58d0fd3d

[soc1c]

Kotti Hardfork is tonight!

https://stats.kotti.goerli.net/

[soc1c]

are you still working on this @meowsbits?

now we only have multi geth, parity ethereum, and hyperledger besu left.

[bobsummerwill]

@meowsbits Also bear in mind that we now have Whiteblock Genesis at our disposal for exactly this kind of cross-client ETC testing need. It would be great if we could make use of that capability for ETC Aztlan/Phoenix testing:

https://genesis.whiteblock.io/

@YazzyYaz has been giving it a whirl. We are also proposing Yaz as a Vice-Chair for the EEA Testing WG with @atoulme and @zscole at the helm.

ETC Cooperative can also take on running instances of ethereum/tests, HIVE, ChainSafe amenome, whatever. I want us to have the same degree of coverage as there is for ETH. Union of "all the things", including ethstats and forkmon instances.

CC @GregTheGreek, @edwardmack.

[bobsummerwill]

And of course, we welcome OTHERS within the ETC ecosystem to "run all the things" as well.

[atoulme]

We have an example with multigeth here: https://github.com/whiteblock/genesis-examples/tree/master/etc

We also have examples featuring besu and parity. Putting all 3 together is just a matter of creating the right configuration.

What would be useful to test?

[GregTheGreek]

We use anemone + some internal infrastructure. Happy to chat about it further. It allows for us to test against the clients

[soc1c]

we need to test aztlan on mordor with whiteblock genesis or chainsafe anemone

[TheEnthusiasticAs]

Can this issue be closed and new issue opened with the refer to here? Mixing Aztlan and Atlantis topics is not the best way to follow the work progress in my opinion. Thx

[meowsbits]

How about I'll change the name to remove the Atlantis pigeon-hole, broadening to something like "fork prep" ?

[TheEnthusiasticAs]

it is also ok for me ). Also, pls lable this issue. Thx