A path forward with Substrate and Frontier

[sorpaas]

This discussion aims at steer up some interests in the Ellaism community for a possible upgrade path using the Substrate blockchain framework and the Frontier toolkit. If and when the upgrade is finished, we can hopefully deal with many of the pressing issues in the community.

• The upgrade will be a hard fork, and it'll be the final necessary hard fork of Ellaism. From there on, we can switch to forkless upgrades, which means that we can then implement new EVM improvements and EIPs in matter of weeks instead of years. With forkless upgrades, there's also much less worry about chain splits.
• The upgrade will make it possible for us to implement decent on-chain voting.

What will not change

If this hard fork is conducted, we aim at preserving all material aspects of Ellaism. This includes:

• All on-chain states.
• EVM compatibility rules.
• Emission curve.
• PoW algorithm.
• Ethereum RPC interface (through Frontier compatibility layer).

What will be changed

As this hard fork will drastically change the existing codebase, the following things will have to be modified:

• Block, transaction, and receipt structures.
• Block merkle tree.

Technical details

The recommended way from me to conduct this upgrade is through what I call wrapper block with relaxed consensus rules. I'll also list two alternative upgrade paths we can take below, for references.

Wrapper block is a strategy that can be used in Substrate to incorporate other blockchains that has not designed with Substrate in mind. In short, we take the existing Ethereum block structure, and wrap it with a Substrate block structure. We make sure that there is a one-to-one correspondence of the Ethereum/Ellaism and the Substrate block.

Relaxed consensus rules mean that we skip the validation of state root, but instead generate our own. This is safe because the security of a PoW blockchain is on the PoW seal. We think this is a good trade-off compared with strict consensus rules, as the later would require us to implement a feature not yet available in Substrate, and thus massively increase the development time.

Pre-upgrade

For pre-upgrade, we build a Substrate node that is capable of importing Ellaism blocks and generating its own states.

Technically, this means that we create a consensus engine in Substrate that takes a complete Ellaism block (with the header, transactions and receipts) as a pre-runtime digest*. The consensus engine is then responsible for checking the PoW seal. Then, the work is passed to runtime. We check that the extrinsic list is empty (because we do not want to take any other external data other than the Ellaism block), and use Frontier toolkit to execute the Ellaism block. With this, we can import all existing Ellaism blocks.

Hard fork

On the hard fork block, the consensus engine is switched from the pre-upgrade importing engine into a real PoW consensus engine. Then, the pre-runtime digest is removed. We then move new extrinsic submissions into the Substrate block structure and execute new blocks normally.

Nodes and user experiences

During the pre-upgrade time, the existing Geth/OE nodes and the new Substrate node must co-exist. This is because at this stage, only Geth/OE can construct new blocks, while the new Substrate node can only import new blocks. After the upgrade, Geth/OE will split and from there on, Substrate nodes will be able to construct new blocks by themselves.

This is certainly slightly more complicated than a normal hard fork, as in that case we always only deal with one executable. We can simplify the user experience by creating a third executable that will run both Geth/OE and the new Substrate nodes, and dispatch RPC requests accordingly.

Alternatives

The above described wrapper block with relaxed consensus rules is one of many possible paths we can take. Alternatives are:

• ReGenesis: We simply use the ReGenesis process as described in Substrate to conduct this hard fork. The advantage of this is that it can be implemented rather quickly -- in a matter of weeks. The disadvantage is that we cannot preserve the current Ellaism block numbers (they'll be reset to 0 as in a new era).
• Wrapper block with strict consensus rules: This is similar to the recommended method as described above, but with state root validation as well. This is more sophisticated. However, the disadvantage is that it may take a long time to be implemented, as it relies on a feature not yet ready in Substrate.

[sorpaas]

I figure out that it's actually possible to validate the state root as well even using the strategy wrapper block with relaxed consensus rules. We do not even need child storage -- just create a separate merkle db alongside the Substrate one, and validate the state root via the consensus layer.

[majordutch]

To clarify, does this mean "wrapper block with relaxed consensus rules" and "wrapper block with strict consensus rules", as described in the original post, are essentially one in the same with this finding?