Sharding introduction R&D compendium

Sharding+Casper/shasper with Ethereum 2.0 is the planned scaling approach for Ethereum; work is in progress on R&D.

Ethereum 1.0 can only process 7-15 transactions per second, the goal of sharding is to partition all network computational resources into shards, so that a node (a single computer as a peer connected to the network) doesn't have to process (download, compute, store, read) every transaction in the history of the blockchain, in order to make a new transaction (write and upload) or otherwise participate in securing and using Ethereum; rather a node can just participate in a single shard, or more if it so chooses. Multiple shards are handled separately by different subsets of securing participants, aka securitors (which include notaries, proposers, miners and validators)[1]. The primary goal is a massive scalability improvement, potentially exponential in phase 6 of the roadmap, although Vitalik questioned whether exponential sharding will even be tenable in an ethresear.ch thread. Quadratic sharding involves having shards at a depth of at most 1 from the main chain, that is, there are no shards within a shard, or a manager shard managing sub-shards; whereas exponential sharding has shards within shards, recursively.

Each one of the shards (currently set to 1024 in the latest spec) will have as high a capacity (and likely more with phase 1) than the current existing Ethereum chain. The latest spec combines sharding with Casper FFG PoS (see e.g. the PoS FAQ and compendium for details) using a RANDAO beacon chain. Previously a contract was planned to exist on the main chain, however that has been scrapped, in order to make sharding implementation easier, as explained e.g. here in a Tweet storm by Vitalik. A smart contract will exist on the main chain, called the sharding manager contract, which manages how data and transactions in shards are accepted as valid by the main chain, via notaries voting on the validity and data availability of collations (collections of data and transactions, analogous to blocks, but where they occur more frequently than blocks) in shards, and proposers proposing blobs (analogous to transactions but without execution in phase 1) which are collected into collations.

Contents

Information
Implementations
Independent research
Grant program
Alternative scaling approaches to sharding
Glossary

Information

For information on sharding, refer to (sorted roughly from the most recent/important information to less recent):

https://github.com/ethereum/eth2.0-specs/blob/master/specs/beacon-chain.md
Sharding FAQs
sharding roadmap
a summary here;
ethresear.ch sharding category (watch new posts);
sharding protocol research tracker
Sharding Fork Choice rule PoC and more info, 2018 May 1. Also see here for an issue tracking protocol specification updates.
Sharding introduction
- networking diagram on slides 82–87
Sharding workshop notes
Ethereum sharding workshop blog post
HackMD note: Ethereum Research Sharding Compendium
Wiki: ethresear.ch Sharding Compendium

Implementations

Implementations under development include:

sharding implementers call
py-evm/Trinity: Gitter, Github: sharding, Github: Trinity. Trinity is the planned/under development EVM that will run a node and network.
Prysm: Ethereum 2.0 client implemented in go. Gitter, website.
Drops of Diamond: using the fast, safe and concurrent Rust programming language, implemented deprecated specifications, it's lead developer @jamesray1 has been working on rust-libp2p, particularly gossipsub, an extensible pubsub messaging protocol.
Lighthouse: implemented the first version of the shasper spec, while work is still underway while waiting for the latest spec to finalize.
Nimbus (doc): an implementation in Nim which compiles to C, and would be good for integrating with Hera, an eWASM implementation in C++. Developed by Status, they are looking to support mobile phones.
PegaSys (ConsenSys): developing a full client that is targeted for enterprise users.
deprecated: sharding utils: has the sharding manager contract and interfaces.