order matters

Release

Quick bump in v0.6.x release to fix a bug that emerged when running test vectors in other languages due to undefined ordering of sets. Also added back in a deposit top-up test that got dropped in the v0.6.2 release.

Thank you @michaelsproul for finding, reporting, and fixing both of these :)

Changelog

The PR showing the full changelog of the v06x branch being released into master can be found here: #1137

• enforce sort order for attester slashings (#1126) [bugfix]
• add back in additional deposit top-up test (#1133)

mic check

Release

The long awaited testing enhancements for v0.6.x!

This minor release focuses entirely on testing, and contains no substantive changes related to the spec. @protolambda drove the process to unify the python spec tests and the YAML test generators, and in doing so has increased YAML test coverage by an order of magnitude.

Other than test generator work and documentation, bls signatures were updated to utilize the correct (SHA-256) hash function (#1121).

Changelog

The PR showing the full changelog of the v06x branch being released into master can be found here: #1130

Tests and generators

• Decorator for pytests to enable usage in test generation (#1052)
• Ensure signatures are correct in test generators. default disable in pytests for speed (#1102)
• Add epoch and sanity test generators and a number of test-format docs (#1114)
• Update py-ecc version to use SHA-256 (#1121)
• ssz_static test-case format is changed to facilitate more efficient parsing. The SSZ type is specified in the key now. (Commit 3781614)
• Minor fix to shuffling generator (#1045)

Tidying Up

Release

This minor release fixes a few bugs from v0.6.0 Spring Cleaning, handles a minor refactor, and adds a 32-byte graffiti field to blocks. Thanks for the bug reports! v0.6.1 tests will be released shortly.

Changelog

Two PRs showing partial diffs of v0.6.1 being backported to master can be found here: #1003, #1038

Phase 0 spec

• features
  • add 32-byte graffiti 🎨 (#1014)
• simplifications/refactors
  • refactor attestation committee to be more generic and useful in phase 1 (#1009)
• bugs
  • fix process_registry_updates missing assignment (#1001)
  • fix two effective_balance bugs (#1004)
  • fix some underflow issues (#1017)
  • fix incorrect var in attestation rewards (#1037)
  • minor bugs related to ordering/typos/var-mismatch/etc ()
• misc
  • remove unused helper get_state_root from state transition doc (#1015)
  • minor formatting/typos/etc PRs (#994, #997, #996, #1013 )

Phase 0 validator

• minor formatting/types/etc PRs (#993)

Phase 0 networking

• minor formatting/types/etc PRs (#910, #907)

Deposit contract

• minor formatting/types/etc PRs (#995)

Simple Serialize

• implement SOS serialization in minimal_ssz.py (#1024)

BLS [none]

Phase 1 spec (warning: not stable) [none]

Light client spec (warning: not stable) [none]

Testing/CI

• minor formatting/typos/etc PRs (#1007, #1006)

Spring Cleaning

Release

Since the relatively stable and well tested v0.5 release ~6 weeks ago, much of the spec development has been focused on efforts to clean and simplify so that the spec can be more easily understood, built, and audited in the coming months. To this end, much of the below changelog for phase 0 is centered around straight simplifications as well as feature additions/modifications that are also ultimately simplifying.

In the spirit of spring cleaning, we've also pulled out the deposit contract and fork choice into separate documents. The beacon chain spec now solely addresses the beacon chain state transition function.

Although much work has gone into testing, test generators, and CI, new YAML test vectors are not to be released immediately with v0.6.0, but instead will follow next week. Stay tuned!

Changelog

PR showing full diff can be found here: #987

Phase 0 spec

• features
  • swap get_shuffling for compute_committee (#776)
  • milder ejections (#784)
  • include recently slashed in exit churn queue (#785)
  • do not allow slashed validator to propose (#757)
  • require mandatory deposits in blocks (#758)
  • separate out effective balance from actual balance (#728)
  • create is_slashable_validator for stricter criteria (#806)
  • Move to SHA256 (#779)
  • probability selected as proposer proportional to balance (#772)
  • remove committee exponential backoff (#808)
  • homogenised start shard (#889)
  • enable transfers of balance proportions > MAX_EFFECTIVE_BALANCE (#936)
• simplifications
  • simplify and cleanup process_attestation (#783)
  • simplify exit_validator (#801)
  • simplify justification and finalization accounting logic (#826)
  • merge attestation verification logic with indexed attestations (#831)
  • cosmetic improvement to reward/penalty functions (#827)
  • cleaner dust checking in transfers (#919)
  • simplify Eth1Data voting (#938)
  • decouple justification and finalization processing (#925, #957, #958)
  • remove custody_bitfield from PendingAttestation (#955)
  • genesis slot/epoch back to 0 (#896)
  • cleanup per-epoch processing presentation (#959)
  • add SSZ object default values (#963)
  • change withdrawal queue to exit queue (#850, #977)
  • remove serialization from consensus (#924)
  • revamp and cleanup incentivization (#949)
• bugs
  • fix get_split_offset (#790)
  • ensure validator active for voluntary exit (#807)
  • limit bit-length of justification bitfield to strict 64 (#890)
  • fix transfer invariant by disallowing
  • add previous and current crosslinks to fix a few crosslink bugs (#874)
  • fix off by one error for eth1 data voting (#992)
  • minor bugs related to ordering/typos/var-mismatch/etc (#873, #921, #922, #923, #961)
• misc
  • set GENESIS_FORK_VERSION to a bytes4 constant (#792)
  • genesis_epoch - 1 for shuffling epoch (#786) [outdated]
  • rename "transaction" to "operation" (#863)
  • rename signed_root to signing_root (#885)
  • aesthetic rework to get_domain (#865)
  • disallow transfers in default constants (#965)
  • signal desired issuance goal (#971)
  • minor formatting/typos/etc PRs (#789, #788, #791, #820, #985 #828, #830, #878, #879, #881, #899, #902, #904, #909, #911, #913, #914, #908, #858, #970, #973, #984, #985, #986)

Phase 0 validator

• mandatory deposits (#759)
• committees deterministic for next epoch (#808)
• minor formatting/types/etc PRs (#847, #912)

Phase 0 networking

• mega networking PR (#763)
• minor formatting/types/etc PRs (#910, #907)

Deposit contract

• add deposit count as an accessible and logged variable (ethereum/deposit_contract#22)
• update deposit data as per #780 (ethereum/deposit_contract#25)
• use hash_tree_root for DepositData and make contract deposit() params explicit (ethereum/deposit_contract#31)
• remove MAX_DEPOSIT_AMOUNT (ethereum/deposit_contract#30)

Simple Serialize

• update to SOS style offset layout (#787) [changes not yet reflected in minimal_ssz.py]
• rename "tuple" to "vector" for fixed sized lists (#794, #777)
• a few fixes to minimal_ssz implementation (#960)
• misc (#894, #929, #969)

BLS

• clarify empty sums in BLS spec (#782)
• Warning: Bug exists in BLS hash to G2 in the current spec. The hash and increment approach will likely be deprecated after standardization so the bug is to be untouched for now (#898)

Significant changes still expected as we standardize BLS12-381 across blockchains

Phase 1 spec (warning: not stable)

• replace custody game with JABs (#812)
• persistent committee size per slot reduced to max 128 (#734)
• replace with empty instead of popping finished challenges (#870)
• minor formatting/types/etc PRs (#818, #868, #915)

Light client spec (warning: not stable)

• add initial pass on light client spec (#766)
• minor formatting/typos/etc PRs (#825, #829, #930, #895)

Testing/CI

• integrate CI testing (#800, #809)
• fix vector implementation in minimal_ssz (#810)
• increase test coverage (#814, #989)
• new more complete test format for ETH 2.0 testing (#877)
• combine specs and test-generators (#851)
• SSZ static testing (#962)
• update CI config: caching of repo, venv, split test install/run (#967)
• generators workflow (#968)
• use configuration system in pytest (#979)
• misc (#839, #892, #897, #888, #933, #951, #952, #953, #978, #990)

Circularity

Release

This minor release patches two critical bugs found in v0.5.0, the more significant involving a circle dependency when attempting to embed block roots in state. Thanks @paulhauner from Sigma Prime for finding and reporting them both.

Changelog

Phase 0 spec

• Fix circular dependency when embedding block roots in state by using signed_root(block) (Issue: #797, PR: #816)
• Fix start shard order of operations (Reported in: #805, PR: #821)

`exec beacon_chain`

Release

The phase 0 beacon chain specification is now executable. This was handled in a number of refactoring PRs (#708, #709, #711). The first set of "sanity check" tests have been created against the executable spec and exist as state test vectors in the tests repo. As we increase testing with the executable spec, we expect to continue to find minor bugs. These will be released frequently via minor version updates to the spec.

As usual, this release also includes a number of feature additions/modifications, clarifications, and minor bug fixes. Things of note include: [1] epoch transition at start of epoch (#732), [2] SSZ refactored and simplified (#696), and [3] parent-root reconstruction + body/header segregation (#649).

Changelog

PR showing full diff can be found here: #755

Phase 0 spec

• pythonize refactors
  • convert transaction sections to python code (#708) [contains no substantive changes]
  • unify deposit processing between genesis and transaction (#709) [contains no substantive changes]
  • pythonize epoch transitions (#711)
    • warning: this PR contains both a pythonizing refactor plus some substantive changes and bug fixes
    • see PR for detailed changelog
  • reorder SSZ types topologically (#731) [contains no substantive changes]
• features
  • make get_beacon_proposer_index safe for next epoch (#694)
  • utilize updated version of ssz signed_root (#725)
  • modify BeaconState and Deposit fields to utilize new ssz fixed length tuples (#723)
  • break LMD GHOST ties in favor of higher hash tree roots (#737)
  • epoch transition at start of epoch (#732, #741)
  • remove merkle_root and instead use hash_tree_root everywhere (#739)
  • weaken criterion for attestation inclusion (#752)
  • disallow duplicate voluntary exits (#751)
  • validator cannot exit until 2048 epochs (#746)
  • proposer slashing based upon epoch rather than slot (#762)
  • fair proposer selection by using epoch (#733)
  • improve fork version/signature domain semantics (#738)
• bugs
  • fix previous epoch committee near genesis (#672)
  • fix the "ready to exit" condition in update_validator_registry (#705)
  • fix latest_randao_mixes initialization value (#713)
  • fix committee assignment bugs (#699)
  • handle when no attestations available for crosslinking (#717)
  • handle get_base_reward when no previous balance (#718)
  • only attempt to reward proposer if attestation exists for validator index (#744)
  • do not check withdrawal_credentials for existing validators (#750)
  • cache justified/finalized roots in state for use in state transition (#756)
  • minor bugs related to ordering/typos/var-mismatch/etc ()
• misc
  • clarify "active validator" in reward/penalty calculations (#703)
  • minor refactor to process_slashings to reduce repeated computation (#714)
  • minor formatting/typos/etc PRs (#693, #688, #715, #743, #770)

Phase 0 validator

• a number of updates getting v-guide in sync with v0.5.0 (#769) [see PR for changelog]
• minor bugs related to ordering/typos/var-mismatch/etc (#689)
• minor formatting/typos/etc PRs (#720, #711)

Deposit contract

• Both the Deposit and Eth2Genesis log have received updates to their interface (ethereum/deposit_contract#21, ethereum/deposit_contract#22)

Simple Serialize

• simplify SSZ spec (#696) [see PR for changelog]
• clarify basic/non-basic types for bytes (#724)

BLS

No changes, but we expect some coming as we standardize BLS12-381 across blockchains

Phase 1 spec (warning: not stable)

• alter proof of custody game to ensure bound on state size (#682)
• add developer notice (#684)
• minor formatting/types/etc PRs (#767)

Clean Signing

Release

This is the third of weekly Friday releases in February -- Clean Signing.

This release includes a number of small feature additions/modifications, clarifications, and minor bug fixes. The most significant feature added was a new unified method of hashing ssz objects for signing (#625, #633).

Additional logic for handling proof of custody was added to the phase 1 spec (#587). The interactive custody game is still under review (#645, #682).

Outside of this immediate release, progress continues to be made on the cross-client testing, and much progress has been made around network specifications. The initial version of the wire protocol is under development and is to be released this week.

Changelog

PR showing full diff can be found here: #683

Phase 0 spec

• features
  • refactor signature hashing to use new unified ssz method (#633)
  • max list size in get_permuted_index (#634)
  • reduce genesis slot to 2**32 (#655)
  • simpler is_power_of_two implementation (#674)
  • specify lexicographic ordering of crosslink_data_root (#676)
  • allow transfers from both non-activated and exited validators (#678)
• bugs
  • fix out-of-bounds in get_shuffling (#641)
  • fix slot range for attestation inclusion (#669)
  • fix previous epoch logic (#672)
  • fix handling of bad proof of possession in deposit (#687)
  • minor bugs related to ordering/typos/var-mismatch/etc (#648, #650, #654, #656, #664, #661)
• misc
  • validator slashed and initiated_exited to booleans (#658)
  • rename shard_block_root -> crosslink_data_root (#662)
  • minor formatting/typos/etc PRs (#640, #643, #646, #666)

Phase 0 validator

• fix epoch start lookup (#671)
• make getting assignments more generic (#673)

Simple Serialize

• unified method for signature hashing (#625)
• add c++ implementation to list (#653)

BLS

No changes, but we expect some coming as we standardize BLS12-381 across blockchains

Phase 1 spec (warning: not stable)

• proof of custody branch and early subkey reveal challenges (#587)

Let There Be Liquidity

Release

This is the second of weekly Friday releases in February -- Let There Be Liquidity.

This release includes a number of small feature additions/modifications, clarifications, and minor bug fixes. The most significant features added were (1) enforced ordering of eth1 deposits (to avoid replay attacks) and (2) enabling transfers from "withdrawable" validator accounts (to open up liquidity options for validators before phase 2).

The core of the phase 1 spec is currently being built, but the exact particulars should be considered unstable and not yet targeted for production development.

Outside of this immediate release, progress continues to be made on the cross-client testing, and much progress have been made around network specifications (#593) and discovery protocol.

Changelog

PR showing full diff can be found here: #631

Phase 0 spec

• features
  • tighten up eth1 deposit flow (#594, #589)
  • enable transfers between "withdrawable" accounts (#601)
  • switch bitfields to little-endian (#575)
  • only reward attestations from committees for previous epoch + note on validator set to reward/penalize (#597)
  • add minimum slashing penalty (#624)
  • refactor withdrawable (#615)
• bugs
  • fix off by one error in attestation validity (#627)
  • fix block slot acceptable processing time (#636)
  • minor bugs related to ordering/typos/var-mismatch/etc (#619, #635, #637)
• misc
  • clarifying note on unix time and leap seconds (#600)
  • safety assert on get_permuted_index (#622)
  • variable name changes for clarity (#534)
  • minor formatting/typos/etc PRs (#560, #607, #608, #614)

Phase 0 validator

• variable name changes (#630)
• minor bugfix when getting epoch_boundary_root (#629)

Simple Serialize

• simplify bytes type (#596)
• fix merkleization 0-chunk bug (#602)
• merkleize container elements (#595)

BLS

No changes, but we expect some coming as we standardize BLS12-381 across blockchains

Phase 1 spec (warning: not stable)

• modify shard-chain fork choice to use both crosslink and shard chain attestations (#586)
• update data committed to in a crosslink (#579, #617)
• updates to persistent committees, shard block structure, and shard block validation (#578, #604)
• variable name changes (#630)

Steady Progress

Release

This is the first of weekly Friday releases in February -- Steady Progress!

This release includes a number of small feature additions/modifications, helpers, clarifications, and minor bug fixes. The most significant feature addition is the introduction of the point-wise shuffle -- "Swap-or-Not" (#576). This shuffling algorithm was suggested by Dan Boneh at a recent research event at Stanford and seems to meet all of our shuffling needs!

Outside of this release, steady progress is also being made on the cross-client testing. Test generators have moved out of the test vector repo and into a standalone eth2.0-test-generators repo. When a release is cut in the generators repo, tests are automatically generated and pushed to eth2.0-tests. In an attempt to learn from eth1.0, this in an effort to make testing PRs more sane.

Changelog

Phase 0 spec

• features
  • introduce "swap-or-not" shuffle (#576)
  • fork choice rule now weighted by balance (#569)
  • mix epoch into seed within generate_seed rather than in get_shuffling (#552)
  • AttestationData includes latest_crosslink rather than just latest_crosslink_root (#585)
• helpers
  • add get_previous_epoch helper (#565, #592)
  • add get_total_balance helper (#567, #569)
  • int_to_bytesN uses little-endian (#564)
• bugfixes
  • minor range fix to verify_bitfield (#573)
• misc
  • Use 2**63 for GENESIS_SLOT (#537) [Note: this is contentious due to collision with signed types and likely to change]
  • Remove vyper contract source from spec. replaced by API and link to deposit_contract repo (#562)
  • minor formatting/typos/etc PRs (#547, #551, #558, #561, #574)

Phase 0 validator

• rename get_next_epoch_crosslink_committees -> get_next_epoch_committee_assignment (#549, #592)
• added registry_change arg to get_next_epoch_committee_assignments returns assigned slot and bool for is_proposer (#549, #592)
• get_next_epoch_committee_assignments returns tuple -- (validators, shard, slot, is_proposer) (#549, #592)

Simple Serialize

• List tree-hash padding consistency fix (#538)
• "internal" vs "external" hash roots padding fix (#543)

BLS

• rename message -> message_hash in all bls functions (#580, #583)

The January pre-Release™️

The marks the first release in a series of weekly releases through February 2019. Phase 0 in v0.1 is relatively feature complete and approaching stable. Subsequent changes will occur on dev branch and only merged into master during a release accompanied by a changelog.

Know phase 0 semi-major items to add/change:

• Pointwise shuffling evaluation
• Enforced ordering of eth1 deposits
• Minor adjustments to BLS (e.g. hash functions, generators, serialization) to conform to standards.
• Adjustment of constants pending testnets and community input
• Bugs.