Merge pull request #3554 from ethereum/dev

release v1.4.0-beta.5

specs/deneb/beacon-chain.md

@@ -11,7 +11,6 @@
 - [Introduction](#introduction)
 - [Custom types](#custom-types)
 - [Constants](#constants)
-  - [Domain types](#domain-types)
   - [Blob](#blob)
 - [Preset](#preset)
   - [Execution](#execution)
@@ -67,12 +66,6 @@ Deneb is a consensus-layer upgrade containing a number of features. Including:
 
 ## Constants
 
-### Domain types
-
-| Name | Value |
-| - | - |
-| `DOMAIN_BLOB_SIDECAR` | `DomainType('0x0B000000')` |
-
 ### Blob
 
 | Name | Value |

specs/deneb/p2p-interface.md

@@ -23,8 +23,9 @@ The specification of these changes continues in the same format as the network s
     - [Topics and messages](#topics-and-messages)
       - [Global topics](#global-topics)
         - [`beacon_block`](#beacon_block)
-        - [`blob_sidecar_{subnet_id}`](#blob_sidecar_subnet_id)
         - [`beacon_aggregate_and_proof`](#beacon_aggregate_and_proof)
+      - [Blob subnets](#blob-subnets)
+        - [`blob_sidecar_{subnet_id}`](#blob_sidecar_subnet_id)
       - [Attestation subnets](#attestation-subnets)
         - [`beacon_attestation_{subnet_id}`](#beacon_attestation_subnet_id)
     - [Transitioning the gossip](#transitioning-the-gossip)
@@ -133,8 +134,6 @@ The new topics along with the type of the `data` field of a gossipsub message ar
 
 ##### Global topics
 
-Deneb introduces new global topics for blob sidecars.
-
 ###### `beacon_block`
 
 The *type* of the payload of this topic changes to the (modified) `SignedBeaconBlock` found in Deneb.
@@ -146,11 +145,30 @@ New validation:
 - _[REJECT]_ The length of KZG commitments is less than or equal to the limitation defined in Consensus Layer --
   i.e. validate that `len(body.signed_beacon_block.message.blob_kzg_commitments) <= MAX_BLOBS_PER_BLOCK`
 
+###### `beacon_aggregate_and_proof`
+
+*[Modified in Deneb:EIP7045]*
+
+The following validation is removed:
+* _[IGNORE]_ `aggregate.data.slot` is within the last `ATTESTATION_PROPAGATION_SLOT_RANGE` slots (with a `MAXIMUM_GOSSIP_CLOCK_DISPARITY` allowance) --
+  i.e. `aggregate.data.slot + ATTESTATION_PROPAGATION_SLOT_RANGE >= current_slot >= aggregate.data.slot`
+  (a client MAY queue future aggregates for processing at the appropriate slot).
+
+The following validations are added in its place:
+* _[IGNORE]_ `aggregate.data.slot` is equal to or earlier than the `current_slot` (with a `MAXIMUM_GOSSIP_CLOCK_DISPARITY` allowance) --
+  i.e. `aggregate.data.slot <= current_slot`
+  (a client MAY queue future aggregates for processing at the appropriate slot).
+* _[IGNORE]_ the epoch of `aggregate.data.slot` is either the current or previous epoch
+  (with a `MAXIMUM_GOSSIP_CLOCK_DISPARITY` allowance) --
+  i.e. `compute_epoch_at_slot(aggregate.data.slot) in (get_previous_epoch(state), get_current_epoch(state))`
+
+##### Blob subnets
+
 ###### `blob_sidecar_{subnet_id}`
 
 *[New in Deneb:EIP4844]*
 
-This topic is used to propagate signed blob sidecars, where each blob index maps to some `subnet_id`.
+This topic is used to propagate blob sidecars, where each blob index maps to some `subnet_id`.
 
 The following validations MUST pass before forwarding the `blob_sidecar` on the network, assuming the alias `block_header = blob_sidecar.signed_block_header.message`:
 
@@ -169,23 +187,6 @@ The following validations MUST pass before forwarding the `blob_sidecar` on the
 - _[REJECT]_ The sidecar is proposed by the expected `proposer_index` for the block's slot in the context of the current shuffling (defined by `block_header.parent_root`/`block_header.slot`).
   If the `proposer_index` cannot immediately be verified against the expected shuffling, the sidecar MAY be queued for later processing while proposers for the block's branch are calculated -- in such a case _do not_ `REJECT`, instead `IGNORE` this message.
 
-###### `beacon_aggregate_and_proof`
-
-*[Modified in Deneb:EIP7045]*
-
-The following validation is removed:
-* _[IGNORE]_ `aggregate.data.slot` is within the last `ATTESTATION_PROPAGATION_SLOT_RANGE` slots (with a `MAXIMUM_GOSSIP_CLOCK_DISPARITY` allowance) --
-  i.e. `aggregate.data.slot + ATTESTATION_PROPAGATION_SLOT_RANGE >= current_slot >= aggregate.data.slot`
-  (a client MAY queue future aggregates for processing at the appropriate slot).
-
-The following validations are added in its place:
-* _[IGNORE]_ `aggregate.data.slot` is equal to or earlier than the `current_slot` (with a `MAXIMUM_GOSSIP_CLOCK_DISPARITY` allowance) --
-  i.e. `aggregate.data.slot <= current_slot`
-  (a client MAY queue future aggregates for processing at the appropriate slot).
-* _[IGNORE]_ the epoch of `aggregate.data.slot` is either the current or previous epoch
-  (with a `MAXIMUM_GOSSIP_CLOCK_DISPARITY` allowance) --
-  i.e. `compute_epoch_at_slot(aggregate.data.slot) in (get_previous_epoch(state), get_current_epoch(state))`
-
 ##### Attestation subnets
 
 ###### `beacon_attestation_{subnet_id}`
@@ -252,6 +253,10 @@ Per `context = compute_fork_digest(fork_version, genesis_validators_root)`:
 
 No more than `MAX_REQUEST_BLOCKS_DENEB` may be requested at a time.
 
+*[Modified in Deneb:EIP4844]*
+Clients SHOULD include a block in the response as soon as it passes the gossip validation rules.
+Clients SHOULD NOT respond with blocks that fail the beacon chain state transition.
+
 ##### BlobSidecarsByRoot v1
 
 **Protocol ID:** `/eth2/beacon_chain/req/blob_sidecars_by_root/1/`
@@ -300,6 +305,10 @@ Clients MUST support requesting sidecars since `minimum_request_epoch`, where `m
 Clients MUST respond with at least one sidecar, if they have it.
 Clients MAY limit the number of blocks and sidecars in the response.
 
+Clients SHOULD include a sidecar in the response as soon as it passes the gossip validation rules.
+Clients SHOULD NOT respond with sidecars related to blocks that fail gossip validation rules.
+Clients SHOULD NOT respond with sidecars related to blocks that fail the beacon chain state transition
+
 ##### BlobSidecarsByRange v1
 
 **Protocol ID:** `/eth2/beacon_chain/req/blob_sidecars_by_range/1/`
@@ -341,7 +350,7 @@ The response MUST consist of zero or more `response_chunk`.
 Each _successful_ `response_chunk` MUST contain a single `BlobSidecar` payload.
 
 Let `blob_serve_range` be `[max(current_epoch - MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS, DENEB_FORK_EPOCH), current_epoch]`.
-Clients MUST keep a record of signed blob sidecars seen on the epoch range `blob_serve_range`
+Clients MUST keep a record of blob sidecars seen on the epoch range `blob_serve_range`
 where `current_epoch` is defined by the current wall-clock time,
 and clients MUST support serving requests of blobs on this range.
 

specs/phase0/p2p-interface.md

@@ -856,6 +856,9 @@ Clients MUST support requesting blocks since the latest finalized epoch.
 Clients MUST respond with at least one block, if they have it.
 Clients MAY limit the number of blocks in the response.
 
+Clients MAY include a block in the response as soon as it passes the gossip validation rules.
+Clients SHOULD NOT respond with blocks that fail the beacon chain state transition.
+
 `/eth2/beacon_chain/req/beacon_blocks_by_root/1/` is deprecated. Clients MAY respond with an empty list during the deprecation transition period.
 
 ##### Ping

tests/core/pyspec/eth2spec/VERSION.txt

@@ -1 +1 @@
-1.4.0-beta.4
+1.4.0-beta.5

tests/core/pyspec/eth2spec/test/bellatrix/fork_choice/test_should_override_forkchoice_update.py

@@ -135,8 +135,6 @@ def test_should_override_forkchoice_update__true(spec, state):
     # Add attestations to the parent block
     temp_state = state.copy()
     next_slot(spec, temp_state)
-    current_time = state.slot * spec.config.SECONDS_PER_SLOT + store.genesis_time + 1
-    on_tick_and_append_step(spec, store, current_time, test_steps)
     attestations = get_valid_attestation_at_slot(
         temp_state,
         spec,

tests/core/pyspec/eth2spec/test/deneb/merkle_proof/test_single_merkle_proof.py

@@ -1,26 +1,39 @@
+import random
+
 from eth2spec.test.context import (
     spec_state_test,
     with_deneb_and_later,
     with_test_suite_name,
 )
 from eth2spec.test.helpers.block import (
     build_empty_block_for_next_slot,
-    sign_block
+    sign_block,
 )
 from eth2spec.test.helpers.execution_payload import (
     compute_el_block_hash,
 )
 from eth2spec.test.helpers.sharding import (
     get_sample_opaque_tx,
 )
+from eth2spec.debug.random_value import (
+    RandomizationMode,
+    get_random_ssz_object,
+)
 
 
-@with_test_suite_name("BeaconBlockBody")
-@with_deneb_and_later
-@spec_state_test
-def test_blob_kzg_commitment_merkle_proof(spec, state):
+def _run_blob_kzg_commitment_merkle_proof_test(spec, state, rng=None):
     opaque_tx, blobs, blob_kzg_commitments, proofs = get_sample_opaque_tx(spec, blob_count=1)
-    block = build_empty_block_for_next_slot(spec, state)
+    if rng is None:
+        block = build_empty_block_for_next_slot(spec, state)
+    else:
+        block = get_random_ssz_object(
+            rng,
+            spec.BeaconBlock,
+            max_bytes_length=2000,
+            max_list_length=2000,
+            mode=RandomizationMode,
+            chaos=True,
+        )
     block.body.blob_kzg_commitments = blob_kzg_commitments
     block.body.execution_payload.transactions = [opaque_tx]
     block.body.execution_payload.block_hash = compute_el_block_hash(spec, block.body.execution_payload)
@@ -44,3 +57,34 @@ def test_blob_kzg_commitment_merkle_proof(spec, state):
         index=spec.get_subtree_index(gindex),
         root=blob_sidecar.signed_block_header.message.body_root,
     )
+
+
+@with_test_suite_name("BeaconBlockBody")
+@with_deneb_and_later
+@spec_state_test
+def test_blob_kzg_commitment_merkle_proof__basic(spec, state):
+    yield from _run_blob_kzg_commitment_merkle_proof_test(spec, state)
+
+
+@with_test_suite_name("BeaconBlockBody")
+@with_deneb_and_later
+@spec_state_test
+def test_blob_kzg_commitment_merkle_proof__random_block_1(spec, state):
+    rng = random.Random(1111)
+    yield from _run_blob_kzg_commitment_merkle_proof_test(spec, state, rng=rng)
+
+
+@with_test_suite_name("BeaconBlockBody")
+@with_deneb_and_later
+@spec_state_test
+def test_blob_kzg_commitment_merkle_proof__random_block_2(spec, state):
+    rng = random.Random(2222)
+    yield from _run_blob_kzg_commitment_merkle_proof_test(spec, state, rng=rng)
+
+
+@with_test_suite_name("BeaconBlockBody")
+@with_deneb_and_later
+@spec_state_test
+def test_blob_kzg_commitment_merkle_proof__random_block_3(spec, state):
+    rng = random.Random(3333)
+    yield from _run_blob_kzg_commitment_merkle_proof_test(spec, state, rng=rng)

tests/core/pyspec/eth2spec/test/helpers/fork_choice.py

@@ -144,6 +144,7 @@ def get_blobs_file_name(blobs=None, blobs_root=None):
 
 
 def on_tick_and_append_step(spec, store, time, test_steps):
+    assert time >= store.time
     spec.on_tick(store, time)
     test_steps.append({'tick': int(time)})
     output_store_checks(spec, store, test_steps)

tests/core/pyspec/eth2spec/test/phase0/fork_choice/test_get_proposer_head.py

@@ -128,8 +128,6 @@ def test_basic_is_parent_root(spec, state):
     slot = state.slot
 
     # Add attestations to the parent block
-    current_time = slot * spec.config.SECONDS_PER_SLOT + store.genesis_time
-    on_tick_and_append_step(spec, store, current_time, test_steps)
     attestations = get_valid_attestation_at_slot(
         state,
         spec,