Platform hashgraph hints design proposal

[cody-littley]

This ticket proposes changes to the core platform needed to support hints. It does not get into the details of the cryptographic libraries that provide various primitive operations.

New fields in the state

Add a new field managed by the platform state: the Candidate Roster. This is the roster we'd like to adopt at the next software upgrade boundary. We may or may not have all of the hints computations completed. As we complete these computations, we will update the candidate roster.

Provide API that allows the application to set a new candidate roster. Whenever the application does this, the platform will start trying to compute hints data for that roster. (Easy optimization: carry forward hints data from the previous/current roster in order to speed things up.)

The roster is going to need several new fields. For each node there will need to be a @Nullable hint. For the roster as a whole, there will need to be a @Nullable address book history proof and an address book digest.

New Upgrade Behavior

When the platform starts up during a software upgrade, it should check the candidate address book. If the candidate address book meets the following criteria, it should use the candidate address book as the new current address book.

• address book digest is present
• address book history proof is present
• the collective weight of all nodes with non-null hints exceeds 2/3 total consensus weight

The platform should not swap the candidate address book in the state until the first round after the upgrade (no out of band state updates!).

New Startup/Reconnect Behavior

When a node first starts up or reconnects, it should pass the candidate address book (from the state) into the HintsCalculator. This component will decide if the node needs to compute a new hint or an address book history proof, and asynchronously take action if necessary.

Hints Calculator

Create a new component (sequential task scheduler) called the HintsCalculator. This component is provided with the candidate address book and is responsible for computing a node's hint, the address book digest, and the address book history proof.

Whenever the hints calculator finishes doing its calculation, it submits system transaction(s) containing the results.

If the candidate address book already has this information, then the hints calculator does not take action.

Hints State Manager

Create some new logic that is encapsulated in a class called the HintsStateManager. The hints state manager executes on the handle transaction thread and is capable of updating the state.

• Whenever a transaction that contains a hint reaches consensus, the hints state manager writes it into the current candidate address book.
• Whenever a transaction that contains an address book digest + address book history proof, then
  • if there is already an address book digest + address book history proof in the candidate address book, then take no action
  • if there is not yet this information in the state and the provided information is valid, write it into the candidate address book
• depending on the needs of various parties, this component will be responsible for updating metrics or scheduling callbacks that describe the status of hints generation.

Wiring