ChainEndpoint has no way to signal readiness

[hdevalence]

Summary

The ChainEndpoint trait defining a backend has no way to signal readiness to the rest of the application, causing problems for stateful backends.

Problem Definition

The ChainEndpoint trait defines the interface between Hermes' execution core and a pluggable, chain-specific backend.

The existing backend, for CosmosSDK chains, is stateless: all state required for the relayer is fetched on demand via RPC. This works because the state is maintained on a fullnode, which has visibility into all state.

For Penumbra, the context is different. Penumbra only has shielded transactions, and user data is not visible to full nodes. Instead, Penumbra clients must scan and synchronize the subset of chain data visible to their user. This functionality is provided by a "view server", akin to a micro-node responsible for syncing and indexing just a single user's data. The Rust implementation of the view server is intended to be usable as a library that can be pulled into other programs as an off-the-shelf component.

To pay fees, a Penumbra relayer must perform shielded spends. To perform shielded spends, the relayer must know what funds it has available, and to learn that, it must sync with the chain. While we've put considerable effort into making this fast, it's not instant.

Thus, we'd ideally like to have a way for the ChainEndpoint to signal readiness to the rest of the application. The CosmosSDK endpoint could return readiness immediately, while the Penumbra endpoint would wait to scan to the current chain tip.

However, it's not clear how best to augment the ChainEndpoint trait to provide this. The code structure is a little confusing, since the ChainEndpoint trait presents a synchronous API, which may internally do async operations, and is invoked through a bespoke message passing system. (While changing this design is out of scope for this issue, it might be worth considering a pure-async design, or perhaps this is already considered, I don't know)

Proposal

???

Acceptance Criteria

---

For Admin Use

• Not duplicate issue
• Appropriate labels applied
• Appropriate milestone (priority) applied
• Appropriate contributors tagged
• Contributor assigned/self-assigned

[romac]

While changing this design is out of scope for this issue, it might be worth considering a pure-async design, or perhaps this is already considered, I don't know

We are indeed working on an async-first (or perhaps even async-generic, not clear yet) re-implementation of Hermes which we call Hermes SDK, which will become the foundation for Hermes v2.

It is still very early and very much in flux at the moment, but we expect that this new architecture will make it much easier to add support for non-SDK chains in the future.