When to move to semantic versioning?

[adizere]

Two questions we're discussing here:

1. What is the impact (problem solved) we seek to have by tagging a specific version line as v1?
2. What is the priority for marking a version line as v1? Put differently, Which set of features in CometBFT should be considered v1?

Ideal outcome of this discussion:

• Agree on the impact of the problem of having v0.- releases.
• Decide which version line (v0.34, 0.37, 0.38, 0.39?) would be first marked as v1.

Feedback we have so far

v0.34 should become v1

From Henry @ Penumbra:

(1) using an 0.x semver version practically limits the information that can be communicated using the version number, in order to convey some vaguely defined sense that software isn’t finished
(2) but as a practical matter, tendermint 0.34 isn’t going away any time soon, it’s deployed on production chains that have a life of their own and will be sticking around for a while
so it seems better to be “honest” about the version number, and call the thing that’s currently deployed “1.0" and a future version that’s not deployed and has breaking changes “2.0”

From a user on Discord, they described the problem for SDK, but said the same is relevant for Comet:

• "are there any plans to switch to proper semver versioning, with bumping a major version (like 0.34.12 -> 1.0.0) in case of breaking changes?"
  Usecase: I had two dependencies requiring cosmos-sdk v0.42.x and v0.44.x, and it wasn't compiling either because v0.42.x didn't have authz types, or because the other dependency was tied to the fact that the address length is constant and was using some constants for that from cosmos-sdk v0.42.x, which were removed in v0.44.x. So eventually I had to fork one of these dependencies to fix it, as golang doesn't provide any means to keep two versions of the same package with the same major version.
  See stackoverflow for context

So the advantages of v0.34 becoming v1 are clear!

Discussion

There's no strong reason why 0.34 couldn't be 1.0. Part of the reasons why this approach has not been implemented yet:

• Facilitate transition from Tendermint Core to CometBFT
  • There are many unknown unknowns (risks we could not foresee nor certainly avoid) with having production networks or projects transition from Tendermint Core to CometBFT.
  • Part of those risks were legal/trademarking in nature (such as using "tendermint" in RPC events or signature types). Others were technical risks: ensure breaks are minimal, both in Go API and state machine changes, also performance degradation or bugs emerging that we would need to be able to patch.
  • We were able to eliminate some of these risks through an approach of let's do the minimal amount of changes that are necessary to call this software CometBFT, while also making the most changes we need to make to reduce legal risk. (This approach was supplemented with legal advising, and multiple weeks of thorough QA.)
  • This was a difficult balance to strike. Marking our first 0.34 release as v1 is not an option because it is not a minimal change in the software (it's a major change) so we postponed it.
  • Specifically on the security/bug front: we need to publish a v0.34 version for the case when projects are already using the v0.34 line and they require security patches. The alternative is to maintain a fork of Tendermint Core and patch that up for many months to come (in parallel to maintaining CometBFT lines).
• Signalling (minor, but relevant concern)
  • It would be explicit and clearer that it is safe to transition from Tendermint Core v0.34 to CometBFT v0.34.
• Priorities
  • There's a strong pull (first priority from consensus council and majority of users) to release ABCI++. Improvements and thorough QA is also a first priority. ABCI++ was planned and is being prepared across v0.37 and v0.38 lines. Having these two items as priorities meant deprioritization of working to understand the impact of adopting semantic versioning (post-v1).
  • Most users would like to have "clear signals of breaking changes" (exactly as Henry pointed out above), which involves post-v1, but there are some subtleties (see the two other feedback points below) so we would need to work on understanding that before we commit to the change. Committing to that is secondary to working on ABCI++ and QA improvements.
• Example: SDK upgrade -- this is speculative, but an interesting point:
  • After the publishing of v0.34.27-alpha.1 it was discovered that SDK 0.46 moving to Comet 0.34 would not be straightforward because of the Go API (package rename) changes. These changes would need to propagate into dependencies (IBC-go & CosmWam) which slows down the process significantly, or is blocked in some cases. The SDK team proposed an alternative (see Rename package back to tendermint/tendermint and use replace to point… #351) which seems to work better. This was discovered very late, and was a surprise.
  • Again, this is speculation on my part: having v1 instead of v0.34 could lead to similar surprises (I would be glad to be wrong).

I will let @thanethomson and @sergio-mena complement the above, as I might miss some context.

Release v1 once ABCI++ included

• We got this feedback from one major user who is maintaining their own fork (they need ABCI++ features) and would like to stop maintaining their fork.

Would be nice to have v1. Should be v1 once you fork and ABCI++ included. Then stick to semantic versioning. And be explicit about breaking changes.

• The reasons for marking v1 only after ABCI++ are:
  • this is the feature which will determine this user to start relying on CometBFT (and renounce their fork). Before ABCI++ is available in CometBFT, they will continue using their fork.
  • signalling: ABCI++ marks a major feature and a level of API maturity.
    • arguably, the same reason applies to marking v0.34 as v1

Important is the expectations from users (not necessarily v1)

• This feedback is from SDK and one other user:

Would be good to have clarification on how upgrades will look like and backwards compatibility. In particular see: [Soft upgrades and version numbers] #122. 0-ver or semantic is beside the point: important is the expectations from users. Versioning proto files would be useful.
Similar to SDK approach, idea: To alleviate the need for forks, consider spinning modules out with their own go.mod, so that each module can evolve on its own. For example, store v1, v2, ...

• This feedback aligns with the idea of having v0.34 as v1.
• Also points to the necessity of thinking of other requirements: versioning of proto files, versioning of modules on their own (to reduce the # of forks).

Preliminary conclusion

There's many things missing in the discussion so far. We'll ask other users to also weigh-in, but some preliminary thoughts would be:

Agree on the impact of the problem of having v0.- releases.

• The problem is: "no clear signalling of breaking changes" and most of the few users we discussed with agree this should be solved.

Decide which version line (v0.34, 0.37, 0.38) would be first marked as v1.

• Priorities and adoption risks are the major reasons why we didn't (yet) release a v0.34* as v1.
• As a potential recommendation: I think we can prioritize releasing a future v0.34 as v1 if: we can determine this task is well scoped and will not impact the "first" priorities (ABCI++ and QA improvements), and if we can derisk adoption (i.e., SDK can adopt v1 and production networks can transition from v0.34 to v1).

[hdevalence]

I think it makes more sense for ABCI++ to be v2, since it's a big step-change in capability and execution model from the current v1.

[adizere]

This makes sense!

It would mean the current v0.38 line would become v2. Is that what you had in mind?

To recall: in the current plan, the features that comprise ABCI++ are planned for release across 2 separate (minor) versions:

1. v0.37 that includes Prepare/ProcessProposal
   • side note: sometimes called ABCI 1.0
2. v0.38 with VoteExtensions and FinalizeBlock
   • sometimes called ABCI 2.0

[hdevalence]

No, I think the current version of ABCI should be called 1.0, so that large changes to the execution model (e.g., Prepare/ProcessProposal) can be versioned.

[thanethomson]

As mentioned in #357 (reply in thread), we want our v0.34-compatible release to be as absolutely non-breaking as possible, and we can't anticipate the impact of suddenly implementing a breaking ABCI version bump in what's supposed to be a non-breaking release. The risk is too high at present.

[gjermundgaraba]

I guess it is clear to everyone, but wanted to echo it just in case: getting over to proper versioning as soon as it safe would be great. I don't particularly care or think it matters a lot in the long term if that is 0.34 or later. The most important part is that we get there, and not in too long.

Just want to throw this out there, but to get the "best" (or worst) of both worlds, could we potentially tag 0.34 as both v0.34 and v1? It's ugly and potentially a bit confusing, yes, but it does allow you to both have a safe 0.34 numbering and get to semver asap. I have not thought this idea all the way through, but wanted to throw it out there in case someone sees any good in it :D

[adizere]

could we potentially tag 0.34 as both v0.34 and v1?

This is a nice insight. We did consider this and discussed it in an internal call. The main hesitation is that it would put our team in the position of needing to maintain multiple versions of the software. This can fragment the user-base across those versions and also give rise to confusion.

It's also unclear how we would deal with v0.37 (QA done and it is actively being integrated in the SDK stack). To satisfy all needs, this might require maintaining these versions:

• v0.34 and v1 (functionally equivalent, but for different users)
• v0.37 and v2 (ditto)
• v3 (would have been v0.38)

That's 5 version lines. By default, we would need to maintain 3 version lines (.34, .37, and .38) for at least the year to come. This is because we envision stopping the maintenance of .34 in 2024, unlikely we can stop in 2023.

We'll continue thinking about it. For the moment, the lowest-disruption (for SDK/current users in production) and highest-impact (for bleeding edge builders) decision seems to be to get onto v1-line after v0.37, so v0.38 would be called v1.

[freak12techno]

Hi I am that user from Discord quoted in the original post, I'm all in for moving to 1.x asap, here's my points why.

1. Golang relies on semver a lot, so it doesn't allow installing two versions of the same library with the same major version. If there's a breaking change in tendermint that is done without bumping the major version, it may cause inconvenience for people using it as a library. Imagine the following theoretical case: there is two versions of Tendermint (let's say 0.37 and 0.38), having a breaking change between them, and imagine you have an app that has two dependencies, one of them relying on Tendermint v0.37, and other on v0.38. This will create a situation when you can't build your project because one of the dependencies won't want to work with Tendermint version currently installed and the only workaround is to fork and maintain your own solution.
2. If a user would upgrade their Tendermint dependency version where only minor/patch version was bumped, thinking it's compatible with the previous one, they can face issues due to breaking changes when testing (or worse in production/mainnet).
3. Regardless of Golang relying on semver a lot, semver itself is a standard used in a lot of places, so users might find it inconvenient that a minor version bump can cause breaking changes, requiring the user to update the code properly.
4. In semver FAQ, answering the question "when to release 1.x" it answers "when your app is used in production, it should be 1.x at least already". Tendermint (and cosmos-sdk as well) are way past the point when people started to use it in production, so it makes sense to push for 1.x release as it's used in production handling a lot of users' assets for quite a long time already.

So in short my suggestion is:

• proceed with 1.x release as soon as it's possible (probably either by making the next major release 1.0, or by publishing the current minor release as 1.x)
• if there are breaking changes later onwards, bump major version

[thanethomson]

4. In semver FAQ, answering the question "when to release 1.x" it answers "when your app is used in production, it should be 1.x at least already". Tendermint (and cosmos-sdk as well) are way past the point when people started to use it in production, so it makes sense to push for 1.x release as it's used in production handling a lot of users' assets for quite a long time already.

This, from my perspective, is probably the strongest reason to move to 1.0 as soon as pragmatically possible.

[sergio-mena]

I will let @thanethomson and @sergio-mena complement the above, as I might miss some context.

An element worth adding to @adizere's initial write-up is that we have moved the ABCI version to (post-)1.0:

• ABCI version in CometBFT/Tendermint Core v0.34.x is (and will continue to be) 0.17.0.
• ABCI version in CometBFT v0.37.x is 1.0.0 (which adds PrepareProposal and ProcessProposal).
• ABCI version in the release including vote extensions and FinalizeBlock will be 2.0.0.
  • We tend to refer to this CometBFT release internally as v0.38.x, but in my view, it could be a good candidate to be CometBFT v1.0, as all the unknown unknowns @adizere was referring to (which apply to releases that are battle tested or release candidates that have already undergone an extensive QA process) don't apply yet to v0.38.x (which, technically speaking, is still main).

Two remarks on the ABCI version:

• RPC: Even if our RPC does not currently provide the ABCI version, we could extend /status, or even add a new endpoint (either of them can be done in a non-breaking way, I think) to return the ABCI version it if our users deem it necessary.
• ABCI method Info (which is called upon state-sync or recovery) already contains a field with the ABCI version (which will contain 1.0.0 in CometBFT v0.37.x and 2.0.0 in the subsequent release). In my view, that's the version that actually matters to an (out-of-process) application, rather than the CometBFT (general) version.

Finally, bear in mind that:

• ABCI 1.0 and 2.0 are substantial, breaking changes with respect to each other, and to the existing 0.17.0.
• However, we are not planning to introduce breaking changes to our RPC until we have a holistic solution to version it. From what I read in this discussion, we can make that happen only after we have reached CometBFT v1.0, if that is the preferred solution for our users.

[hdevalence]

Since ABCI is already in production, and Prepare/ProcessProposal is a substantial breaking change to it, wouldn't it be better to call the existing ABCI protocol 1.0 and use versions to indicate successive substantial breaking changes? What is the advantage of locking existing users out of the versioning?

[sergio-mena]

In theory, that could indeed be done (i.e., changing ABCI 0.17.0 to 1.0.0 on CometBFT v0.34.x). However, our (sometimes painful) experience is that, the least we fiddle with v0.34.x, the easier it will be for us, and for the wide range of existing users of that version. That's why we thought leaving the current ABCI version as 0.17.0 as it is (and has been for a long time) was the best compromise.
Less importantly, although relevant, we adopted this ABCI versioning strategy, suggested by the SDK team, a few months back. So there has been a communication effort going on during this time, where "ABCI 1.0" and "ABCI 2.0" are already commonly used terms, sometimes in substitution of (the more simplistic) ABCI++. So, telling now everyone that ABCI 1.0 is actually ABCI 2.0, and ABCI 2.0 is 3.0, makes me feel the pain directly from the future :-)

[thanethomson]

Since ABCI is already in production, and Prepare/ProcessProposal is a substantial breaking change to it, wouldn't it be better to call the existing ABCI protocol 1.0 and use versions to indicate successive substantial breaking changes? What is the advantage of locking existing users out of the versioning?

While in principle this makes sense, as @sergio-mena says, pragmatically we don't know whose systems we're going to be breaking by suddenly introducing a major version number bump in what is effectively supposed to be a non-breaking release.

Beyond this, we currently don't intend on maintaining a legacy ABCI protocol (i.e. pre-Prepare/ProcessProposal) in future versions of our software.

[hdevalence]

A major version number isn't an indication of a maintenance commitment, though, it's a way to describe versioning compatibility.

And while the Informal/CometBFT team may not intend to support it, that doesn't change whether or not it will continue to exist in the ecosystem. (For instance, we were burned badly enough by the 0.35 experience that we have no concrete plans to migrate to any version of ABCI++ until it's in a proven release.)

[adizere]

Dropping this reference here, which I found recently and is very good: https://github.com/tendermint/tendermint/blob/master/docs/architecture/adr-060-go-api-stability.md.
The previous team already did a lot of due diligence with respect to preparing the public APIs for a v1 release.

[adizere]

We have decided to release v1 tentatively as the next version of CometBFT. Follow-up with plans here: #578. There's still a few unknowns (in particular proto renaming) but unlikely these are blockers on releasing a v1.

[adizere]

Update: We published the semantic versioning scheme for CometBFT releases with v1 release -- https://github.com/cometbft/cometbft/blob/v1.0.0-alpha.1/UPGRADING.md#versioning