Formalize API handling and semantic version rules

[jonahgraham]

This has been on the todo list for a while and is required for us to graduate the project.

It was most recently discussed in the mailing list but there have been other discussions such as #537.

The consensus is to formalize something along the lines of:

• Protocol classes and interfaces do not follow API central rules and therefore not (semantic versioning](https://wiki.eclipse.org/Version_Numbering), but instead try to match as closely with the definitions from the DAP and LSP spces.
• Supporting classes (e.g. Launcher, adapters) should follow API rules and semantic versioning

This means consumers of LSP4J will probably want a very tight bound on LSP4J version they use all the time. Therefore, one of the other requirements is that LSP4J must not become a singleton to that in an OSGi environment multiple versions can be imported if needed.

One of the biggest consumers of LSP4J in an OSGi environment is LSP4E which is a singleton bundle.

This issue is to get approval from the community and document the above "formally". The earlier mailing list discussion probably counts as approval already, but please comment anyway.

[pisv]

Off the top of my head:

• In an ideal world, a project should strive for a stable API, but breaking API changes seem to be unavoidable in the LSP4J case due to LSP evolution and the differences between TypeScript and Java type systems. As Miro said on the mailing list, "Workarounds for protocol changes could accumulate over time and lead to a weird API". If Eclipse API Tooling cannot be used for LSP4J projects, the problem becomes even more complex.

• Regarding "consumers of LSP4J will probably want a very tight bound on LSP4J version they use all the time." It depends on what policy will be used for managing breaking API changes (semantic versioning or not), but in any case, consumers of LSP4J need to be prepared to be regularly broken by API changes and that's probably the main (and unfortunate) point.

• Given this, the suggested division between "Protocol classes and interfaces" and "Supporting classes", if taken as a formal rule and not a general guidance, does not seem to be very practical, especially when coupled with inability to use API Tooling. (Because, in any case, consumers of LSP4J need to be prepared to be regularly broken by API changes.) The main goal seems to be just to minimize disruptions for consumers, i.e. the work involved in adopting a new LSP4J version.

• So the question is whether to use semantic versioning or not. (As mentioned above, I would suggest to answer this question without introducing division between "protocol" and "supporting" classes.) From the adopter's POV, I see no significant difference. In either case, you would need to change the version range when a new LSP4J version with breaking API changes comes out (i.e. almost always). I would probably say it is better to follow semantic versioning then. It is always important to be honest :-)

This is just to get the ball rolling.

[pisv]

A related question is whether LSP4J will continue to be versioned as a whole (each "part" having the same version).

[pisv]

In an attempt to make my vague comments above a bit more clear, here are more concrete suggestions (post 1.0):

• Treat all APIs equally (API is API)

• Follow semantic versioning, without any exceptions (exactly as described in https://wiki.eclipse.org/Version_Numbering). In case of breaking API changes, increase the major version segment and document the changes in CHANGELOG.

• To minimize disruption for consumers, version each bundle (and probably each exported package) independently, as breaking changes are likely to occur only in few packages.

• To facilitate it, enable API tooling if possible (Consider adding plug-in nature to LSP4J projects #542)

By way of example:

LSP4J 1.0.0 is released. All bundles and exported packages have the same version number, 1.0.0.

An OSGi consumer of LSP4J imports some packages:

 org.eclipse.lsp4j;version="[1.0.0,2.0.0)",
 org.eclipse.lsp4j.jsonrpc;version="[1.0.0,2.0.0)",
 org.eclipse.lsp4j.jsonrpc.json;version="[1.0.0,2.0.0)",
 org.eclipse.lsp4j.jsonrpc.messages;version="[1.0.0,2.0.0)",
 org.eclipse.lsp4j.jsonrpc.services;version="[1.0.0,2.0.0)",
 org.eclipse.lsp4j.launch;version="[1.0.0,2.0.0)",
 org.eclipse.lsp4j.services;version="[1.0.0,2.0.0)",
 ...

Due to LSP evolution, a breaking API change occurs in the org.eclipse.lsp4j package. The version number of that package (and, hence, of its containing bundle) is increased to 2.0.0. There have been no breaking API changes in other packages, so all of the other packages (and all of the other bundles) stay at the 1.x.y version.

LSP4J 2.0.0 is released. The version number clearly indicates that there have been some breaking API changes. When the above mentioned consumer is ready to move to the new LSP4J version, it needs to change the version range only for the org.eclipse.lsp4j package:

 org.eclipse.lsp4j;version="[2.0.0,3.0.0)",
 org.eclipse.lsp4j.jsonrpc;version="[1.0.0,2.0.0)",
 org.eclipse.lsp4j.jsonrpc.json;version="[1.0.0,2.0.0)",
 org.eclipse.lsp4j.jsonrpc.messages;version="[1.0.0,2.0.0)",
 org.eclipse.lsp4j.jsonrpc.services;version="[1.0.0,2.0.0)",
 org.eclipse.lsp4j.launch;version="[1.0.0,2.0.0)",
 org.eclipse.lsp4j.services;version="[1.0.0,2.0.0)",
 ...

(and adapt to the breaking API changes in that package as documented in CHANGELOG).

LSP4J 2.1.0 is released. The version number clearly indicates that there have been no breaking API changes. The consumer can use the new LSP4J version without any adaptation.

[jonahgraham]

I am fine with proper Semantic Versioning - but that probably means releasing a new major version most every time. e.g. #534 required a breaking API change and I would argue that the benefit of adding #534 would have been worth a major version change. Reading through the changelog there aren't many releases that haven't broken API in the past.

IMHO* we should have all packages have the same version in a bundle.

BTW does Eclipse API tooling support version checking on package version exports?

* strong opinion - weakly held

[pisv]

I am fine with proper Semantic Versioning - but that probably means releasing a new major version most every time.

Yes, and IMHO there is nothing wrong with it per se. Breaking changes are breaking changes, and this needs to be clearly communicated. Besides, regular major releases should allow us more flexibility in deprecating and removing the old stuff.

IMHO* we should have all packages have the same version in a bundle.

I see no reason to disagree :-) After all, there are only a few packages in a bundle.

[jonahgraham]

Besides, regular major releases should allow us more flexibility in deprecating and removing the old stuff.

👍

Lets see what others have to say - but I am all for having semantic versioning. It is easier to explain to people!

[pisv]

Yes, and I even think it is sort of requirement for Eclipse projects, or at least a strong recommendation.

[jonahgraham]

Yes, and I even think it is sort of requirement for Eclipse projects, or at least a strong recommendation.

<Somewhat off topic>
Even the Eclipse Platform does not actually follow semantic versioning in removing API https://wiki.eclipse.org/Eclipse/API_Central/Deprecation_Policy
</Somewhat off topic>

[pisv]

Even the Eclipse Platform does not actually follow semantic versioning in removing API

I see. However, this seems to be just a thoroughly arranged exception process to the general rule. And it seems to be not without its own issues.

There is a difference between the Eclipse Platform and LSP4J. Consumers of the Platform have learned to expect API stability and not to be regularly broken by API changes, while unfortunately this cannot be the case for consumers of LSP4J, where breaking changes will probably occur in just about every release (based on the current and past experience).

E.g. currently LXTK uses the version range [0.12, 0.13) for LSP4J, because I know from experience that almost certainly it will not work with the next version of LSP4J. And if proven wrong, I will just adjust the version range to [0.12, 0.14) then. Semantic versioning would just make such 'workarounds' no longer necessary. On the other hand, it is not clear (to me, currently) what benefit can be gained from not using proper semantic versioning for all API in the LSP4J case.

[pisv]

BTW does Eclipse API tooling support version checking on package version exports?

No, unfortunately it does not. Just checked it. Either we will have to maintain package version exports manually or not use them at all.

[jonahgraham]

BTW does Eclipse API tooling support version checking on package version exports?

No, unfortunately it does not. Just checked it. Either we will have to maintain package version exports manually or not use them at all.

In that case I recommend not having them until/unless there is a strong demand for it. We should continue to document breaking API changes in the changelog so that consumers can easily identify what code they need to review/refactor.

[pisv]

BTW does Eclipse API tooling support version checking on package version exports?

No, unfortunately it does not. Just checked it. Either we will have to maintain package version exports manually or not use them at all.

In that case I recommend not having them until/unless there is a strong demand for it. We should continue to document breaking API changes in the changelog so that consumers can easily identify what code they need to review/refactor.

+1 for that.