HEP 2: Release and deprecation policies

[maximlt]

Warning: builds on #386

---

This PR adds HEP 2: Release and deprecation policies. It is motivated by a discussion we had in holoviz/panel#5221, and by my interest in ensuring that the HoloViz projects apply consistent policies, both for its users and maintainers. The HEP formalizes practices that have been in place for a long time and adds a few more rules on top of those. For instance, I've tried to come up with a clearer process for maintainers to adopt when they deprecate a feature. I've also suggested a minimum deprecation period of 18 months before removal, I'm sure there will be opinions about this period, and I'm also quite convinced we need to define such a period.

I drew some inspiration from:

• https://docs.djangoproject.com/en/dev/internals/release-process/#release-cadence
• https://numpy.org/neps/nep-0023-backwards-compatibility.html

I haven't touched all things on this topic, either because I didn't want to or didn't know what to suggest, I'd say these can be out of scope for now:

• I've addressed deprecation + removal, the HEP however says nothing about what should be done when changing the behavior of some API without removal (e.g. announcing the change of a default parameter value)
• Can a single maintainer decide to move/remove some API without getting anyone else involved? What if maintainers/contributors/users disagree?

[hoxbro]

Overall, this HEP is great.

However, I'm very much against being unable to put a version of when a deprecation is removed.

Some technical details have also made it into this HEP. An example of this is the release project section. It is not that it is wrong, but if we, at some point, want to change it to no longer be triggered by pushing a tag, we need to update this HEP and get the approval.

[hoxbro]

I don't buy this. Adding a version when the removal is being made is fine for both the users and us as developers.

I also think defining the deprecation period of 18 months is too long for some packages (Panel) because a lot of releases can have happened in that time period, whereas for other packages, it may not be enough (colorcet). In a worst-case scenario, a user could download a package on the last day of the 17th month and start using it, only to see it removed the next day because of a new release.

By keeping a table, updating it, and checking it, we also add extra burden on us as maintainers to keep it up to date. Having the deprecations as close to the code as possible makes it easier to avoid it from getting out of sync.

Again, I'm very much on having at least the option to put the version where the removal is gonna happen in the warning message, like what Matplotlib and xarray does. To be clear, this is not what all packages do. An example of this is pandas, which postpone it to the future, but they seem to be moving towards more major releases with breaking changes.

[maximlt]

Adding a version when the removal is being made is fine for both the users and us as developers.

My main goal here is to define a minimum deprecation period that all HoloViz projects can adhere to. I think it'd have benefits for its users and maintainers. I could see allowing projects to specify the removal version as long as they ensure that it meets the minimum deprecation period. For example, if a project releases 1.0.0 and announces a feature will be removed in 1.3.0, the feature can only be removed X months after 1.0.0, even if 1.3.0 happens before.

I also think defining the deprecation period of 18 months is too long for some packages (Panel) because a lot of releases can have happened in that period, whereas for other packages, it may not be enough (colorcet).

I don't think the number or frequency of releases matters so much for the deprecation period. The deprecation period is mostly a tradeoff between letting users have enough time to adapt (apps and libraries) and reducing the cost of maintaining a project. I want HoloViz as a whole project to agree on a minimum deprecation period. Individual projects are free to adopt longer deprecation periods.

In a worst-case scenario, a user could download a package on the last day of the 17th month and start using it, only to see it removed the next day because of a new release.

This isn't a very good counter-example, as this is all about probability, and in that case well, bad luck! The same could happen with a shorter or longer deprecation period. Just to be clear, I'm not saying you should make a new release right after the deprecation period is over.

By keeping a table, updating it, and checking it, we also add extra burden on us as maintainers to keep it up to date. Having the deprecations as close to the code as possible makes it easier to avoid it from getting out of sync.

I don't think HoloViz projects are used to deprecating and removing features, while I'd argue they should do it more to clean up their ever-growing API that sometimes confuses their users. For instance, I personally rarely check whether some code could be removed when I do a new minor release. I haven't built that muscle working on HoloViz projects. How about other maintainers? So I'm suggesting here a systematic approach to dealing with deprecated features, I don't think it's so much extra burden, and I anyway think it's part of the maintainer's job and more precisely of the release manager.

Again, I'm very much on having at least the option to put the version where the removal is gonna happen in the warning message, like what Matplotlib and xarray does. To be clear, this is not what all packages do. An example of this is pandas, which postpone it to the future, but they seem to be moving towards more major releases with breaking changes.

• Matplotlib has great docs on how to deprecate a feature (https://matplotlib.org/devdocs/devel/api_changes.html#rules), it's a minimum of 2 minor releases later and possibly more. Its 3.x releases were: 3.8 (09/2023), 3.7 (02/2023), 3.6 (06/2022), 3.5 (11/2021), 3.4 (03/2021), 3.3 (07/2020), 3.2 (03/2020), 3.1 (05/2019), 3.0 (09/2018). The time between two minor releases is in months: 18, 14, 12, 16, 15, 15 and 15. It's a pretty stable release cadence so I think it's fair for them to announce they will remove a feature in 2 releases. HoloViz projects could also decide to adopt a regular-ish release cadence, though it might be more extra burden than what I'm suggesting.
• Xarray's contributor docs doesn't say you should say when a feature is removed, maybe their docs and code isn't in sync https://docs.xarray.dev/en/stable/contributing.html#backwards-compatibility.

[hoxbro]

I think the tag message should be removed or at least loosened, as it is, in my opinion, not important.

[maximlt]

I suggested that because I observed that the message was pretty inconsistent across releases in a project, and across projects. I like consistency. But you're also right that it is not the most important part of the HEP.

[hoxbro]

This could also explicitly say that no new features can be added after the first RC, and only regressions and bugs should be handled.

Must is also a strong word. I would say another of the dev releases is also good enough for a patch release.

An expectation of a release candidate for some is that it has a time period where people have a chance to test it. What I see in our cases is that we make a release candidate to test if our CI runs correctly (with downstream test), and when it is finished, we make the official release.

[maximlt]

This could also explicitly say that no new features can be added after the first RC, and only regressions and bugs should be handled.

Yes, I didn't think about that this is interesting.

Must is also a strong word. I would say another of the dev releases is also good enough for a patch release.

I'm suggesting replacing the alpha releases (often just one) made before a patch release with release candidates.

An expectation of a release candidate for some is that it has a time period where people have a chance to test it.

Yes, that's true. The HEP could explain that release candidates are always announced sometime before the final release.

What I see in our cases is that we make a release candidate to test if our CI runs correctly (with downstream test), and when it is finished, we make the official release.

Not always, we have applications that test release candidates before the final release. It's not the most common case though, even if it's difficult to say whether other users rely on dev releases in their internal code bases.

[maximlt]

Thanks for your review @hoxbro !

However, I'm very much against being unable to put a version of when a deprecation is removed.

I've commented on that in #388 (comment).

Some technical details have also made it into this HEP. An example of this is the release project section. It is not that it is wrong, but if we, at some point, want to change it to no longer be triggered by pushing a tag, we need to update this HEP and get the approval.

Yep true that's a bit too technical and I can remove it, it's already documented in the HoloViz Contributing Guide.

But I have to say I'm not sure yet what exactly could and should get into a HEP. For instance, Matplotlib developer docs (https://matplotlib.org/devdocs/devel/) include a section with MEPs and many other sections on e.g. Dependency version policy (the equivalent of our HEP 1) or API Changes (the equivalent of this HEP).

So in which case should we need a new HEP (if we need HEPs at all?)? @droumis pinging you since you told me you were thinking about writing HEP0.

[maximlt]

I just talked with Simon and told him I was open to adopting another way to define the deprecation period, as long as there's some sort of consistency across the projects. I suggested a minimum deprecation period of 18 months, he prefers a minimum number of minor versions (correct me if I'm wrong). I still prefer a minimum deprecation period, but I think that in the end whatever we adopt will be better than the status quo!

[maximlt]

Simon opened holoviz/geoviews#701 to remove some features of GeoViews. I started to have a look at it and felt it was a good use case, and well, it was just in front of my eyes at the right time! This helps me understand the whole deprecation process and will hopefully help improve the HEP. By the way, this is all very factual, I don't intend to blame anyone :)

Shared

• The deprecations were released in GeoViews 1.10.0 on the 25th of May 2023. They were planned to be removed one minor release later in GeoViews 1.11. Version 1.11.0 was released on the 1st of November 2023, 5 months later. Removing the deprecations was not done in this first patch release, it seems this is going to happen in version 1.11.1.
• It looks like GeoViews has some code to error when the removal version matches the current working version (in https://github.com/holoviz/geoviews/blame/main/geoviews/_warnings.py#L63). I guess it didn't catch this because the deprecated features were not exerted in the test suite.
• The warning emitted is a GeoviewsDeprecationWarning, a subclass of DeprecationWarning. Given how users run their code, this type of warning is not always emitted (see the section at the end)

load_tiff

• It's not clear to me why gv.util.load_tiff was deprecated in the first place, one user reported in deprecated synthaxe to open tiff using xarray geoviews#639 that this function was using a deprecated API. Maybe load_tiff could have been adapted instead.
• Reading the PR that deprecated gv.util.load_tiff (Deprecate load_tiff geoviews#640), it isn't clear to me that the class methods Image.load_tiff and RGB.load_tiff were also deprecated.
• The deprecation was included in the release notes (Github and website) of version 1.10.0. The version targeted for removal was not mentioned there (thought it could have been inferred from the indication that the Wikipedia tile source was to be removed in 1.11.0)
• The deprecation was not mentioned in the docstrings. GeoViews has no API docs.
• gv.util.load_tiff isn't used in the documentation, so it didn't have to be updated :)
• The programmatic warning included an alternative with geoviews.util.from_xarray(rioxarray.open_rasterio(filename). The release notes/documentation included an alternative too but not as clear as the programmatic one, with rioxarray.open_rasterio to load GeoTIFFs into a xarray.DataArray.
• load_tiff isn't used much out there (I can't tell if it was more used before 1.10 was released). The first two code bases have no test suite.
  • https://github.com/venuswku/data-visualizer/blob/eebc31e71f499b4f37ba48861c8abe9c62d2bd5f/data_visualizer/components/DataMap.py#L403
  • https://github.com/GlacioHack/geoprofile/blob/0a48380f05bf2f8824d98ece14237f537e345f5c/geoprofile/core/core.py#L357
  • EarthSim:
    • https://github.com/holoviz-topics/EarthSim/blob/5d6bde55feaa6e95e292a90575dfc5179f5b3beb/earthsim/grabcut.py#L384
    • (gv.RGB.load_tiff) https://github.com/holoviz-topics/EarthSim/blob/5d6bde55feaa6e95e292a90575dfc5179f5b3beb/examples/user_guide/WMTS_Example.ipynb#L102

Wikipedia WMTS tile

• The issue (Deprecate Wikipedia tile geoviews#623) and the PR (Deprecate Wikipedia tile and CI maintenance geoviews#630) don't mention why this is deprecated.
• Tile source replacement warnings holoviews#4941 (from May 2021) has more context on this deprecation. I have no time to dig more into it, there's a lot of discussion about how things should be deprecated, with a Parameter controlling some of the warning/error behavior. It seems the way this was deprecated in HoloViews vs GeoViews was quite different. The deprecated API was removed in HoloViews in Remove deprecations holoviews#5654 (March 2023). The default tile was changed from Wikipedia to OSM in hvPlot in Switch default tile source to OSM hvplot#610 (May 2021), the documentation wasn't updated accordingly (I'll open a PR for that).
• The deprecation was included in the release notes (Github and website) of version 1.10.0, stating the version targeted for removal was 1.11.0.
• Wikipedia is an instance of WMTS, there wasn't any docstring to update :)
• The documentation was updated in the PR, replacing Wikipedia with OSM.
• The programmatic warning included an alternative with OSM. The release notes/documentation suggested users to pick OSM too.
• It seems gv.tile_sources.Wikipedia has some more usage, some examples:
  • https://github.com/umr-lops/xsar/blob/cd3bfa72afaae95aa544f2efeb8f5c7f29a3ad79/docs/examples/xsar_tops_slc.ipynb#L50
  • https://github.com/umr-lops/xsarsea/blob/b12c73140137ba95197f8693c648c0e2d52bea67/docs/examples/streaks.ipynb#L34
  • https://github.com/OpenEnergyPlatform/open-MaStR/blob/bfa686f509a19a447610d3073fc40cfe07737406/postprocessing/helpers.py#L30
  • https://github.com/otro-mas1/ZBS_Madrid/blob/1dcee97e045c576188f7f6b5a1bf80c8a6f1c62f/ZBS_Mapa_Interactivo.ipynb#L79
  • https://github.com/fccoelho/epigrass/blob/5a94cbc236dbfc60a82d71e8af4ab367779bf44c/Epigrass/epipanel.py#L11

---

I hope I don't get this wrong as Python warnings confuse me a little, but running the code below with python foo.py doesn't print the warning. It does print it if I disable the default warning filters with python -W default foo.py, showing:

/home/liquetm/dev/bar.py:5: DeprecationWarning: Bottom is deprecated
return bottom()

# foo.py
from bar import top

top()

# bar.py
import warnings

def top():
    return bottom()

def bottom():
    warnings.warn('Bottom is deprecated', DeprecationWarning, stacklevel=2)

[maximlt]

My take away from this as a maintainer:

• I'd like us to reach a consensus on how to deprecate a feature, the difference between how the Wikipedia tile source was deprecated in HoloViews and GeoViews is quite big, this can only lead to endless discussions
• I'd like much better guidelines, the whole deprecation cycle involves many actions and is quite complex to get right

[maximlt]

Made some changes:

• Removed the too detailed section on the release process as suggested by Simon (989e935)
• Clarified definition/expectation of release candidates (1e5a7ad)
• Improved the deprecation guidelines and added an explanation on Python warnings (44f52dd)

I think more work is needed:

• (Re)-exploring the various ways Python warnings are filtered/displayed depending on their type and the context made me realize the guidelines should be more clear about them.
• I suggested maintainers should update a deprecation table somewhere, I want to see how this might be done with Param. Simon on his side has looked into exploring a code base to find its warnings and parse them (with the AST). It'd be great to find a systematic approach that isn't too over-engineered.

[maximlt]

I finally got around to dedicating some time to the HEP and updated it in 43cb239. I probably need to give it a good review with fresh eyes, but the next step for me will be to introduce it to the team in a HoloViz meeting.

[maximlt]

@philippjfr here are some deprecation periods for other libraries:

• Django:
  • Feature releases are made roughly every 8 months. Deprecations need to wait for a minimum of 2 feature releases (carrying over major versions if needed). For example, Django 4.2 deprecated something, next version is Django 5.0, the feature is removed in Django 5.1.
  • So this is a minimum of 16 months
• Numpy:
  • Removal shall be done after at least 2 releases assuming the current 6-monthly release cycle; if that changes, there shall be at least 1 year between deprecation and removal. Removal can be done in any minor, but not bugfix, release.
• Matplotlib:
  • Removal minimum of 2 minor releases later and possibly more.
  • Matplotlib 3.x releases were: 3.8 (09/2023), 3.7 (02/2023), 3.6 (06/2022), 3.5 (11/2021), 3.4 (03/2021), 3.3 (07/2020), 3.2 (03/2020), 3.1 (05/2019), 3.0 (09/2018). The time between two minor releases is in months: 18, 14, 12, 16, 15, 15 and 15.
• Scipy:
  • Wait till at least 6 months after the release date of the release that introduced the DeprecationWarning before removing the functionality

18 months is for sure quite high. Listening to your suggestions on:

• a shorter grace period
• special-casing deprecations made before a major release, perhaps with an even shorter grace period

[maximlt]

@jlstevens suggested it would be great if there was automation to enforce some of the requirements. I think that:

• Ideally, yes, but building and setting up automation over multiple repos and in the correct way would require some effort
• Proper review of PRs deprecating and removing features should already be sufficient to make sure this is done correctly
• The most important part, which is to decide to deprecate something and to implement it, cannot be automated.

[droumis]

Looks good, added a couple suggestions

[droumis]

Let's add a concise 'Resolution' section after 'Summary' for a quick recap of the technical plan of action. Something like:

## Resolution
- Adopt a tiered warning strategy for deprecations:
  - Initially emit a `DeprecationWarning`, ensuring it's set with the appropriate stacklevel for accurate source identification.
  - After a 12-month period, elevate the warning to a `FutureWarning` to capture the attention of all users, including those who might not be configured to see DeprecationWarnings.
- Allow exceptions in rare cases, such as beginning with a `PendingDeprecationWarning` for potentially reversible or highly disruptive changes, or immediately using a `FutureWarning` for urgent cases.

[maximlt]

I don't really want to summarize the plan of action as I think all the steps of the deprecation cycle are important.

[jlstevens]

If automation is too high a bar, then the next best thing is establishing a clear, concrete process that does things correctly. For instance, maybe we should have a standardized approach/template for a 'deprecation PR' that we need to create every release.

That way a maintainer can just look up at a reference deprecation PR (e.g. the previous one), follow the steps and without too much cognitive effort be fairly confident that the recommendations in this HEP have been followed. At a minimum I would want a standardized PR name (e.g. Deprecations for version X.X.X) and the first comment to link to this HEP!

[maximlt]

If automation is too high a bar, then the next best thing is establishing a clear, concrete process that does things correctly. For instance, maybe we should have a standardized approach/template for a 'deprecation PR' that we need to create every release.

That way a maintainer can just look up at a reference deprecation PR (e.g. the previous one), follow the steps and without too much cognitive effort be fairly confident that the recommendations in this HEP have been followed. At a minimum I would want a standardized PR name (e.g. Deprecations for version X.X.X) and the first comment to link to this HEP!

But isn't the clear process the Specification section of the HEP?

[jlstevens]

But isn't the clear process the Specification section of the HEP?

Yes but you also want something concrete as well that integrates well into your release cycle. Checking the Specification section of all HEPs at release for instructions is a rather painful workflow.

I'm trying to suggest an approach to make our lives easier, not harder! :-)

[philippjfr]

I would remove the sentence about "significant new functionality", I think we should be perfectly comfortable making a major release solely to make API changes.

[maximlt]

The sentence says should normally include significant new functionality so it doesn't make it a strong requirement.

By the way, I literally copied these definitions from a comment from @jbednar holoviz/panel#5221 (comment).

[jbednar]

Right after that comment I wrote "I'd be interested in whether other team members think this description is accurate.", but no one engaged with it. Reading it now, I'm happy for @philippjfr to make the call -- would we normally provide some carrot along with the stick, i.e. provide some enticement to make it worth switching? You make more releases than I do, so I leave it to your discretion.

[maximlt]

FWIW I re-used your description as I found it appropriate.

[philippjfr]

Does this belong here? I don't think it's strictly related to the deprecation discussion.

[philippjfr]

Ah, sorry, I realize this HEP is meant to cover both release policies and deprecation. Personally I'd almost say those are two separate proposals.

[maximlt]

Yes. They're together because it's a bit difficult to talk about deprecations without having defined a clear version policy. So I think that if two separate proposals were made Version Policy should come first.

[jbednar]

I think https://jacobtomlinson.dev/effver/ captures more of our process than SemVer does; maybe link to that.

[philippjfr]

I would say new deprecations should never be issued in a patch release.

[hoxbro]

Sometimes, a deprecation can be needed if something changes outside our control.

[maximlt]

I would say new deprecations should never be issued in a patch release.

I don't have strong opinions on this, so I could add that to the HEP.

Sometimes, a deprecation can be needed if something changes outside our control.

True, but we control what type of release we decide to make. And, there will always be exceptions to the rule.

[philippjfr]

Should we not standardize on one mechanism or are we worried about the typing_extensions dependency?

[maximlt]

Given deprecated is a decorator I don't think is can be used in all the places you may want to use warn?

[philippjfr]

I don't think prescribing a fixed time period really works for us. In theory semantic versioning solves this problem by setting clear expectation that a major release can break things at will, even without any announcement whatsoever. If people care about that they can appropriately pin major releases.

My main point here is that we can attempt to provide some predictability around the API with this but in practice there are major changes that can happen with relatively little lead time, e.g. think of the Bokeh layout refactor and its effect on Panel 1.x. Until about a month or two before the release it was still quite unclear what the actual effect of the change would be, so despite the fact that API changes are telegraphed that actually guarantees very little about compatibility between major versions.

[maximlt]

I don't think prescribing a fixed time period really works for us.

The HEP doesn't prescribe a fixed time period, instead, it sets a minimum time period.

In theory semantic versioning solves this problem by setting clear expectation that a major release can break things at will, even without any announcement whatsoever. If people care about that they can appropriately pin major releases.

It doesn't work quite like that in practice right 🙃

My main point here is that we can attempt to provide some predictability around the API with this but in practice there are major changes that can happen with relatively little lead time, e.g. think of the Bokeh layout refactor and its effect on Panel 1.x. Until about a month or two before the release it was still quite unclear what the actual effect of the change would be, so despite the fact that API changes are telegraphed that actually guarantees very little about compatibility between major versions.

Yes, my main point is to add consistency and indeed predictability. I agree that major versions are a special case and I can update the HEP accordingly, but I would still recommend giving people enough time when possible. And yes, there will always be exceptions, but I hope there will be just a few of them. The example you give is typically the sort of thing that I hope would not have happened in such a short time, for example, that didn't leave so much time to plan how to migrate CSS.

[maximlt]

In the last round of updates I have:

• Attempted to make it clear that exceptions to the rules are fine
• Turned the deprecation cycle guidelines into a checklist (cc @jlstevens)
• Changed the "must respect a minimum period of 18 months" to "recommend a minimum period of 6 months", given the low interest I saw in adopting a strict minimum deprecation period, and especially one of 18 months. Yet, I still would like to try to ensure some consistency across the projects.

[maximlt]

Another update to align some recommendations with the previous update:

• Upgrade from DeprecationWarning to FutureWarning after a few months
• When removing the feature, ensure the warning type was visible enough

[maximlt]

@holoviz/holoviz-dev this is ready for another round of review. Let me know if I have not addressed one of your previous comments and if you disagree with how I addressed them.

[maximlt]

Add two small things after discussing with Simon:

• Projects should distribute the release candidates before a major release and all final releases as Github releases
• The deprecation policy applies to what each Project considers to be its user-facing public API

[ahuang11]

I don't see stacklevel being used in @deprecated

[maximlt]

Yep true, although it's part of its signature so there might be some reasons to use it for weird edge cases (can't think of one). I can update the text to reflect that. I also found out the runtime behavior of @deprecated can be disabled by setting category to None which means you can used both @deprecated and warnings.warn if needed. Which I think means that typing_extensions doesn't have to be added as a runtime dependency.

I also saw that warnings.warn gained the new skip_file_prefixes parameter in Python 3.12. If I understand what it does I could mention it in the HEP.

[maximlt]

Pushed some updates in b5def36 and f8e0067 about the above.

[maximlt]

• b5def36: Enhanced the warnings.warn / @deprecated section by putting @deprecated first and documenting that calling @deprecated(msg, category=None) disables the run-time warning.
• f8e0067: Added docs on the skip_file_prefixes parameter of warnings.warn added in Python 3.12 that I think could be used in place of setting stacklevel manually or with find_stack_level.

[maximlt]

HEP2 is ready for review. I think the first reviews and discussions have helped resolve some points of contention, but there may still be some. According to HEP0, we should aim to reach a consensus. To make sure I capture everyone's perspective, please add a comment. It could be as simple as 👍, if there are parts of the HEP you disagree with please raise them so they can be further discussed.

cc @jbednar @philippjfr @jlstevens @MarcSkovMadsen @hoxbro @droumis @ahuang11

[droumis]

minor comment about docs, but either way 👍

[jbednar]

I've given it a detailed review, and made a bunch of tiny wording fixes and clarifications, none of which I think should be controversial. I have a couple of minor questions below, but overall, I'm happy with it and happy for it to be merged. Good job! (And I vote for a 6 month period; longer than that and we will have no memory of what we were doing!)

[maximlt]

@jbednar I didn't include the main channel originally as that feels like this shouldn't be the responsibility of a HoloViz maintainer, but of an Anaconda employee, and it turns out the intersection of both is close to 100% at the moment (it may not always be the case).