CKAN Versions support going forward

[amercader]

We are planning on significantly extending and improving the current features and for that we need to focus our efforts. New features are python3 dependent, so users will need to choose the right ckanext-validation version depending on their CKAN version, with the goal that eventually everybody will be using CKAN 2.9+ on Python 3.

• ckanext-validation 0.0.8 supports CKAN <= 2.8
• ckanext-validation 1.0.0 will support CKAN 2.8 and 2.9 (py2 and py3). This is almost done in CKAN 2.9 / Python 3 #55
• ckanext-validation 2.0.0 will support CKAN >= 2.9 with Python 3 support only. Here's where we will do the planned work, including replacing goodtables with frictionless-py, which is Python 3 only

Once #55 is done I'll release the new version and update the README with these guidelines

cc @aivuk @ThrawnCA

edit: added support for CKAN 2.8 in 1.0.0

[ThrawnCA]

Are you sure it's really that difficult to keep supporting multiple versions? We're already doing it in the qld-gov-au fork and it doesn't seem to be all that hard.

[amercader]

ooookay ookay, we'll include support for 2.8 in 1.0.0 :)
Personally I think compatibility hacks like the ones to support calls to Flask or Pylons test apps make the test harder to read (and extend!) but we'll make an effort to keep up with the core support policy

[ThrawnCA]

I like the idea of a version that supports all CKAN versions. It means you can update components one at a time, instead of having to simultaneously update CKAN and the plugin.

The flipside of tests being harder to read is that making use of helper functions can actually make the tests easier to maintain. It encourages good refactoring and DRY code.

[ThrawnCA]

This extension is only now becoming compatible with newer CKAN+Python. CKAN 2.9 has been out for a while, 2.8 hasn't been accepting pull requests except serious bug fixes, but if the plugins aren't compatible, how can end users upgrade? Dropping 2.8 compatibility when people haven't even had the chance to properly try out 2.9 yet is too quick. There needs to be a transition period.

We recently updated to CKAN 2.9 because of the lack of support for 2.8, but we had to update a bunch of extensions ourselves in order to make it work, and we're still dealing with a lot of bugs in the fallout, things that were overlooked in different extensions, etc. It's nice that the upstream for different plugins is now being updated, but it would have been nicer if that happened before people were being pushed to drop 2.8.

Anyway, rant over, I'm happy to see that 2.8 compatibility is being retained for 1.0.0.

[amercader]

This extension is only now becoming compatible with newer CKAN+Python. CKAN 2.9 has been out for a while, 2.8 hasn't been accepting pull requests except serious bug fixes, but if the plugins aren't compatible, how can end users upgrade? Dropping 2.8 compatibility when people haven't even had the chance to properly try out 2.9 yet is too quick. There needs to be a transition period.

We (CKAN team) make an effort before a new CKAN version is released to upgrade the officially supported extensions (this was done for 2.9 and it is being done for 2.10). Of course we can not cover every single extension out there, specially others maintained by other organizations. In this case, we (OKF) just didn't have the time or resources to update this particular extension until now (or previously, when I started #55 but other priorities came in). To the question of how can end users upgrade in this situation I'd say that one way to help is to contribute as many compatibility fixes as possible to the upstream repos to help bring up extensions up to speed.

We recently updated to CKAN 2.9 because of the lack of support for 2.8, but we had to update a bunch of extensions ourselves in order to make it work, and we're still dealing with a lot of bugs in the fallout, things that were overlooked in different extensions, etc.

As I said, support for 2.9 has been in place for a while in most official extensions. If there are bugs or things overlooked in the migration again the best thing to do is to send the patch upstream (which I'm aware you do in many cases). CKAN is mostly run by volunteers and the community.

It's nice that the upstream for different plugins is now being updated, but it would have been nicer if that happened before people were being pushed to drop 2.8.

Again, we'd love to keep all things always up to date but that's just not possible. Users are encouraged to drop 2.8 (released on 2018, 4 years ago) because there are more modern versions that are safer and the CKAN team can actually maintain, that's just how open source software works. It's unfortunate that this particular extension could not keep up with the core release cycle for a while but hopefully it will from now on.

[duttonw]

Hi @amercader,
Per ckan supported versions documentations.
At any one time, the CKAN Tech Team will support the latest patch release of the last released minor version plus the last patch release of the previous minor version.

2.9.x is the latest offical release at this time, as such 2.8.x still needs to be supported. Once 2.10.x drops, then you could possibly drop py2 support at that time also. Though when things like ckan/ckan#7095 happen. It does not seem like 2.10 is yet viable.

[amercader]

@duttonw yes, while 2.10 is not out (because it's in active development) 2.8 is still supported. py2 support will be dropped at the end of the year (which will hopefully coincide with the 2.10 release). Here's all the plan laid out: https://ckan.org/blog/new-supported-versions-policy-going-forward