Augur errors when jsonschema version 4 is installed

[corneliusroemer]

Description

Augur pins jsonschema to version 3 in setup.py and Bioconda recipe as current code does not work with jsonschema version 4.

Despite those declared version constraints, we've had numerous reports from users ending up with jsonschema version 4 and running into errors. This can happen in a Conda environment if Augur is first installed through Bioconda then other Python packages are installed using pip. If the Python package requested depends on jsonschema somewhere in the dependency tree, pip ignores Biopython version constraints and will resolve to the latest version of jsonschema supported by the dependency tree.

This is not a great experience for users who are managing their own environments and do something like pip install jupyter (not uncommon for our users), then run into the cryptic error below.

Errors/warnings when Augur is used with jsonschema version 4

KeyError: 'tree'

Validating schema of 'auspice/tb.json'...
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/jsonschema/validators.py", line 1153, in resolve_fragment
    document = document[part]
KeyError: 'tree'

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
…
jsonschema.exceptions._RefResolutionError: Unresolvable JSON pointer: '$defs/tree'
An error occurred (see above) that has not been properly handled by Augur.
To report this, please open a new issue including the original command and the error above:
    <https://github.com/nextstrain/augur/issues/new/choose>

Deprecations

+  DEPRECATED: jsonschema.RefResolver is deprecated as of v4.18.0, in favor of the https://github.com/python-jsonschema/referencing library, which provides more compliant referencing behavior as well as more flexible APIs for customization. A future release will remove RefResolver. Please file a feature request (on referencing) if you are missing an API for the kind of customization you need.
+  
+  DEPRECATED: Accessing Draft6Validator.resolver is deprecated as of v4.18.0, in favor of the https://github.com/python-jsonschema/referencing library, which provides more compliant referencing behavior as well as more flexible APIs for customization.

User workarounds

1. Install jsonschema version 3
2. Set up a non-ambient Nextstrain runtime (instructions on Nextstrain install)

Developer solutions

1. (quick fix) Check jsonschema version during runtime (ref)
2. (proper solution) Update Augur to support jsonschema version 4

[tsibley]

Yep. I suggested the newer APIs in the work which added the older API usage.

[victorlin]

It's worth noting that installation of other packages in the same environment can pull in jsonschema>4. Looks like pip doesn't prevent that from happening. Example:

$ micromamba install -c conda-forge python=3.9

$ pip install nextstrain-augur

$ pip list | grep jsonschema
jsonschema          3.2.0

$ pip install "latch[snakemake]"==2.37.1
…
Collecting jsonschema>=4.5.1 (from lytekit==0.15.3->latch==2.37.1->latch[snakemake]==2.37.1)
  Using cached jsonschema-4.21.1-py3-none-any.whl.metadata (7.8 kB)
…
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
nextstrain-augur 24.0.0 requires jsonschema==3.*,>=3.0.0, but you have jsonschema 4.21.1 which is incompatible.
Successfully installed …

I've seen this happen a few times recently, with Latch in the example above and today in an internal Slack thread. Both scenarios were surfaced to devs because they resulted in an error, not just a deprecation warning:

Validating schema of 'auspice/tb.json'...
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/jsonschema/validators.py", line 1153, in resolve_fragment
    document = document[part]
KeyError: 'tree'

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
…
jsonschema.exceptions._RefResolutionError: Unresolvable JSON pointer: '$defs/tree'
An error occurred (see above) that has not been properly handled by Augur.
To report this, please open a new issue including the original command and the error above:
    <https://github.com/nextstrain/augur/issues/new/choose>

[corneliusroemer]

Yep I've also gotten errors in the past but I didn't mention them here because I thought it was "user error" - i.e. if I were to use the official nextstrain environments and not custom install various things I would not have encountered the error.

And technically, there's nothing wrong with the nextstrain-augur Python package saying it has certain version requirement. That pip ignores it is a pip bug more than an augur bug ;) And augur should never be forced to run inside a (constrained) nextstrain environment.

But agreed that it'd be good to move to v4, after all I made this issue :)

[victorlin]

Agreed that this is not an Augur bug! While it would be nice to have everyone use the official Nextstrain environments (managed runtimes), in practice we have many users (both internal and external) that custom install things and will face this error. With the error messages alone, it's easy to think that something is wrong with the file or Augur's code, though in the end it's just due to having a newer version of jsonschema installed, which most likely got installed as a dependency of something completely unrelated.

[victorlin]

On Slack, @jameshadfield and @joverlee521 uncovered that pip list / pip show is not a good indicator of what version of jsonschema is used by Augur due to potential interference by another package manager like Conda. The proper way to inspect the version is to check with the Python interpreter directly:

python -c 'from importlib import metadata; print(metadata.version("jsonschema"))'

[victorlin]

This was brought up again on Slack - seems like pip install juypter also resolves to jsonschema v4 which breaks Augur.

[victorlin]

This was reported again in #1461. I've updated the issue with a more comprehensive description and workarounds.

[genehack]

python -c 'from importlib import metadata; print(metadata.version("jsonschema"))'

I wonder if a quick-and-dirty improvement here (until the work can be done to update to the new API) would be to version-detect jsonschema when augur is starting up, and throw an informative error message when jsonschema v4 is detected?

[victorlin]

I wonder if a quick-and-dirty improvement here (until the work can be done to update to the new API) would be to version-detect jsonschema when augur is starting up, and throw an informative error message when jsonschema v4 is detected?

Added as a solution in the issue description.