refine throws error "TypeError: '<' not supported between instances of 'NoneType' and 'str'"

[corneliusroemer]

Current Behavior

When I run refine with data that's somehow not according to requirements (will figure out how), refine throws an uncaught error

TypeError: '<' not supported between instances of 'NoneType' and 'str'

Expected behavior

We validate inputs and report a useful error message to the end user.

How to reproduce

augur refine             --tree results/lineage-b.1/tree_fixed.nwk             --alignment results/lineage-b.1/masked.fasta             --metadata results/lineage-b.1/metadata.tsv             --output-tree results/lineage-b.1/tree.nwk             --root ref_in_coord             --divergence-unit mutations             --keep-polytomies             --use-fft             --output-node-data results/lineage-b.1/branch_lengths.json

Files: refine-type-error.tzst.txt

Augur 23.1.1 with treetime 0.11.1

Full error:

augur refine             --tree results/lineage-b.1/tree_fixed.nwk             --alignment results/lineage-b.1/masked.fasta             --metadata results/lineage-b.1/metadata.tsv             --output-tree results/lineage-b.1/tree.nwk             --root ref_in_coord             --divergence-unit mutations             --keep-polytomies             --use-fft             --output-node-data results/lineage-b.1/branch_lengths.json
augur refine is using TreeTime version 0.11.1

44.89   ***WARNING: TreeAnc._check_alignment_tree_gtr_consistency: NO SEQUENCE FOR
        LEAF: 'None'

44.89   ***WARNING: TreeAnc: 1 nodes don't have a matching sequence in the
        alignment. POSSIBLE ERROR.
updated tree written to results/lineage-b.1/tree.nwk
Traceback (most recent call last):
  File "/Users/corneliusromer/code/augur/augur/__init__.py", line 66, in run
    return args.__command__.run(args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/corneliusromer/code/augur/augur/refine.py", line 349, in run
    write_json(node_data, node_data_fname)
  File "/Users/corneliusromer/code/augur/augur/utils.py", line 125, in write_json
    json.dump(data, handle, indent=indent, sort_keys=sort_keys, cls=AugurJSONEncoder)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/json/__init__.py", line 179, in dump
    for chunk in iterable:
  File "/opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/json/encoder.py", line 432, in _iterencode
    yield from _iterencode_dict(o, _current_indent_level)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/json/encoder.py", line 406, in _iterencode_dict
    yield from chunks
  File "/opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/json/encoder.py", line 354, in _iterencode_dict
    items = sorted(dct.items())
            ^^^^^^^^^^^^^^^^^^^
TypeError: '<' not supported between instances of 'NoneType' and 'str'


An error occurred (see above) that has not been properly handled by Augur.
To report this, please open a new issue including the original command and the error above:
    <https://github.com/nextstrain/augur/issues/new/choose>

Interestingly, treetime logs " NO SEQUENCE FOR LEAF: 'None'". The None is suspicious.

[corneliusroemer]

The issue is that my input tree has an unnamed terminal node which is apparently something we assume is not the case but do not validate.

We probably don't want to accept unnamed terminals so the solution for now is to validate for this and error.

Tree builders don't usually output unnamed terminals so we don't usually encounter this situation. What's happened here is that we have a tree_fix.py script and that seems to output that unnamed terminal node.

[ayotomiwaa]

@corneliusroemer Did you solve this issue. I am currently running into this same error with augur refine and cannot move further in my analysis