Handling polytomies

[davidebbo]

Alright, I get to start the first discussion! 😄

We briefly discussed this, and you probably answered it, but I didn't retain everything. I'm trying to better understands how OneZoom handles polytomies.

Taking for example the genus Tupaia (treeshrews), EoL has it as a pure polytomy:

(Tupaia_palawanensis_ott30082,Tupaia_nicobarica_ott30084,Tupaia_minor_ott67360,Tupaia_belangeri_ott106287,Tupaia_tana_ott222037,Tupaia_moellendorffi_ott337533,Tupaia_chrysogaster_ott337554,Tupaia_montana_ott513235,Tupaia_salatana_ott513936,Tupaia_chinensis_ott513958,Tupaia_longipes_ott513961,Tupaia_splendidula_ott551390,Tupaia_glis_ott574727,Tupaia_picta_ott574859,Tupaia_gracilis_ott603401,Tupaia_javanica_ott603403,Tupaia_dorsalis_ott603407,Tupaia_castanea_ott7655205,Tupaia_clarissa_ott7655206,Tupaia_concolor_ott7655207,Tupaia_dissimilis_ott7655208,Tupaia_frenata_ott7655209,Tupaia_lacernata_ott7655210,Tupaia_longicauda_ott7655211,Tupaia_obscura_ott7655212,Tupaia_ultima_ott7655213)Tupaia_ott574730;

While OneZoom's bespoke tree has a complex structure (extracted from my generated AllLife_full_tree.phy):

(((((Tupaia_javanica_ott603403:19.60396,Tupaia_picta_ott574859:19.60396):0.0,(Tupaia_palawanensis_ott30082:5.678925,Tupaia_moellendorffi_ott337533:5.678925):13.925035):0.0,Tupaia_longipes_ott513961:19.60396):0.0,(Tupaia_minor_ott67360:19.60396,(((Tupaia_chrysogaster_ott337554:19.60396,Tupaia_dorsalis_ott603407:19.60396):0.0,Tupaia_glis_ott574727:19.60396):0.0,((Tupaia_gracilis_ott603401:19.60396,(Tupaia_splendidula_ott551390:19.60396,Tupaia_tana_ott222037:19.60396):0.0):0.0,(Tupaia_nicobarica_ott30084:19.60396,Tupaia_belangeri_ott106287:19.60396):0.0):0.0):0.0):0.0):0.0,Tupaia_montana_ott513235:19.60396)Tupaia_ott574730:0.155587;

My question is whether this resolution is happening because:

1. There are reasons to think that this is the correct phylogeny tree
2. We need to get rid of all polytomies because our UI can only handle binary trees, and we picked this arbitrary structure

[hyanwong]

That's a really good question. For public users, most of the the details at https://www.onezoom.org/data_sources.html. For more technical users, sources for the data are noted at the top of the .phy files in a comment. Here, for example, it's this file:

https://github.com/OneZoom/OZtree/blob/main/OZprivate/data/OZTreeBuild/AllLife/BespokeTree/include_noAutoOTT/BoreoeutheriaOneZoom_altered.phy

In this case it's noted that we are using the arrangement from Bininda-Emonds et. al (https://doi.org/10.1038/nature05634), for most of the placental mammals except the primates & colugos. So we aren't taking the OpenTree topologies into account for this part of the OneZoom tree. Unfortunately, since this annotation is done by hand, often the exact details are missing. I've just looked at the Bininda-Emonds tree (https://static-content.springer.com/esm/art%3A10.1038%2Fnature05634/MediaObjects/41586_2007_BFnature05634_MOESM362_ESM.txt) and it looks like all of Tupaia is in a big polytomy. It could be that we tweaked it by hand and forgot to annotate the files, or it could be that we took a slightly different tree from the publicly published one in that paper, as this is one of the files from the original OneZoom mammal tree (@jrosindell might know a bit better here)

A clue can be found if you search for those species in the .phy file. You'll find them embedded like this:

(((((Tupaia_javanica:25.2,Tupaia_picta:25.2):0.0,(Tupaia_palawanensis:7.3,Tupaia_moellendorffi:7.3):17.9):0.0,Tupaia_longipes:25.2):0.0,(Tupaia_minor:25.2,(((Tupaia_chrysogaster:25.2,Tupaia_dorsalis:25.2):0.0,Tupaia_glis:25.2):0.0,((Tupaia_gracilis:25.2,(Tupaia_splendidula:25.2,Tupaia_tana:25.2):0.0):0.0,(Tupaia_nicobarica:25.2,Tupaia_belangeri:25.2):0.0):0.0):0.0):0.0):0.0,Tupaia_montana:25.2)Tupaia:0.2):0.1)Tupaiidae:5.8)SCANDENTIA:60.5)

It looks like there are a lot of 0-length internal branches, which implies that (most) of the polytomies have been arbitrarily broken (apart from the pairing of Tupaia_palawanensis & Tupaia_moellendorffi). Some of the files form the original OneZoom had arbitrarily broken polytomies like this. I don't know where the palawanensis/moellendorffi growing come from, sorry.

It's a bit ugly that the polytomy breaking is hand-coded in the file: this should probably be removed (nevertheless, we do need to make sure that the polytomies are always resolved in the same way every time we run the tree-building script: this is handled in the code, I think). Zero-length branches will appear as a polytomy when switching to the "polytomy" view on OneZoom:

So hopefully the final outcome for the user in this case is to note that it is a polytomy. This is also shown in the traditional (bifurcating) tree view because zero length internal branches cause that part of the tree to be shown in a lighter grey colour, with no highlighted internal nodes:

[hyanwong]

Ideally, of course, we would find a Tupaia phylogeny online that broke at least some of those polytomies, and input it here

https://tree.opentreeoflife.org/curator

There's a mtDNA and 16S rRNA tree for a few of them here:

https://www.nature.com/articles/s41598-022-04907-7

a tree based on 13 protein-coding genes is in their supplementary material too, but as usual (and highly annoyingly) they don't actually provide the Newick file for download in the data availability section, so if we wanted to input that paper into OpenTree we would need to digitise it by hand (I would tend to use the protein-coding tree, I think).

There might be other published trees of Tupaia elsewhere too. I haven't looked. I sometimes do that for areas of particular interest, but usually hit problems like the one above that stop the trees being able to be easily incorporated automatically into OneZoom or even input into the OpenTree curator.

[davidebbo]

Thanks @hyanwong for the detailed explanation! I'm realizing that there is a lot of great material on the OneZoom site that I have not read, so I definitely need to do this. I also did not realize that there was a polytomy view. I just tried it and it's fantastic! Tupaia link for my own reference.

we do need to make sure that the polytomies are always resolved in the same way every time we run the tree-building script

I assume this is to preserve tree stability and not confuse users, as any resolution is technically correct. So if we were to come up with logic to auto-resolve, the constraint would be:

1. Stability from run to run. And ideally some level of stability if a species is added/removed, though that may be more challenging.
2. Balance: we wouldn't want it to resolve in a linear way that goes super deep, but in a balanced way that tries to have the same number of leaves on each side of a Node (so depth would be ~log2(n)). Right now, the manual tree is sort of like this, but not optimally (e.g. how did T. montana end up by itself as 'basal'?)

As for the mtDNA/rRNA article, it's indeed unfortunate that they don't include a Newick. At least they have a diagram which makes it easy to come up with a Newick. Maybe I'll try doing that for the fun, and to learn about the EoL submission process.

[hyanwong]

Yes, any resolution is correct. We currently use the dendropy resolve_polytomies function. As an alternative to DendroPy, I am slightly hankering to use my own group's tree processing software, which is explicitly designed for large trees, and for which we have a draft for fast newick reading (and have a polytomy resolving routine), but our library has a different focus so is much less flexible than DendroPy, and may not have the ability to do what we need for OneZoom, especially when creating some branches with zero lengths (impossible in our toolkit) and unknown lengths.

[davidebbo]

This is what I came up with based on that diagram:

(Tupaia_chrysogaster_ott337554:19.60396,Tupaia_salatana_ott513936:19.60396,Tupaia_chinensis_ott513958:19.60396,Tupaia_longipes_ott513961:19.60396,Tupaia_glis_ott574727:19.60396,Tupaia_picta_ott574859:19.60396,Tupaia_gracilis_ott603401:19.60396,Tupaia_javanica_ott603403:19.60396,Tupaia_dorsalis_ott603407:19.60396,Tupaia_castanea_ott7655205:19.60396,Tupaia_clarissa_ott7655206:19.60396,Tupaia_concolor_ott7655207:19.60396,Tupaia_dissimilis_ott7655208:19.60396,Tupaia_frenata_ott7655209:19.60396,Tupaia_lacernata_ott7655210:19.60396,Tupaia_longicauda_ott7655211:19.60396,Tupaia_obscura_ott7655212:19.60396,Tupaia_ultima_ott7655213:19.60396,(Tupaia_palawanensis_ott30082:5.678925,Tupaia_moellendorffi_ott337533:5.678925):13.925035,(Tupaia_belangeri_ott106287:17.25,((((Tupaia_splendidula_ott551390:6.5,Tupaia_montana_ott513235:6.5):3.25,Tupaia_tana_ott222037:9.75):3.25,Tupaia_minor_ott67360:13):2.5,Tupaia_nicobarica_ott30084:15.5):1.75):2.35396):0;

But now I just realized that they have another diagram which has more species, but doesn't have proper date ranges (most branch lengths are 1). Maybe it's a matter of combining the two in a clever way? Well, this is obviously a complex problem, and I'm way out of my league :)

[hyanwong]

That's great! It's good to contribute this sort of thing into the OpenTree (NB not EoL - I think you had a typo above) - now that you've done the hard work you might as well submit it so it's there for the future (https://tree.opentreeoflife.org/curator). They aren't so worried about dates, but it's a nice-to-have, so I wouldn't be too worried about the other diagram (although I see that the one you have is only a mitogenic tree, which don't always reflect nuclear genome inheritance)

I can talk you through the submission process if you like. You would have to drop the Tupaia species not in the printed tree in the paper, and add one (or ideally all) of the "outgroup" taxa or rodents, lagomorphs, primates, and Dermoptera), then mark _ Tupaia_ as the ingroup (i.e. the clade of interest to this study).

[hyanwong]

But now I just realized that they have another diagram which has more species

That's a 16S rRNA tree. The branch lengths that you mention aren't 1: the numbers on the plot represent bayesian posterior probabilities of support for each node on the tree. The lengths would either have to be measured by hand (yuck!) or dug out of some file somewhere. Those lengths aren't dates, either, they are essentially measures of molecular change, which is nearly always how these trees are drawn. That's scientifically more reasonable, and involves fewer assumptions, but often not very useful when trying to figure out true dates.

[hyanwong]

Also note the protein coding gene tree in the supplementary. Also without branch lengths explicitly written down, grr:

Since it looks to me to have the same topology as the mtDNA tree, I would be more happy with the mtDNA tree being a good choice to submit to the OpenTree.

[hyanwong]

Actually, I probably got the palawanensis/moellendorffi grouping from this paper, and forgot to record it:

https://www.sciencedirect.com/science/article/abs/pii/S1055790311002156

That claims to have a phylogeny of all the species (pasted here as behind a paywall):

Fig. 3. Phylogeny and estimated divergence times from BEAST analysis. Hatched bars show the median and 95% credibility intervals for calibrated nodes, with the calibration distributions, while gray bars show the same values for uncalibrated nodes. Line type indicates posterior probability for bipartitions. Letters at tips of some branches identify individuals as in Table 1.

Again, no computer-readable trees, although there is a NEXUS file (without trees in it0 of the data that will allow you to run your own analysis.

[hyanwong]

Again, no computer-readable trees

You can see why it's hard to get studies into these large "synthetic" phylogenies like the OpenTree, when there are millions of species to look at and very little machine-readable data made available.

It would be nice to get this particular tree in, I suppose, but we can't be doing that for all 2 million species on OneZoom.

[davidebbo]

You can see why it's hard to get studies into these large "synthetic" phylogenies like the OpenTree, when there are millions of species to look at and very little machine-readable data made available.

Indeed, I can see that it would be an enormous task given the scope of the problem. e.g. I analyzed mammalia for genera with >= 50 polytomies and found a few (from OpenTree newick):

Genus	Common name	Polynomy count
Crocidura	White-toothed shrews	286
Sorex	Long-tailed shrews	112
Rhinolophus	Horseshoe bats	93
Myotis	Mouse-eared bats	82
Lepus	Hares	68
Acomys	Spiny mice	60
Rattus	Rats	51
Murina	Tube-nosed bats	50

For Tupaia, that latest study you found is indeed much more complete! I'm surprised that these things are behind paywalls, as I naively expected most things in the science world to be wide open.

For what it's worth, I turned it into a newick, which I will put here for the record, whether we use it or not. I added indentation for readability.

A few notes:

• The first two are interesting, because they're not in the Tupaia genus. Doesn't putting them in this tree essentially make a statement that they are Tupaia? In fact, Urogale everetti is listed as Tupaia everetti on Wikipedia (but not in the study).
• I ignored that it listed T. belangeri 4 times, because I didn't know what to make of it :)
• I did not include branch length, as it's hard to infer them precisely from the diagram
• I removed all the species not in the study
• I added the suggested outgroups. But how do I mark Tupaia as the ingroup?

(
  (
    Rodentia_ott864593,
    Lagomorpha_ott644242
  ),
  (
    (
      Primates_ott913935,
      Dermoptera_ott987673
    ),
    (
      (
        Anathana_ellioti_ott603405,
        (
          Urogale_everetti_ott768484,
          (
            (
              Tupaia_longipes_ott513961,
              Tupaia_chrysogaster_ott337554
            ),
            (
              Tupaia_glis_ott574727,
              Tupaia_belangeri_ott106287
            )
          )
        )
      ),
      (
        (
          Tupaia_dorsalis_ott603407,
          Tupaia_gracilis_ott603401
        ),
        (
          (
            Tupaia_javanica_ott603403,
            Tupaia_nicobarica_ott30084
          ),
          (
            (
              Tupaia_moellendorffi_ott337533,
              Tupaia_palawanensis_ott30082
            ),
            (
              Tupaia_minor_ott67360,
              (
                Tupaia_tana_ott222037,
                (
                  Tupaia_picta_ott574859,
                  (
                    Tupaia_montana_ott513235,
                    Tupaia_splendidula_ott551390
                  )
                )
              )
            )
          )
        )
      )
    )Tupaia_ott574730
  )
)Euarchontoglires_ott392222;

Rendering using http://etetoolkit.org/treeview/:

This leaves the following uncategorized species, for the record:

(
  Tupaia_salatana_ott513936,
  Tupaia_chinensis_ott513958,
  Tupaia_castanea_ott7655205,
  Tupaia_clarissa_ott7655206,
  Tupaia_concolor_ott7655207,
  Tupaia_dissimilis_ott7655208,
  Tupaia_frenata_ott7655209,
  Tupaia_lacernata_ott7655210,
  Tupaia_longicauda_ott7655211,
  Tupaia_obscura_ott7655212,
  Tupaia_ultima_ott7655213
)Tupaia_ott574730;

Anyway, I will leave it to you to decide whether it's worth contributing to OpenTree. I'm happy to do it, if only as a learning exercise.

[hyanwong]

I think just this once it's worth contributing both trees if possible. As you say, this is a good learning exercise, and I guess you've already found out how surprisingly tricky it is to deal with the data sources. Here are some replies to your questions, which are probably relevant for submission:

I'm surprised that these things are behind paywalls, as I naively expected most things in the science world to be wide open.

It's getting much better now. Also in the last 5 years there has been a move to preprinting your studies, so even if the original is behind a paywall, a preprint is sometimes available.

The first two are interesting, because they're not in the Tupaia genus. Doesn't putting them in this tree essentially make a statement that they are Tupaia? In fact, Urogale everetti is listed as Tupaia everetti on Wikipedia (but not in the study).

Scandentia contains a few things that aren't Tupaia. If there's enough evidence from different studies that they are nested within Tupaia then eventually some taxonomist will change the genus names, e.g. putting all as Tupaia. This may have happened already in the case of everetti, which is probably why wikipedia has done this (they cite exactly this paper, see the discussion at https://en.wikipedia.org/wiki/Tupaia_(mammal)#Classification_and_taxonomic_history). Regardless of which, the OpenTree should synonymize them: if you type "Tupaia everetti" into the search it will know that you mean Urogale everetti (or the other way round).

I ignored that it listed T. belangeri 4 times, because I didn't know what to make of it :)

You should probably put those in. It just means that (if the tree is correct) some individual T. belangeri genomes are more closely related to a glis genome than others. It may be that T. belangeri represents multiple species, that it should be a subspecies of T. glis or that there has been hybridisation. Better to put it into the OpenTree and let the algorithm sort it out.

I did not include branch length, as it's hard to infer them precisely from the diagram

Yes, I see why. But actually I think it would probably be better (for OneZoom) to put in a rough estimate, even if it isn't exactly correct.

I removed all the species not in the study

Great.

I added the suggested outgroups. But how do I mark Tupaia as the ingroup?

That's something that you do when adding the data to the OpenTree. If you have a go, you will probably see how it works, or I can talk you through it.

By the way, note that you don't need the OTT numbers for input. There's a reasonably intuitive interface for mapping the species names to OTT ids within the curator app on the OpenTree website.

[davidebbo]

Hmmm, looking at OpenTree, Scandentia is closer to the Glires than to the Primates, so the way I placed the outgroups is probably going to clash (I followed the OneZoom model). Anyway, easy to adjust to match theirs.

[hyanwong]

My guess is Scandentia is one of those taxa where different positions with respect to Glires vs Primates is supported by different studies, as a result of a short divergence time and "incomplete lineage sorting" (ILS). This is another reason why I try in some case to use "expert opinion" (i.e. me, after reading through the studies) to decide on a particular resolution of a polytomy. In this case, I know that both ((Primates, Scandentia),Rodents) and ((Rodents, Scandentia), Primates) are found in studies, but ((Rodents, Primates),Scandentia) tends not to be. If we placed a polytomy ((Rodents, Scandentia, Primates)) the random resolution algorithm could easily pick the unsupported order. So I have deemed it better to choose something that I know is supported by several studies. In fact, in this case I'm pretty sure that nearly all studies suggest ((Primates, Scandentia),Rodents), so I have gone with that.

I have omitted the position of the colugos because that again is contentious. However there is a nice new paper from 2022 which shows the support for various resolutions of this. I paste the summary diagram below, which supports the order of branching we use (hoorah!)

This one is open access: https://www.mdpi.com/2073-4425/13/5/774

[davidebbo]

Thanks for all the feedback, Yan! I think I understand all the needed changes for the tree, and I'll make that a weekend project. Thanks for also explaining the reasoning behind ((Primates, Scandentia), Glires), which makes complete sense.

I think just this once it's worth contributing both trees if possible

By 'both trees', do you mean both the first one I had derived from this diagram (with just Tupaia 6 species), and the much more complete one from the paywall article? As far as I can see, the second one is a superset of the first (i.e. with no clashes), which makes the first redundant. But I may be misunderstanding.

For the branch lengths, one challenge is that the study only shows the time scale, but not the branch lengths (as I understand, it's called an ultrametric trees or dendrogram). So I need convert that to a tree with branch lengths (an "additive tree").

One possible plan for doing that in a less error prone way is to author a transitional tree that directly matches the dendrogram, and then write code to convert it into an additive tree.

e.g. if A split from B 7 MYA, and C split from A&B 20 MYA, I'd write:

(
  (
    A,
    B
  ):7,
  C
):20

And the code (e.g. using dendropy) would turn that into a proper newick tree:

(
  (
    A:7,
    B:7
  ):13,
  C:20
)

The first is a clear abuse of the newick branch length syntax, but no one would see that one. Hope I'm making sense! :)

[davidebbo]

Ok, here is the next iteration, with the following changes:

• Removed OTT numbers
• Changed (Primates, Dermoptera, Tupaia) to a polytomy, since that's what OneZoom does
• Added all the Tupaia belangeri instances, matching the study
• Estimated the ages from the study, and calculated the resulting branch lengths. That was the harder part. I wrote a utility for this (using the approach I suggested above) that I sent a PR for (Add two new utilities #626).

Here is the resulting newick:

((Rodentia:5.2,Lagomorpha:20.0):2.4,(Primates:5.0,Dermoptera:55.0,((Anathana_ellioti:17.5,(Urogale_everetti:16.0,((Tupaia_chrysogaster:6.0,Tupaia_longipes:6.0):7.0,((Tupaia_glis:7.0,Tupaia_belangeri:7.0):2.0,(Tupaia_belangeri:4.0,(Tupaia_belangeri:3.0,Tupaia_belangeri:3.0):1.0):5.0):4.0):3.0):1.5):2.5,((Tupaia_dorsalis:15.0,Tupaia_gracilis:15.0):3.5,((Tupaia_javanica:13.0,Tupaia_nicobarica:13.0):3.5,((Tupaia_moellendorffi:1.0,Tupaia_palawanensis:1.0):13.0,(Tupaia_minor:13.0,(Tupaia_tana:8.5,(Tupaia_picta:8.0,(Tupaia_montana:7.0,Tupaia_splendidula:7.0):1.0):0.5):4.5):1.0):2.5):2.0):1.5)Tupaia:50.0):5.0);

Which looks like this (I switched to https://beta.phylo.io/ for rendering):

Questions:

• Should I add the interior node taxa names (e.g. Glire), or does it not matter?
• For the outgroups (e.g. Lagomorpha), I don't like that the rendering makes them look 'extinct'. Should I:
  1. Keep it this way
  2. Pick one random species for each outgroup? Either in addition of or instead of including the parent taxon.
  3. Just keep the outgroup taxon, but pretend that it has age 0, so that it will extend all the way to now. This is sort of what they do in the study diagram with genera like Tarsius (which is shown lined up with Tupaia species).