Thoughts on the recently published paper testing several GEM tools, including MICOM?

[PathogeNish]

Hi, I recently happened upon the following publication . They have compared in vitro mono and co-culture growth rates for ~30 microbes to growth rate estimates obtained using various GEM tools, including MICOM.

Their results seem to indicate that none of the SoTA GEM tools, including MICOM, were able to reliably capture growth rate estimates in an accurate manner.

I was wondering, @cdiener, what you thought of the publication, the accuracy of the tests, the accuracy of the defined media and the possible improvements they suggested (implementation of flux sampling, Pareto optimality, better GEM reconstructions etc.).

Thanks in advance for your thoughts!

[cdiener]

It embeds itself into various other benchmarks, for instance there is also this one. I think the study is mostly correct, though they are comparing apples to oranges a bit since growth rates were not measured for any of their co-cultures. They used abundance ratios to match to growth rates which are not the same thing. There is a small issue with how MICOM is used since MICOM expects abundances to be steady state abundances and not the inoculation abundance which is what was used there.

The results match out observations that especially for small communities grown in vitro predictions are sometimes off because individual models quality and the absence of a real steady state system violate the model assumptions. See for instance Fig. 2 here that compares predictions from low and high diversity in vitro communities. Pareto optimality is already used in MICOM. I don't think flux sampling will ever be feasible in models of this size, and better GEM reconstructions are of course always desirable 😄.