Focus Group: OpenMS Visualization Tools and Web apps

[matteopilz]

We should discuss how to proceed with the different visualization tools and if we want to use a common package / language. I can give a quick overview of what I have been adding so far.

1. We have the plotting functions (https://pyopenms.readthedocs.io/en/latest/apidocs/_autosummary/pyopenmssubmodules/pyopenms.plotting.html#module-pyopenms.plotting) which take a MSSpectrum object from OpenMS to visualize.
2. We have potentially a conversion function to convert from our MSSpectrum to spectrum utils (https://spectrum-utils.readthedocs.io/en/latest/plotting.html).
   Both tools are exclusively for plotting mass spectra and depend on matplotlib. Spectrum utils also allows interactive plotting with altair, which should also be possible after converting our MSSpectrum with our conversion function.

It would be good to discuss the major applications we need for visualization in OpenMS like spectrum, chromatogram, interactivity, etc. It would also be good to discuss whether we want to build something included in OpenMS or something separate which OpenMS uses as dependency.

[matteopilz]

@timosachsenberg, @jpfeuffer, @Arslan-Siraj, @axelwalter, @jcharkow, @JeeH-K

[Ayesha-Feroz]

@matteopilz Include me too; I am also interested.

[timosachsenberg]

Thanks for starting this discussion, I think this is a great and also necessary discussion prior to the retreat.

I think these are good point and I will add a few more:

A. First, we should summarize the current state of our visualization capabilities (interactive vs. static) and data supported (spectra, chromatograms, feature maps, mobilograms, 2D peak maps).

B. Discuss where interactive plotting is needed, e.g. using Plotly with Streamlit or JavaScript integrations with Vue as Streamlit component. How to handle big files?

C. What are the implications to maintenance and automated testing. We all know: getting it to run once is easy but keeping it running and maintained is hard.

D. what should be our priorities? How can we consolidate efforts? e.g. Can there be a joint library that can be published?

E. come up with a list of tasks and tackle them in the team?

[axelwalter]

Good points.

A. For the web apps interactive plots are certainly useful. Is there a downside to using an interactive library and for both interactive and static plots? I would avoid duplicating work here.

B. For large datasets plotly WebGL could be an option up to a certain point. How far can we push the limitations e.g. with binning, rasterization using datashader?

D. Extending the pyopenms plotting library for the more common OpenMS datastructures would be great. However, duplicating work that has already been done in e.g. spectrum utils should be avoided. @timosachsenberg do you know if they would be open to plotting functionalities besides spectra (such as feature maps etc.). If yes we could perhaps contribute there and fill the gaps. Otherwise rely on spectrum utils for actual spectra and extend pyopenms plotting for other data structures.

[JeeH-K]

B. I think Datashader would be a great library to use, but I never succeeded in using it within Streamlit...
Also, for plotly WebGL, I experienced quite a fash rendering with Dash, but not with Streamlit. And if the dataset is extensively large, a web app would not be a smart way to go since a web browser has some memory limitations. Maybe having two options can be a solution? For large data (e.g., 5GB), on python, and more interactive work on web browser?

[t0mdavid-m]

For large datafiles there is also the option of not keeping all datasets in memory. For instance within the docker container one could setup a Restful API (e.g. with Flask; I looked into it briefly and could not find any way to do it with Streamlit) exposing some type of lightweight database. This way data could be loaded on demand.

[matteopilz]

@t0mdavid-m

[matteopilz]

With regards to point A stated by Timo, it would be great, if everyone could write a few sentences about what they have implemented respectively. I am not sure, who has built tools themselves, but I think I remember @JeeH-K and @t0mdavid-m doing something with Streamlit and Javascript.

[axelwalter]

So far I have good experience with plotly, using plotly WebGL for larger plots such as 2D maps and forms, to avoid constant re-rendering. You can get pretty far with if data is prepared in advance in dataframes and saved in data formats such as feather, parquet or pickle for quick access.

[t0mdavid-m]

I implemented TaggerViewer which is based on FLASHViewer and is a visualization tool for FLASHTagger.

It introduces new/modified Vue.js components for sequence tag visualization. Of course I also adapted the dataflow but I did not conceptually change anything (so it is as discussed in the presentation by @JeeH-K).

Since sequence tags can be detected for multiple proteins, I adapted the sequence view to update the sequence based on the selected protein in the table:

The shading of amino acids denotes the coverage and the tag selected in the appropriate table is also highlighted. I think we should think about how to combine both versions of this component (e.g. have one parent SequenceView component, which is modular and can be adapted for both FlashViewer and TaggerViewer).

Furthermore, I modified the PlotlyLineplot component (which is used for spectral visualization by FlashViewer). Sequence tags are visualized on top of the deconvolved spectrum and the relevant peaks are highlighted:

Clicking on the mass labels of one of the highlighted peaks updates the plot to show the raw spectrum with peaks belonging to the the selected mass highlighted:

I think this component could be generalized to highlight arbitrary masses and could also be of use for similar visualization tasks.

[JeeH-K]

So, adding to Tom, FLASHViewer has 2D Spectrum plots, heatmaps, tables, 3D signal/noise plot, and some protein sequence-related components (as in the image below). Plotting/figure options are not that challenging to implement on the TypeScript/JavaScript side; I believe the current obstacle is how we want to design/develop the data structure for this. FLASHViewer (and I believe FLASHTagger as well) has quite specific input types, and we need to update those for more "general" cases.

[axelwalter]

A starting point for discussions / focus group tasks for the web apps:

Deployment

• develop a standard/secure protocol to add/update apps
• central hub for all OpenMS apps (on OpenMS website?)

Testing

• test Windows builds (check if app starts without errors)
• fix error in windows packages with streamlit 1.30

How to organize modules such as workflow manager / visualization?

• part of streamlit template?
• separate modules hosted on PyPi?

Update and polish the template

• add documentation how to build an app as a page
• add documentation on visualization options (including OpenMS plotting functions and Javascript components)
• update entry page
• review existing pages (update if required)
• add admin section (view workspaces etc)
• indicator for server load?

Workflow manager framework

• feedback, how can it be improved?
• support for any CLI tool/custom Python scripts with auto parameters etc
• add test with streamlit testing framework

[axelwalter]

Could the JavaScript components be wrapped in a streamlit streamlit custom component and published to PyPi?
Would that be even useful or are they very app specific and not re-usable?

[JeeH-K]

I believe we can. Currently, FLASHViewer implements vis components via TypeScript to have a more concise structure, then translates codes to javascript for usage in FLASHViewer. In this case, more structural implementation is possible, and I believe this is better for maintaining purpose as well.
I wasn't actually aware that we can publish as a custom component, but it seems like an interesting idea. However, if components are wrapped in a Streamlit custom component, it can be quite restrictive in some ways... We cannot use that component outside of Streamlit.

[axelwalter]

During the Developer retreat we can give an intro on the current state of web apps and visualization.

Some ideas:

• streamlit-template and deployment
• new workflow manager framework
• short overview of current web apps (FLASHViewer, UmetaFlow)
• visualization components with JavaScript
• visualization of MS data (spectrum utils, pyopenms plotting)

I could do the template, deployment, workflow framework and UmetaFlow.
@JeeH-K maybe JavaScript stuff and FLASHViewer?
@matteopilz maybe spectrum utils, pyopenms plotting?
@t0mdavid-m you have a web app as well, right?

Any other suggestions welcome.

[t0mdavid-m]

Yes I implemented TaggerViewer based on FlashViewer. Sure I can give an introduction about its current state.

[matteopilz]

I can prepare some slides for the plotting functions.

[jcharkow]

Several members of the roestlab have recently implemented MassDash. This is a streamlit based DIA/Targeted visualization so it is capable of chromatogram viewing, some 2D heatmap plots with ion mobility, and some 3D plots. These plots are all interactive using bokeh or plotly libraries. https://github.com/Roestlab/massdash . Unfortunately due to a family emergency, I won't be able to make it to the retreat this year but I can prepare some slides which describe the capabilities of the application. Depending on the time maybe I can zoom in for the discussion.

[timosachsenberg]

Several members of the roestlab have recently implemented MassDash. This is a streamlit based DIA/Targeted visualization so it is capable of chromatogram viewing, some 2D heatmap plots with ion mobility, and some 3D plots. These plots are all interactive using bokeh or plotly libraries. https://github.com/Roestlab/massdash . Unfortunately due to a family emergency, I won't be able to make it to the retreat this year but I can prepare some slides which describe the capabilities of the application. Depending on the time maybe I can zoom in for the discussion.

Sorry to hear that. Family emergencies can be overwhelming, so please don't feel obliged to zoom in for a discussion.

@axelwalter @matteopilz @JeeH-K @t0mdavid-m, could you please take a look at MassDash? I'm particularly interested in whether the visualization components could be utilized by us. I'm curious about how they compare to the JS components in terms of functionalities and maintainability.

I'm still hopeful for a single-language solution ;). Even if that's not feasible immediately or for the first version, I believe it's the right time to consider how to modularize. For example, we could encapsulate the JS Vue into components or explore emerging works in progress (WIP) like Streamlit components (Bokeh 3, etc.) that could serve as a slimmer, more maintainable replacement in the future.

While I haven't delved into the details, it appears that there are ongoing developments that might provide more lightweight interfaces with JS components, allowing users to stay in the Python world (which imho is a huge win if we want to gain external users).

[t0mdavid-m]

I just looked into everything in more detail and what I found is that Streamlit rerenders the entire page (i.e. all components) when their input data is updated:

https://discuss.streamlit.io/t/avoiding-chart-redraws-on-irrelevant-changes/33171

The bad news is that this is simply the default behavior of streamlit – when one input is changed, it reruns the whole app. Normally you solve issues like this by using st.experimental_memo to cache slow parts of your app’s code (e.g. fetching data). And, st.experimental_memo has recently been updated so that you actually can cache charts like this, but in my experimentation it didn’t seem to add much performance benefit (presumably actually rendering all those charts on the screen, even if they are pre-cached, is still slow).

@JeeH-K previously reported performance issues when creating the graphs with Plotly Python. I think this might explain it. Basically, the Vue.js component seems to be a workaround on what I figured is a fundamental limitation of Streamlit at the moment.

Instead of having multiple components that are all redrawn every time new input data is selected, we have a single Vue.js component (containing all plots and tables) that does not change from a Streamlit point of few. Within the Vue.js component we are able to dynamically update the plots instead of redrawing the entire page.

Unfortunately, this would mean that, if we were to modularize the Vue.js plots/tables into individual Streamlit compoents, we would loose the performance benefit experienced with Vue.js. My experience with MassDash was also consistent with this (although it is not as problematic as there are less plots and they do not need to be updated as frequently as with FlashViewer/TaggerViewer).

[timosachsenberg]

Would that also be the case with current experimental developments? e.g., https://discuss.streamlit.io/t/partial-reruns-preview-share-your-feedback-and-cool-implementations/60851/20

[axelwalter]

The partial re-run feature would be a game changer. Until now the only (without JavaScript) solution was putting large data graphs in separate pages or limiting displayed information.
Also this could be really useful for dynamic input widgets. With forms the constant re-run is disabled, however, the widgets can not dynamically react to other widget values within the form.

[t0mdavid-m]

This feature does indeed look very promising!

To my understanding it allows the creation of containers that act as "independent Streamlit pages". Thus, in situations where all plots/tables are interdependent on each other (as is the case in FlashViewer/TaggerViewer) it seems like the plots would need to be placed in a single container to facilitate updates. Hence, to me it does not seem like there would be much benefit for this specific use case.

[axelwalter]

Could you chain multiple partial functions so that if input happens somewhere in the middle, the earlier widgets don't update?

[JeeH-K]

Hi, folks in this discussion! @axelwalter @t0mdavid-m @timosachsenberg and I will have a discussion about this focus group via Zoom tomorrow (Feb 19) at 11 AM CET We will use the "MS SIG" Zoom Meeting room for this, and everyone is welcome to join us. Let me know if you need the link for the meeting

[JeeH-K]

Summary of the discussion:

1. Let's implement the "ToppView" alternative in the Streamlit template. This can validate the usage of Streamlit in most of the "general" cases. We need to start with the simplest cases (e.g., input file types of mzML, featureXML, idXML / chromatogram view, scan table, spectrum plots / etc.)
2. Stick with the Python side for plotting for now. Current JS components (i.e., FLASHViewer/TaggerViewer) are implemented for specific use cases. "Universal plotting" components can (probably) be implemented only in Python, and we can test this by implementing the "ToppView" alternative.
3. JS components are preferred to be used together with the Python visualization components. In any case, all components need to be modularized. Make use of the upcoming "partial rerun" feature from Streamlit when it's available.

Correct me if anything is missing or wrong here.

Also, we will have a short Zoom meeting to discuss the initial design of the ToppView alternative this Thursday (Feb 22) at 11 am CET. Feel free to join if you have any good ideas!

[timosachsenberg]

something that might be relevant for the discussion (e.g., data exchange / storage) https://github.com/bigbio/quantms.io

[timosachsenberg]

As it does not cover spectra: https://github.com/compomics/ThermoRawFileParser/blob/master/Writer/ParquetSpectrumWriter.cs#L115-L133 could be something that could be built upon (e.g. supporting ion mobility)

[timosachsenberg]

[axelwalter]

Action points for web apps / visualizations:

streamlit web app template:

• update documentation, create figures for publication
• Windows executables:
  • add testing (make sure app launches without errors, on each page)
  • fix issues with latest streamlit
• TOPP workflow framework:
  • improve based on experience with real use case
• together with docs team: web app hub on OpenMS website
• implement basic TOPPView capabilities using visualization module

visualization module:

• separate repository for MS data visualization with pyopenms
• useful not only in web apps, but also in Jupyter notebooks etc
• schema for reading data structures (e.g MSExperiment) into pandas/polars Dataframe
• plotters for common use cases, spectra, chromatograms, 2D peak maps, mirror plots...
• explore new streamlit functionality useful for visualizations: partials, bidirectional tables/figures
• optional: chose plotting back-end
• optional: chose static/interactive (see plotting back-end)

[jcharkow]

I think what could also be cool is a port to run locally. E.g. using https://github.com/whitphx/stlite
Will require additional compiling though so not sure if it is worth the headache.

We have some DIA plotting capabilities in massDash so it would be nice to incorporate some of that.

Currently loading large MzMLFiles still takes a long time even with OnDIscImplementation() unless metadata is not loaded. Workaround where only loading parts of the metadata might be useful. Alternatively, we could support vendor specific formats.

[axelwalter]

The serverless streamlit sounds really cool! Would you be interested in checking that out?
We could use your DIA plotting for sure, would be a welcome addition.
We will start with some regular meetings for this focus group soon, would like to participate?

[jcharkow]

Yes I would like to participate as well. Please make the meeting an afternoon time (e.g. 3 or 4) because of the time difference.

I checked out stlite and it is a bit more complicated than originally thought. Because pyopenms is not a pure python package we would have to integrate it in pyodide. I've tried briefly but running into some issues. I'm not sure if all the extra effort is worth it at this time right now.

[timosachsenberg]

maybe interesting for the vue.js guys: https://discuss.streamlit.io/t/a-method-to-create-components-almost-entirely-in-python/68357