Alphatims export selection - need help

[witszymanski]

Hi all!
I am a bit lost in the CLI of the Alphatims and I am sure it's just trivial, but I can't make it work. So In general I want to recreate this:

by using a command line and save png on the drive.

I was trying to run this:

(alphatims) vitia:~$ alphatims export selection --format png --ion_type precursors --rt_bounds -18109,-18110 --threads 6 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
*******************
* AlphaTims 0.3.2 *
*******************
Usage: alphatims export selection [OPTIONS] BRUKER_RAW_DATA
Try 'alphatims export selection -h' for help.

Error: Invalid value for '--rt_bounds': '-18109,-18110' is not a valid float.

I was trying different things (brackets, spaces etc.) with the float argument, but nothing is working. How should it look like?

I'd be grateful for the help!
Greets
Witek

[swillems]

Hey @witszymanski . If you just remove the comma, it does work on my system:
alphatims export selection --format png --ion_type precursors --rt_bounds -18109 -18110 --threads 6 220324_DQ_000_Pr_1_Slot2-26_1_10233.d

[witszymanski]

Hi! Thanks for the fast reply. So indeed it was just the coma. I get still some errors though.

If I run it on the .d folder, I get this:

(alphatims) vitia:/media/share/2022_03/DIA-diann-SP3$ alphatims export selection --format png --ion_type precursors --rt_bounds -18109 -18110 --threads 6 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
*******************
* AlphaTims 0.3.2 *
*******************
2022-04-28 13:32:08> Platform information:
2022-04-28 13:32:08> system        - Linux
2022-04-28 13:32:08> release       - 5.13.0-39-generic
2022-04-28 13:32:08> version       - #44-Ubuntu SMP Thu Mar 24 15:35:05 UTC 2022
2022-04-28 13:32:08> machine       - x86_64
2022-04-28 13:32:08> processor     - x86_64
2022-04-28 13:32:08> cpu count     - 8
2022-04-28 13:32:08> cpu frequency - 1.79 Mhz
2022-04-28 13:32:08> ram           - 13.4/38.8 Gb (available/total)
2022-04-28 13:32:08> 
2022-04-28 13:32:08> Python information:
2022-04-28 13:32:08> alphatims  - 0.3.2
2022-04-28 13:32:08> bokeh      - 2.2.3
2022-04-28 13:32:08> click      - 8.1.2
2022-04-28 13:32:08> datashader - 0.12.1
2022-04-28 13:32:08> h5py       - 3.6.0
2022-04-28 13:32:08> hvplot     - 0.7.1
2022-04-28 13:32:08> numba      - 0.55.1
2022-04-28 13:32:08> pandas     - 1.4.2
2022-04-28 13:32:08> psutil     - 5.9.0
2022-04-28 13:32:08> python     - 3.8.13
2022-04-28 13:32:08> python-lzf - 0.2.4
2022-04-28 13:32:08> pyzstd     - 0.15.2
2022-04-28 13:32:08> selenium   - 3.141.0
2022-04-28 13:32:08> tqdm       - 4.64.0
2022-04-28 13:32:08> 
2022-04-28 13:32:09> Current AlphaTims version is up-to-date with GitHub.
2022-04-28 13:32:09> 
2022-04-28 13:32:09> Running CLI command `alphatims export selection` with parameters:
2022-04-28 13:32:09> bruker_raw_data     - 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
2022-04-28 13:32:09> disable_log_stream  - False
2022-04-28 13:32:09> format              - ('png',)
2022-04-28 13:32:09> intensity_bounds    - ((None, None),)
2022-04-28 13:32:09> ion_type            - ('precursors',)
2022-04-28 13:32:09> log_file            - /home/vitia/alphatims/alphatims/logs/log_20220428133208.txt
2022-04-28 13:32:09> mobility_bounds     - ((None, None),)
2022-04-28 13:32:09> precursor_bounds    - ((None, None),)
2022-04-28 13:32:09> quad_mz_bounds      - ((None, None),)
2022-04-28 13:32:09> rt_bounds           - ((-18109.0, -18110.0),)
2022-04-28 13:32:09> threads             - 6
2022-04-28 13:32:09> tof_mz_bounds       - ((None, None),)
2022-04-28 13:32:09> x_axis              - tof_mz
2022-04-28 13:32:09> y_axis              - mobility
2022-04-28 13:32:09> 
2022-04-28 13:32:10> Loading raw data.
2022-04-28 13:32:10> Importing data from 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
2022-04-28 13:32:10> Reading frame metadata for 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
2022-04-28 13:32:11> Reading 45,281 frames with 2,735,573,863 detector events for 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
 70%|██████████████████████████████████████████████████████████████████████████████████████████▋                                      | 31826/45281 [09:21<2:35:45,  1.44it/s]
Killed

If I do it on the .hdf file generated before by Alphatims, i get this:

(alphatims) vitia:/media/share/2022_03/DIA-diann-SP3$ alphatims export selection --format png --ion_type precursors --rt_bounds -18109 -18110 --threads 6 220324_DQ_000_Pr_1_Slot2-26_1_10233.hdf 
*******************
* AlphaTims 0.3.2 *
*******************
2022-04-28 14:01:14> Platform information:
2022-04-28 14:01:14> system        - Linux
2022-04-28 14:01:14> release       - 5.13.0-39-generic
2022-04-28 14:01:14> version       - #44-Ubuntu SMP Thu Mar 24 15:35:05 UTC 2022
2022-04-28 14:01:14> machine       - x86_64
2022-04-28 14:01:14> processor     - x86_64
2022-04-28 14:01:14> cpu count     - 8
2022-04-28 14:01:14> cpu frequency - 1.97 Mhz
2022-04-28 14:01:14> ram           - 13.2/38.8 Gb (available/total)
2022-04-28 14:01:14> 
2022-04-28 14:01:14> Python information:
2022-04-28 14:01:14> alphatims  - 0.3.2
2022-04-28 14:01:14> bokeh      - 2.2.3
2022-04-28 14:01:14> click      - 8.1.2
2022-04-28 14:01:14> datashader - 0.12.1
2022-04-28 14:01:14> h5py       - 3.6.0
2022-04-28 14:01:14> hvplot     - 0.7.1
2022-04-28 14:01:14> numba      - 0.55.1
2022-04-28 14:01:14> pandas     - 1.4.2
2022-04-28 14:01:14> psutil     - 5.9.0
2022-04-28 14:01:14> python     - 3.8.13
2022-04-28 14:01:14> python-lzf - 0.2.4
2022-04-28 14:01:14> pyzstd     - 0.15.2
2022-04-28 14:01:14> selenium   - 3.141.0
2022-04-28 14:01:14> tqdm       - 4.64.0
2022-04-28 14:01:14> 
2022-04-28 14:01:14> Current AlphaTims version is up-to-date with GitHub.
2022-04-28 14:01:14> 
2022-04-28 14:01:14> Running CLI command `alphatims export selection` with parameters:
2022-04-28 14:01:14> bruker_raw_data     - 220324_DQ_000_Pr_1_Slot2-26_1_10233.hdf
2022-04-28 14:01:14> disable_log_stream  - False
2022-04-28 14:01:14> format              - ('png',)
2022-04-28 14:01:14> intensity_bounds    - ((None, None),)
2022-04-28 14:01:14> ion_type            - ('precursors',)
2022-04-28 14:01:14> log_file            - /home/vitia/alphatims/alphatims/logs/log_20220428140114.txt
2022-04-28 14:01:14> mobility_bounds     - ((None, None),)
2022-04-28 14:01:14> precursor_bounds    - ((None, None),)
2022-04-28 14:01:14> quad_mz_bounds      - ((None, None),)
2022-04-28 14:01:14> rt_bounds           - ((-18109.0, -18110.0),)
2022-04-28 14:01:14> threads             - 6
2022-04-28 14:01:14> tof_mz_bounds       - ((None, None),)
2022-04-28 14:01:14> x_axis              - tof_mz
2022-04-28 14:01:14> y_axis              - mobility
2022-04-28 14:01:14> 
2022-04-28 14:01:15> Loading raw data.
2022-04-28 14:01:15> Importing data from 220324_DQ_000_Pr_1_Slot2-26_1_10233.hdf
2022-04-28 14:01:48> Succesfully imported data from 220324_DQ_000_Pr_1_Slot2-26_1_10233.hdf
2022-04-28 14:01:48> Filtering datapoints.
2022-04-28 14:01:51> Selected 523513 datapoints.
2022-04-28 14:01:52> Something went wrong, execution incomplete!
Traceback (most recent call last):
  File "/home/vitia/alphatims/alphatims/cli.py", line 114, in parse_cli_settings
    yield kwargs
  File "/home/vitia/alphatims/alphatims/cli.py", line 446, in export_selection
    import alphatims.plotting
  File "/home/vitia/alphatims/alphatims/plotting.py", line 7, in <module>
    import hvplot.pandas
  File "/home/vitia/miniconda3/envs/alphatims/lib/python3.8/site-packages/hvplot/__init__.py", line 8, in <module>
    import holoviews as _hv
  File "/home/vitia/miniconda3/envs/alphatims/lib/python3.8/site-packages/holoviews/__init__.py", line 12, in <module>
    from .annotators import annotate                         # noqa (API import)
  File "/home/vitia/miniconda3/envs/alphatims/lib/python3.8/site-packages/holoviews/annotators.py", line 10, in <module>
    from panel.pane import PaneBase
  File "/home/vitia/miniconda3/envs/alphatims/lib/python3.8/site-packages/panel/__init__.py", line 3, in <module>
    from . import layout # noqa
  File "/home/vitia/miniconda3/envs/alphatims/lib/python3.8/site-packages/panel/layout/__init__.py", line 1, in <module>
    from .accordion import Accordion # noqa
  File "/home/vitia/miniconda3/envs/alphatims/lib/python3.8/site-packages/panel/layout/accordion.py", line 5, in <module>
    from .base import NamedListPanel
  File "/home/vitia/miniconda3/envs/alphatims/lib/python3.8/site-packages/panel/layout/base.py", line 13, in <module>
    from ..io.model import hold
  File "/home/vitia/miniconda3/envs/alphatims/lib/python3.8/site-packages/panel/io/__init__.py", line 8, in <module>
    from ..config import config
  File "/home/vitia/miniconda3/envs/alphatims/lib/python3.8/site-packages/panel/config.py", line 20, in <module>
    from .io.notebook import load_notebook
  File "/home/vitia/miniconda3/envs/alphatims/lib/python3.8/site-packages/panel/io/notebook.py", line 16, in <module>
    import bokeh.embed.notebook
  File "/home/vitia/miniconda3/envs/alphatims/lib/python3.8/site-packages/bokeh/embed/__init__.py", line 23, in <module>
    from .server import server_document, server_session
  File "/home/vitia/miniconda3/envs/alphatims/lib/python3.8/site-packages/bokeh/embed/server.py", line 25, in <module>
    from ..core.templates import AUTOLOAD_REQUEST_TAG, FILE
  File "/home/vitia/miniconda3/envs/alphatims/lib/python3.8/site-packages/bokeh/core/templates.py", line 42, in <module>
    from jinja2 import Environment, FileSystemLoader, Markup
ImportError: cannot import name 'Markup' from 'jinja2' (/home/vitia/miniconda3/envs/alphatims/lib/python3.8/site-packages/jinja2/__init__.py)

Any ideas?

[swillems]

Good to hear it works (at least it starts...)!

For the first issue, I am not completely sure. I expect it to be a memory issue, which should be fixed in v1.0.0 due to the usage of mmaps instead of RAM. You can test this one out by updateing to the latest development branch, which is not officially released just yet with: pip install "git+https://github.com/MannLabs/alphatims.git@develop#egg=alphatims[stable,plotting-stable]". Please note that the development branch might contain other unknown bugs though...

For the second issue with jinja you are in more luck. Probably this will be fixed by downgrading jinja with pip install jinja2==3.0.2 -U. Version 3.1 contains a known bug in combination with bokeh, which should be fixed in the near future. Depending on which kind of system you are using, you might also get a message stating that you need to conda install -c conda-forge firefox geckodriver

[witszymanski]

So, in order to make it work, I had to do all 3 things mentioned above :) I was doing it step by step.

1. Updating to the dev branch
   pip install "git+https://github.com/MannLabs/alphatims.git@develop#egg=alphatims[stable,plotting-stable]"
   It went further but got the jinja2 erroir

(alphatims) vitia:/media/share/2022_03/DIA-diann-SP3$ alphatims export selection --format png --ion_type precursors --rt_bounds -18109 -18110 -t 6 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
*******************
* AlphaTims 0.3.2 *
*******************
2022-04-29 09:03:03> Platform information:
2022-04-29 09:03:03> system        - Linux
2022-04-29 09:03:03> release       - 5.13.0-39-generic
2022-04-29 09:03:03> version       - #44-Ubuntu SMP Thu Mar 24 15:35:05 UTC 2022
2022-04-29 09:03:03> machine       - x86_64
2022-04-29 09:03:03> processor     - x86_64
2022-04-29 09:03:03> cpu count     - 8
2022-04-29 09:03:03> cpu frequency - 2.15 Mhz
2022-04-29 09:03:03> ram           - 24.2/38.8 Gb (available/total)
2022-04-29 09:03:03> 
2022-04-29 09:03:03> Python information:
2022-04-29 09:03:03> alphatims  - 0.3.2
2022-04-29 09:03:03> bokeh      - 2.2.3
2022-04-29 09:03:03> click      - 8.1.2
2022-04-29 09:03:03> datashader - 0.12.1
2022-04-29 09:03:03> h5py       - 3.6.0
2022-04-29 09:03:03> hvplot     - 0.7.1
2022-04-29 09:03:03> numba      - 0.55.1
2022-04-29 09:03:03> pandas     - 1.4.2
2022-04-29 09:03:03> psutil     - 5.9.0
2022-04-29 09:03:03> python     - 3.8.13
2022-04-29 09:03:03> python-lzf - 0.2.4
2022-04-29 09:03:03> pyzstd     - 0.15.2
2022-04-29 09:03:03> selenium   - 3.141.0
2022-04-29 09:03:03> tqdm       - 4.64.0
2022-04-29 09:03:03> 
2022-04-29 09:03:03> Current AlphaTims version is up-to-date with GitHub.
2022-04-29 09:03:03> 
2022-04-29 09:03:03> Running CLI command `alphatims export selection` with parameters:
2022-04-29 09:03:03> bruker_raw_data     - 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
2022-04-29 09:03:03> disable_log_stream  - False
2022-04-29 09:03:03> format              - ('png',)
2022-04-29 09:03:03> intensity_bounds    - ((None, None),)
2022-04-29 09:03:03> ion_type            - ('precursors',)
2022-04-29 09:03:03> log_file            - /home/vitia/alphatims/alphatims/logs/log_20220429090302.txt
2022-04-29 09:03:03> mobility_bounds     - ((None, None),)
2022-04-29 09:03:03> precursor_bounds    - ((None, None),)
2022-04-29 09:03:03> quad_mz_bounds      - ((None, None),)
2022-04-29 09:03:03> rt_bounds           - ((-18109.0, -18110.0),)
2022-04-29 09:03:03> threads             - 6
2022-04-29 09:03:03> tof_mz_bounds       - ((None, None),)
2022-04-29 09:03:03> x_axis              - tof_mz
2022-04-29 09:03:03> y_axis              - mobility
2022-04-29 09:03:03> 
2022-04-29 09:03:04> Loading raw data.
2022-04-29 09:03:04> Importing data from 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
2022-04-29 09:03:04> Reading frame metadata for 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
2022-04-29 09:03:05> Reading 45,281 frames with 2,735,573,863 detector events for 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 45281/45281 [09:57<00:00, 75.77it/s]
2022-04-29 09:13:03> Indexing 220324_DQ_000_Pr_1_Slot2-26_1_10233.d...
2022-04-29 09:13:03> Opening handle for 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
2022-04-29 09:13:04> Fetching mobility values from 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
2022-04-29 09:13:04> Closing handle for 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
2022-04-29 09:13:04> Opening handle for 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
2022-04-29 09:13:04> Fetching mz values from 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
2022-04-29 09:13:04> Closing handle for 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
2022-04-29 09:13:04> Indexing quadrupole dimension
2022-04-29 09:13:08> Succesfully imported data from 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
2022-04-29 09:13:08> Filtering datapoints.
2022-04-29 09:13:09> Selected 523513 datapoints.
2022-04-29 09:13:10> Something went wrong, execution incomplete!
Traceback (most recent call last):
  File "/home/vitia/alphatims/alphatims/cli.py", line 114, in parse_cli_settings
    yield kwargs
  File "/home/vitia/alphatims/alphatims/cli.py", line 446, in export_selection
    import alphatims.plotting
  File "/home/vitia/alphatims/alphatims/plotting.py", line 7, in <module>
    import hvplot.pandas
  File "/home/vitia/miniconda3/envs/alphatims/lib/python3.8/site-packages/hvplot/__init__.py", line 8, in <module>
    import holoviews as _hv
  File "/home/vitia/miniconda3/envs/alphatims/lib/python3.8/site-packages/holoviews/__init__.py", line 12, in <module>
    from .annotators import annotate                         # noqa (API import)
  File "/home/vitia/miniconda3/envs/alphatims/lib/python3.8/site-packages/holoviews/annotators.py", line 10, in <module>
    from panel.pane import PaneBase
  File "/home/vitia/miniconda3/envs/alphatims/lib/python3.8/site-packages/panel/__init__.py", line 3, in <module>
    from . import layout # noqa
  File "/home/vitia/miniconda3/envs/alphatims/lib/python3.8/site-packages/panel/layout/__init__.py", line 1, in <module>
    from .accordion import Accordion # noqa
  File "/home/vitia/miniconda3/envs/alphatims/lib/python3.8/site-packages/panel/layout/accordion.py", line 5, in <module>
    from .base import NamedListPanel
  File "/home/vitia/miniconda3/envs/alphatims/lib/python3.8/site-packages/panel/layout/base.py", line 13, in <module>
    from ..io.model import hold
  File "/home/vitia/miniconda3/envs/alphatims/lib/python3.8/site-packages/panel/io/__init__.py", line 8, in <module>
    from ..config import config
  File "/home/vitia/miniconda3/envs/alphatims/lib/python3.8/site-packages/panel/config.py", line 20, in <module>
    from .io.notebook import load_notebook
  File "/home/vitia/miniconda3/envs/alphatims/lib/python3.8/site-packages/panel/io/notebook.py", line 16, in <module>
    import bokeh.embed.notebook
  File "/home/vitia/miniconda3/envs/alphatims/lib/python3.8/site-packages/bokeh/embed/__init__.py", line 23, in <module>
    from .server import server_document, server_session
  File "/home/vitia/miniconda3/envs/alphatims/lib/python3.8/site-packages/bokeh/embed/server.py", line 25, in <module>
    from ..core.templates import AUTOLOAD_REQUEST_TAG, FILE
  File "/home/vitia/miniconda3/envs/alphatims/lib/python3.8/site-packages/bokeh/core/templates.py", line 42, in <module>
    from jinja2 import Environment, FileSystemLoader, Markup
ImportError: cannot import name 'Markup' from 'jinja2' (/home/vitia/miniconda3/envs/alphatims/lib/python3.8/site-packages/jinja2/__init__.py)

2. Doing the downgrade of jinja pip install "git+https://github.com/MannLabs/alphatims.git@develop#egg=alphatims[stable,plotting-stable]"
   Again, different error connected to the firefox thing:

(alphatims) vitia:/media/share/2022_03/DIA-diann-SP3$ alphatims export selection --format png --ion_type precursors --rt_bounds -18109 -18110 -t 6 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
*******************
* AlphaTims 0.3.2 *
*******************
2022-04-29 09:14:02> Platform information:
2022-04-29 09:14:02> system        - Linux
2022-04-29 09:14:02> release       - 5.13.0-39-generic
2022-04-29 09:14:02> version       - #44-Ubuntu SMP Thu Mar 24 15:35:05 UTC 2022
2022-04-29 09:14:02> machine       - x86_64
2022-04-29 09:14:02> processor     - x86_64
2022-04-29 09:14:02> cpu count     - 8
2022-04-29 09:14:02> cpu frequency - 2.08 Mhz
2022-04-29 09:14:02> ram           - 24.0/38.8 Gb (available/total)
2022-04-29 09:14:02> 
2022-04-29 09:14:02> Python information:
2022-04-29 09:14:02> alphatims  - 0.3.2
2022-04-29 09:14:02> bokeh      - 2.2.3
2022-04-29 09:14:02> click      - 8.1.2
2022-04-29 09:14:02> datashader - 0.12.1
2022-04-29 09:14:02> h5py       - 3.6.0
2022-04-29 09:14:02> hvplot     - 0.7.1
2022-04-29 09:14:02> numba      - 0.55.1
2022-04-29 09:14:02> pandas     - 1.4.2
2022-04-29 09:14:02> psutil     - 5.9.0
2022-04-29 09:14:02> python     - 3.8.13
2022-04-29 09:14:02> python-lzf - 0.2.4
2022-04-29 09:14:02> pyzstd     - 0.15.2
2022-04-29 09:14:02> selenium   - 3.141.0
2022-04-29 09:14:02> tqdm       - 4.64.0
2022-04-29 09:14:02> 
2022-04-29 09:14:02> Current AlphaTims version is up-to-date with GitHub.
2022-04-29 09:14:02> 
2022-04-29 09:14:02> Running CLI command `alphatims export selection` with parameters:
2022-04-29 09:14:02> bruker_raw_data     - 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
2022-04-29 09:14:02> disable_log_stream  - False
2022-04-29 09:14:02> format              - ('png',)
2022-04-29 09:14:02> intensity_bounds    - ((None, None),)
2022-04-29 09:14:02> ion_type            - ('precursors',)
2022-04-29 09:14:02> log_file            - /home/vitia/alphatims/alphatims/logs/log_20220429091402.txt
2022-04-29 09:14:02> mobility_bounds     - ((None, None),)
2022-04-29 09:14:02> precursor_bounds    - ((None, None),)
2022-04-29 09:14:02> quad_mz_bounds      - ((None, None),)
2022-04-29 09:14:02> rt_bounds           - ((-18109.0, -18110.0),)
2022-04-29 09:14:02> threads             - 6
2022-04-29 09:14:02> tof_mz_bounds       - ((None, None),)
2022-04-29 09:14:02> x_axis              - tof_mz
2022-04-29 09:14:02> y_axis              - mobility
2022-04-29 09:14:02> 
2022-04-29 09:14:03> Loading raw data.
2022-04-29 09:14:03> Importing data from 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
2022-04-29 09:14:03> Reading frame metadata for 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
2022-04-29 09:14:04> Reading 45,281 frames with 2,735,573,863 detector events for 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 45281/45281 [10:03<00:00, 74.98it/s]
2022-04-29 09:24:09> Indexing 220324_DQ_000_Pr_1_Slot2-26_1_10233.d...
2022-04-29 09:24:09> Opening handle for 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
2022-04-29 09:24:09> Fetching mobility values from 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
2022-04-29 09:24:09> Closing handle for 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
2022-04-29 09:24:09> Opening handle for 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
2022-04-29 09:24:09> Fetching mz values from 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
2022-04-29 09:24:09> Closing handle for 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
2022-04-29 09:24:09> Indexing quadrupole dimension
2022-04-29 09:24:14> Succesfully imported data from 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
2022-04-29 09:24:14> Filtering datapoints.
2022-04-29 09:24:14> Selected 523513 datapoints.
2022-04-29 09:24:20> Exporting results to /media/share/2022_03/DIA-diann-SP3/220324_DQ_000_Pr_1_Slot2-26_1_10233_selection.png
2022-04-29 09:24:22> Something went wrong, execution incomplete!
Traceback (most recent call last):
  File "/home/vitia/alphatims/alphatims/cli.py", line 114, in parse_cli_settings
    yield kwargs
  File "/home/vitia/alphatims/alphatims/cli.py", line 479, in export_selection
    hv.save(plot, f"{output_file_name_base}.png", fmt="png")
  File "/home/vitia/miniconda3/envs/alphatims/lib/python3.8/site-packages/holoviews/util/__init__.py", line 820, in save
    return renderer_obj.save(obj, filename, fmt=fmt, resources=resources,
  File "/home/vitia/miniconda3/envs/alphatims/lib/python3.8/site-packages/holoviews/plotting/renderer.py", line 624, in save
    plot.layout.save(basename, embed=True, resources=resources, title=title)
  File "/home/vitia/miniconda3/envs/alphatims/lib/python3.8/site-packages/panel/viewable.py", line 721, in save
    return save(self, filename, title, resources, template,
  File "/home/vitia/miniconda3/envs/alphatims/lib/python3.8/site-packages/panel/io/save.py", line 142, in save
    return save_png(model, filename=filename, template=template,
  File "/home/vitia/miniconda3/envs/alphatims/lib/python3.8/site-packages/panel/io/save.py", line 45, in save_png
    state.webdriver = webdriver_control.create()
  File "/home/vitia/miniconda3/envs/alphatims/lib/python3.8/site-packages/bokeh/io/webdriver.py", line 150, in create
    driver = self._create(kind)
  File "/home/vitia/miniconda3/envs/alphatims/lib/python3.8/site-packages/bokeh/io/webdriver.py", line 168, in _create
    raise RuntimeError("Neither firefox and geckodriver nor a variant of chromium browser and " \
RuntimeError: Neither firefox and geckodriver nor a variant of chromium browser and chromedriver are available on system PATH. You can install the former with 'conda install -c conda-forge firefox geckodriver'.

3. Doing the conda firefox thing: conda install -c conda-forge firefox geckodriver
   It worked:

(alphatims) vitia:/media/share/2022_03/DIA-diann-SP3$ alphatims export selection --format png --ion_type precursors --rt_bounds -18109 -18110 -t 6 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
*******************
* AlphaTims 0.3.2 *
*******************
2022-04-29 09:25:27> Platform information:
2022-04-29 09:25:27> system        - Linux
2022-04-29 09:25:27> release       - 5.13.0-39-generic
2022-04-29 09:25:27> version       - #44-Ubuntu SMP Thu Mar 24 15:35:05 UTC 2022
2022-04-29 09:25:27> machine       - x86_64
2022-04-29 09:25:27> processor     - x86_64
2022-04-29 09:25:27> cpu count     - 8
2022-04-29 09:25:27> cpu frequency - 2.11 Mhz
2022-04-29 09:25:27> ram           - 24.9/38.8 Gb (available/total)
2022-04-29 09:25:27> 
2022-04-29 09:25:27> Python information:
2022-04-29 09:25:27> alphatims  - 0.3.2
2022-04-29 09:25:27> bokeh      - 2.2.3
2022-04-29 09:25:27> click      - 8.1.2
2022-04-29 09:25:27> datashader - 0.12.1
2022-04-29 09:25:27> h5py       - 3.6.0
2022-04-29 09:25:27> hvplot     - 0.7.1
2022-04-29 09:25:27> numba      - 0.55.1
2022-04-29 09:25:27> pandas     - 1.4.2
2022-04-29 09:25:27> psutil     - 5.9.0
2022-04-29 09:25:27> python     - 3.8.13
2022-04-29 09:25:27> python-lzf - 0.2.4
2022-04-29 09:25:27> pyzstd     - 0.15.2
2022-04-29 09:25:27> selenium   - 3.141.0
2022-04-29 09:25:27> tqdm       - 4.64.0
2022-04-29 09:25:27> 
2022-04-29 09:25:27> Current AlphaTims version is up-to-date with GitHub.
2022-04-29 09:25:27> 
2022-04-29 09:25:27> Running CLI command `alphatims export selection` with parameters:
2022-04-29 09:25:27> bruker_raw_data     - 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
2022-04-29 09:25:27> disable_log_stream  - False
2022-04-29 09:25:27> format              - ('png',)
2022-04-29 09:25:27> intensity_bounds    - ((None, None),)
2022-04-29 09:25:27> ion_type            - ('precursors',)
2022-04-29 09:25:27> log_file            - /home/vitia/alphatims/alphatims/logs/log_20220429092527.txt
2022-04-29 09:25:27> mobility_bounds     - ((None, None),)
2022-04-29 09:25:27> precursor_bounds    - ((None, None),)
2022-04-29 09:25:27> quad_mz_bounds      - ((None, None),)
2022-04-29 09:25:27> rt_bounds           - ((-18109.0, -18110.0),)
2022-04-29 09:25:27> threads             - 6
2022-04-29 09:25:27> tof_mz_bounds       - ((None, None),)
2022-04-29 09:25:27> x_axis              - tof_mz
2022-04-29 09:25:27> y_axis              - mobility
2022-04-29 09:25:27> 
2022-04-29 09:25:28> Loading raw data.
2022-04-29 09:25:28> Importing data from 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
2022-04-29 09:25:28> Reading frame metadata for 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
2022-04-29 09:25:29> Reading 45,281 frames with 2,735,573,863 detector events for 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 45281/45281 [10:06<00:00, 74.71it/s]
2022-04-29 09:35:37> Indexing 220324_DQ_000_Pr_1_Slot2-26_1_10233.d...
2022-04-29 09:35:37> Opening handle for 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
2022-04-29 09:35:37> Fetching mobility values from 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
2022-04-29 09:35:37> Closing handle for 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
2022-04-29 09:35:37> Opening handle for 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
2022-04-29 09:35:37> Fetching mz values from 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
2022-04-29 09:35:37> Closing handle for 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
2022-04-29 09:35:37> Indexing quadrupole dimension
2022-04-29 09:35:43> Succesfully imported data from 220324_DQ_000_Pr_1_Slot2-26_1_10233.d
2022-04-29 09:35:43> Filtering datapoints.
2022-04-29 09:35:43> Selected 523513 datapoints.
2022-04-29 09:35:51> Exporting results to /media/share/2022_03/DIA-diann-SP3/220324_DQ_000_Pr_1_Slot2-26_1_10233_selection.png
2022-04-29 09:35:57> Analysis done in 629.67 seconds.

Question:
Can I also get this plot somehow generated?

Thanks for the help!! If this will work how I hope, it will be of a great help to generate some standard reporting outputs!
All the best
Witek

[swillems]

Hmm, your update from the dev branch still uses Alphatims 0.3.2. My bad, you probably should have appended a -U to upgrade to v1.0.0 (I create a new environment, so upgrade and install are essentially the same): pip install "git+https://github.com/MannLabs/alphatims.git@develop#egg=alphatims[stable,plotting-stable]", which at the very least should have removed the jinja error. You got it working though, that's probably what you care about the most at this time anyways.

Funnily enough, I have actually not implemented a TIC export before even though this is most likely a simple copy-paste from the nbs tutorial to the CLI. If you want to help and make a PR, feel free to try it out and implement it (git pull from dev branch first). Otherwise I will probably get to this in a later release, but I have no idea about that timeline...

[swillems]

Another side note: if you use v1.0.0 and make an hdf first, this should be significantly faster to do a data selection you have done now (seems to take 629s, probably can be done 100 times faster if you have an hdf format)

[witszymanski]

But you still need a lot of time to create the hdf files ;)

Anyway, so it means that in the current state, I can't export the TICs right?

To be honest, I am mainly a mass spectrometrist and not a bioinformatician (only partly so, because of the necessity), so I would have no clue how to implement this in the code myself :(

[witszymanski]

From hdf:

(alphatims) vitia:/media/share/2022_03/DIA-diann-SP3$ alphatims export selection --format png --ion_type precursors --rt_bounds -18109 -18110 -t 6 220324_DQ_000_Pr_1_Slot2-26_1_10233.hdf 
*******************
* AlphaTims 0.3.2 *
*******************
2022-04-29 11:31:25> Platform information:
2022-04-29 11:31:25> system        - Linux
2022-04-29 11:31:25> release       - 5.13.0-39-generic
2022-04-29 11:31:25> version       - #44-Ubuntu SMP Thu Mar 24 15:35:05 UTC 2022
2022-04-29 11:31:25> machine       - x86_64
2022-04-29 11:31:25> processor     - x86_64
2022-04-29 11:31:25> cpu count     - 8
2022-04-29 11:31:25> cpu frequency - 2.25 Mhz
2022-04-29 11:31:25> ram           - 25.6/38.8 Gb (available/total)
2022-04-29 11:31:25> 
2022-04-29 11:31:25> Python information:
2022-04-29 11:31:25> alphatims  - 0.3.2
2022-04-29 11:31:25> bokeh      - 2.2.3
2022-04-29 11:31:25> click      - 8.1.2
2022-04-29 11:31:25> datashader - 0.12.1
2022-04-29 11:31:25> h5py       - 3.6.0
2022-04-29 11:31:25> hvplot     - 0.7.1
2022-04-29 11:31:25> numba      - 0.55.1
2022-04-29 11:31:25> pandas     - 1.4.2
2022-04-29 11:31:25> psutil     - 5.9.0
2022-04-29 11:31:25> python     - 3.8.13
2022-04-29 11:31:25> python-lzf - 0.2.4
2022-04-29 11:31:25> pyzstd     - 0.15.2
2022-04-29 11:31:25> selenium   - 3.141.0
2022-04-29 11:31:25> tqdm       - 4.64.0
2022-04-29 11:31:25> 
2022-04-29 11:31:25> Current AlphaTims version is up-to-date with GitHub.
2022-04-29 11:31:25> 
2022-04-29 11:31:25> Running CLI command `alphatims export selection` with parameters:
2022-04-29 11:31:25> bruker_raw_data     - 220324_DQ_000_Pr_1_Slot2-26_1_10233.hdf
2022-04-29 11:31:25> disable_log_stream  - False
2022-04-29 11:31:25> format              - ('png',)
2022-04-29 11:31:25> intensity_bounds    - ((None, None),)
2022-04-29 11:31:25> ion_type            - ('precursors',)
2022-04-29 11:31:25> log_file            - /home/vitia/alphatims/alphatims/logs/log_20220429113125.txt
2022-04-29 11:31:25> mobility_bounds     - ((None, None),)
2022-04-29 11:31:25> precursor_bounds    - ((None, None),)
2022-04-29 11:31:25> quad_mz_bounds      - ((None, None),)
2022-04-29 11:31:25> rt_bounds           - ((-18109.0, -18110.0),)
2022-04-29 11:31:25> threads             - 6
2022-04-29 11:31:25> tof_mz_bounds       - ((None, None),)
2022-04-29 11:31:25> x_axis              - tof_mz
2022-04-29 11:31:25> y_axis              - mobility
2022-04-29 11:31:25> 
2022-04-29 11:31:26> Loading raw data.
2022-04-29 11:31:26> Importing data from 220324_DQ_000_Pr_1_Slot2-26_1_10233.hdf
2022-04-29 11:31:58> Succesfully imported data from 220324_DQ_000_Pr_1_Slot2-26_1_10233.hdf
2022-04-29 11:31:58> Filtering datapoints.
2022-04-29 11:31:59> Selected 523513 datapoints.
2022-04-29 11:32:04> Exporting results to /media/share/2022_03/DIA-diann-SP3/220324_DQ_000_Pr_1_Slot2-26_1_10233_selection.png
2022-04-29 11:32:09> Analysis done in 43.80 seconds.

[swillems]

Slower than I expected it (I probably can speed that up a little more), but at least you don't have to get a coffee while waiting anymore;)

[swillems]

Yes, but that conversion is a one time thing. I presume you want to do more with your data than just visualize a single plot, especially since you already mention you want a TIC as well, and a second plot means reading in the data at least twice in the current implementation.

Yes, you are right that currently exporting the TIC from CLI is not a (convenient) option. In principle you can select all data and export an XIC, which amounts to the same thing. Realistically this will probably kill your memory though...

No worries about the programming skills, I'll see if I can get this on my short-term todo list without making any promises just yet;)

[witszymanski]

Realistically this will probably kill your memory though...

I tried this, it does kill everything :D

Thanks for the help, and I'll follow new developments here!
Greets!