History parser

[kidpixo]

Hi,
we are using Research FT-IR Microscope HYPERION and I was trying to get the microscope data out of OPUS file.

brukeropusreader works, but I get one single spectrum with all 100 observation (10x10 grid) sequentially stored.

I did split the data, looking for the first and last point of a couple of spectra I saved as txt, but this is too hack-ish.

I wrote an history parser that works also on single spectra, maybe could be useful for this repo?

The code is here, the output and the parsing could absolutely be improved.

[tdyku]

Hi!
Thanks for suggestion. Do you have example of OPUS file where such a parser could be useful?

[kidpixo]

Hi,
absolutely! I have to dig something when I'll be back at work.
The one I'm using now is from our student and she going to publish the result at some point.
We have for sure some file when she was learning how to use the microscope.

From Microscope data I got :

opus_data['AB'] firs value            : 0
opus_data['AB'][1:8]                  : "random" values
opus_data['AB'][8:16006]              : 0
opus_data['AB'].shape                 : 787206 # total lenght of spectral data
opus_data['AB Data Parameter']['NPT'] : 7674 # single spectrum lenght or number of (X) points

I found the start-end of the first few spectra looking for the data from txt files. The spectra are separated by 39 "random" values, maybe there are some information written there.

spectrum effective length : 7673
gap_length    : 39

The history parser was my idea because I found some interesti information in the History (like image position in the microscope coordinate system) and I was hoping to find something more. And example of parsed data is attached below.

[kidpixo]

I was trying also opusFC but stuart-cls/python-opusfc-dist#9 doesn't let me read the data and I cannot figure out if their code is public.

[kidpixo]

[['Operator:Admin',
  'Version 8.1 Build: 8, 1, 29 20180416',
  'Y0146_ICT_all spectra.0',
  'Unvalidated environment\x00Extract Data',
  'Refl/Multiple',
  '2019/05/14 15:58:42 (GMT+2)\x00\x00'],
 {'comment': 'Extract DataScSm/Multiple2019/05/14 15:58:42 (GMT+2)',
  'PostrunExtract': {'Preamble': ['"C:\\Users\\Public\\Documents\\Bruker\\OPUS_8.1.29\\MEAS\\Hyp-Mct-Refl-15x_190514.2" 1:Refl/Multiple'],
   'EXS': 0, 'EXE': 0, 'ENT': 0, 'ENE': 0, 'ECO': 3,
   'XTP': 'D:\\Claudia\\Y0146',
   'XTN': 'Y0146_ICT_all spectra.0',
   'XTI': 0, 'EAB': 0, 'ELF': 0, 'EIL': '0-99', 'EUL': 1, 'EDM': 1}},
 {'comment': 'Extract DataIgSm/Multiple2019/05/14 15:58:42 (GMT+2)',
  'PostrunExtract': {'Preamble': ['"C:\\Users\\Public\\Documents\\Bruker\\OPUS_8.1.29\\MEAS\\Hyp-Mct-Refl-15x_190514.2" 1:Refl/Multiple'],
   'EXS': 0, 'EXE': 0, 'ENT': 0, 'ENE': 0, 'ECO': 3,
   'XTP': 'D:\\Claudia\\Y0146',
   'XTN': 'Y0146_ICT_all spectra.0',
   'XTI': 0, 'EAB': 0, 'ELF': 0, 'EIL': '0-99', 'EUL': 1, 'EDM': 1}},
 {'comment': ['Modify annotationAnnotation2019/06/03 13:39:52 (GMT+2)',
   {'Operation': ' Add',
    'XPos': 46576.313129,
    'YPos': 13867.008921,
    'Width': 46663.414875,
    'Height': 15660.280157}],
  'PostrunExtract': {'Preamble': ['"C:\\Users\\Public\\Documents\\Bruker\\OPUS_8.1.29\\MEAS\\Hyp-Mct-Refl-15x_190514.2" 1:Refl/Multiple'],
   'EXS': 0, 'EXE': 0, 'ENT': 0, 'ENE': 0, 'ECO': 3,
   'XTP': 'D:\\Claudia\\Y0146',
   'XTN': 'Y0146_ICT_all spectra.0',
   'XTI': 0, 'EAB': 0, 'ELF': 0, 'EIL': '0-99', 'EUL': 1, 'EDM': 1}},
 {'comment': ['Modify annotationAnnotation2019/06/03 13:39:57 (GMT+2)',
   {'Operation': ' Modify', 'Annotation ID': 0}],
  'ModifyAnnotation': {'Preamble': ['"::this::":Annotation'],
   'ANO': 'ADD',
   'AXP': 46576.313129,
   'AYP': 13867.008921,
   'AZP': 0.0,
   'ACX': 46663.414875,
   'ACY': 15660.280157,
   'ACZ': 0.0, 'ATY': 12, 'AFI': 0, 'ACO': 0, 'ARP': 0,
   'AHD': 0, 'AGP': 0, 'AAT': 0, 'AAX': 0.0, 'AAY': 0.0,
   'AAZ': 0.0, 'AAW': 0.0, 'AAH': 0.0, 'AAD': 0.0, 'ANX': 0,
   'ANY': 0, 'ANZ': 0, 'APX': 0, 'APY': 0, 'APZ': 0, 'AAE': 0,
   'APW': 0.0, 'APH': 0.0, 'APA': 0.0, 'ATT': ''}},
 {'comment': ['Modify annotationAnnotation2019/06/03 13:40:31 (GMT+2)',
   {'Operation': ' Modify', 'Annotation ID': 0}],
  'ModifyAnnotation': {'Preamble': ['"::this::":Annotation'],
   'ANO': 'MOD',
   'AXP': 46673.66151,
   'AYP': 13759.412515,
   'AZP': 0.0,
   'ACX': 46663.414063,
   'ACY': 15660.280273,
   'ACZ': 0.0, 'ATY': 12, 'AFI': 0, 'ACO': 0, 'ARP': 0,
   'AHD': 0, 'AGP': 0, 'AAT': 0, 'AAX': 0.0, 'AAY': 0.0,
   'AAZ': 0.0, 'AAW': 0.0, 'AAH': 0.0, 'AAD': 0.0, 'ANX': 0,
   'ANY': 0, 'ANZ': 0, 'APX': 0, 'APY': 0, 'APZ': 0, 'AAE': 0,
   'APW': 0.0, 'APH': 0.0, 'APA': 0.0, 'ATT': ''}},
 {'comment': ['Modify annotationAnnotation2019/06/03 13:40:41 (GMT+2)',
   {'Operation': ' Modify', 'Annotation ID': 0}],
  'ModifyAnnotation': {'Preamble': ['"::this::":Annotation'],
   'ANO': 'MOD',
   'AXP': 46673.66151,
   'AYP': 13759.412515,
   'AZP': 0.0,
   'ACX': 46663.414063,
   'ACY': 15660.280273,
   'ACZ': 0.0, 'ATY': 12, 'AFI': 0, 'ACO': 0, 'ARP': 0,
   'AHD': 0, 'AGP': 0, 'AAT': 0, 'AAX': 0.0, 'AAY': 0.0,
   'AAZ': 0.0, 'AAW': 0.0, 'AAH': 0.0, 'AAD': 0.0, 'ANX': 0,
   'ANY': 0, 'ANZ': 0, 'APX': 0, 'APY': 0, 'APZ': 0,
   'AAE': 0, 'APW': 0.0, 'APH': 0.0, 'APA': 0.0, 'ATT': ''}},
 {'comment': ['Modify annotationAnnotation2019/06/03 13:45:06 (GMT+2)',
   {'Operation': ' Add',
    'XPos': 45541.339444,
    'YPos': 13841.39076,
    'Width': 45607.946662,
    'Height': 15655.156525}],
  'ModifyAnnotation': {'Preamble': ['"::this::":Annotation'],
   'ANO': 'MOD',
   'AXP': 46673.66151,
   'AYP': 13759.412515,
   'AZP': 0.0,
   'ACX': 45607.945849,
   'ACY': 13759.412763,
   'ACZ': 0.0, 'ATY': 12, 'AFI': 0, 'ACO': 0, 'ARP': 0,
   'AHD': 0, 'AGP': 0, 'AAT': 0, 'AAX': 0.0, 'AAY': 0.0,
   'AAZ': 0.0, 'AAW': 0.0, 'AAH': 0.0, 'AAD': 0.0, 'ANX': 0,
   'ANY': 0, 'ANZ': 0, 'APX': 0, 'APY': 0, 'APZ': 0,
   'AAE': 0, 'APW': 0.0, 'APH': 0.0, 'APA': 0.0, 'ATT': ''}},
 {'comment': ['Modify annotationAnnotation2019/06/03 13:45:12 (GMT+2)',
   {'Operation': ' Modify', 'Annotation ID': 1}],
  'ModifyAnnotation': {'Preamble': ['"::this::":Annotation'],
   'ANO': 'ADD',
   'AXP': 45541.339444,
   'AYP': 13841.39076,
   'AZP': 0.0,
   'ACX': 45607.946662,
   'ACY': 15655.156525,
   'ACZ': 0.0, 'ATY': 12, 'AFI': 0, 'ACO': 0, 'ARP': 0,
   'AHD': 0, 'AGP': 0, 'AAT': 0, 'AAX': 0.0, 'AAY': 0.0,
   'AAZ': 0.0, 'AAW': 0.0, 'AAH': 0.0, 'AAD': 0.0, 'ANX': 0,
   'ANY': 0, 'ANZ': 0, 'APX': 0, 'APY': 0, 'APZ': 0,
   'AAE': 0, 'APW': 0.0, 'APH': 0.0, 'APA': 0.0, 'ATT': ''}},
 {'comment': 'Undo Changes2019/06/03 13:49:38 (GMT+2)',
  'ModifyAnnotation': {'Preamble': ['"::this::":Annotation'],
   'ANO': 'MOD',
   'AXP': 45602.823429,
   'AYP': 13754.288879,
   'AZP': 0.0,
   'ACX': 45607.945313,
   'ACY': 15655.15625,
   'ACZ': 0.0, 'ATY': 12, 'AFI': 0, 'ACO': 0, 'ARP': 0,
   'AHD': 0, 'AGP': 0, 'AAT': 0, 'AAX': 0.0, 'AAY': 0.0,
   'AAZ': 0.0, 'AAW': 0.0, 'AAH': 0.0, 'AAD': 0.0, 'ANX': 0,
   'ANY': 0, 'ANZ': 0, 'APX': 0, 'APY': 0, 'APZ': 0,
   'AAE': 0, 'APW': 0.0, 'APH': 0.0, 'APA': 0.0, 'ATT': ''}},
 {'comment': '', 'Restore': {}}]

[AlbertEmil]

I was trying also opusFC but stuart-cls/python-opusfc-dist#9 doesn't let me read the data and I cannot figure out if their code is public.

opusFC code is available here: https://github.com/stuart-cls/python-opusfc-dist

[kidpixo]

I was trying also opusFC but stuart-cls/python-opusfc-dist#9 doesn't let me read the data and I cannot figure out if their code is public.

opusFC code is available here: https://github.com/stuart-cls/python-opusfc-dist

Maybe I'm missing something, but I see only the documentation and no code. From python-opusfc-dist/README.md :

Documentation and issue-tracking for opusFC cython/python module

For user-targeted software that can load and analyze OPUS files, please see https://quasar.codes

Online documentation can be found at https://stuart-cls.github.io/python-opusfc-dist/

Binary wheels are available from pypi:

    pip install opusFC

I understand only binary wheels are available. Please correct me if am I wrong.

[jroshk]

Did anyone sort this out for FTIR imaging?