tomo name error if the name starts with a 0 in TS alignment Relion5 #1079
Comments
ASKrebs
changed the title
|
Hi, |
|
Sure. I am copying the one that caused the error: data_global loop_ |
|
thanks, I'll try to replicate the issue |
|
Maybe the tiltseries star file (#2 in the loop of the star file before ie 00255.star) might help you too. I am pasting it below with only the first input line. To get rid of the error, I need to change the data_00255 to the name with letters too. data_00255 loop_ |
|
thanks, I've replicated the error myself so I'll do a little digging, thanks for the report! |
|
Ok, I've found the issue, it's actually not a RELION bug, it's a bug with a starfile reader which RELION uses. Either way, good spot! Should be fixed at some point, but for now, just name your mdoc files with a prefix like TS_. Thanks |
|
Sorry for taking so long with this, it required fixes across multiple projects, it should be fixed within a few days |
I am running into a bug during TS alignment when my tiltseries is named with 00s at the beginning and the name consists only of digits. When I change the name (rlnTomoName) in the star files to also contain letters, it runs. So the name 00255 doesn't work but TS00255 does.
A short command where it doesn't work is:
relion_python_tomo_align_tilt_series IMOD:fiducials --nominal-fiducial-diameter-nanometers 10 --tilt-series-star-file CtfFind/tilt_series_ctf.star --output-directory AlignTiltSeries/ --pipeline_control AlignTiltSeries/
I get the following (shortened due to max.characters) error message in the terminal:
[09:35:26] Extracting metadata for tilt series. cli_fiducials.py:29
/opt/conda/envs/relion-5.0/lib/python3.10/site-packages/starfile/parser.py:177: FutureWarning: The 'delim_whitespace' keyword in pd.read_csv is deprecated and will be removed in a future version. Use
sep='\s+'insteaddf = pd.read_csv(
/opt/conda/envs/relion-5.0/lib/python3.10/site-packages/starfile/parser.py:194: FutureWarning: errors='ignore' is deprecated and will raise in a future version. Use to_numeric without passing
errorsand catch exceptions explicitly insteadself.dataframes[key] = df.apply(pd.to_numeric, errors='ignore')
/opt/conda/envs/relion-5.0/lib/python3.10/site-packages/starfile/parser.py:177: FutureWarning: The 'delim_whitespace' keyword in pd.read_csv is deprecated and will be removed in a future version. Use
sep='\s+'insteaddf = pd.read_csv(
/opt/conda/envs/relion-5.0/lib/python3.10/site-packages/starfile/parser.py:194: FutureWarning: errors='ignore' is deprecated and will raise in a future version. Use to_numeric without passing
errorsand catch exceptions explicitly insteadself.dataframes[key] = df.apply(pd.to_numeric, errors='ignore')
/opt/conda/envs/relion-5.0/lib/python3.10/site-packages/starfile/parser.py:177: FutureWarning: The 'delim_whitespace' keyword in pd.read_csv is deprecated and will be removed in a future version. Use
sep='\s+'insteaddf = pd.read_csv(
/opt/conda/envs/relion-5.0/lib/python3.10/site-packages/starfile/parser.py:194: FutureWarning: errors='ignore' is deprecated and will raise in a future version. Use to_numeric without passing
errorsand catch exceptions explicitly insteadself.dataframes[key] = df.apply(pd.to_numeric, errors='ignore')
/opt/conda/envs/relion-5.0/lib/python3.10/site-packages/starfile/parser.py:177: FutureWarning: The 'delim_whitespace' keyword in pd.read_csv is deprecated and will be removed in a future version. Use
sep='\s+'insteaddf = pd.read_csv(
/opt/conda/envs/relion-5.0/lib/python3.10/site-packages/starfile/parser.py:194: FutureWarning: errors='ignore' is deprecated and will raise in a future version. Use to_numeric without passing
errorsand catch exceptions explicitly insteadself.dataframes[key] = df.apply(pd.to_numeric, errors='ignore')
/opt/conda/envs/relion-5.0/lib/python3.10/site-packages/starfile/parser.py:177: FutureWarning: The 'delim_whitespace' keyword in pd.read_csv is deprecated and will be removed in a future version. Use
sep='\s+'insteaddf = pd.read_csv(
/opt/conda/envs/relion-5.0/lib/python3.10/site-packages/starfile/parser.py:194: FutureWarning: errors='ignore' is deprecated and will raise in a future version. Use to_numeric without passing
errorsand catch exceptions explicitly insteadself.dataframes[key] = df.apply(pd.to_numeric, errors='ignore')
/opt/conda/envs/relion-5.0/lib/python3.10/site-packages/starfile/parser.py:177: FutureWarning: The 'delim_whitespace' keyword in pd.read_csv is deprecated and will be removed in a future version. Use
sep='\s+'insteaddf = pd.read_csv(
/opt/conda/envs/relion-5.0/lib/python3.10/site-packages/starfile/parser.py:194: FutureWarning: errors='ignore' is deprecated and will raise in a future version. Use to_numeric without passing
errorsand catch exceptions explicitly insteadself.dataframes[key] = df.apply(pd.to_numeric, errors='ignore')
/opt/conda/envs/relion-5.0/lib/python3.10/site-packages/starfile/parser.py:177: FutureWarning: The 'delim_whitespace' keyword in pd.read_csv is deprecated and will be removed in a future version. Use
sep='\s+'insteaddf = pd.read_csv(
/opt/conda/envs/relion-5.0/lib/python3.10/site-packages/starfile/parser.py:194: FutureWarning: errors='ignore' is deprecated and will raise in a future version. Use to_numeric without passing
errorsand catch exceptions explicitly insteadself.dataframes[key] = df.apply(pd.to_numeric, errors='ignore')
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /opt/conda/envs/relion-5.0/lib/python3.10/site-packages/pandas/core/indexes/base.py:3802 in │
│ get_loc │
│ │
│ 3799 │ │ """ │
│ 3800 │ │ casted_key = self._maybe_cast_indexer(key) │
│ 3801 │ │ try: │
│ ❱ 3802 │ │ │ return self._engine.get_loc(casted_key) │
│ 3803 │ │ except KeyError as err: │
│ 3804 │ │ │ if isinstance(casted_key, slice) or ( │
│ 3805 │ │ │ │ isinstance(casted_key, abc.Iterable) │
│ │
│ ╭──────────────────────────────── locals ────────────────────────────────╮ │
│ │ casted_key = '00255' │ │
│ │ key = '00255' │ │
│ │ self = Index([255, 256, 257], dtype='int64', name='rlnTomoName') │ │
│ ╰────────────────────────────────────────────────────────────────────────╯ │
│ │
│ in pandas._libs.index.IndexEngine.get_loc:153 │
│ │
│ in pandas._libs.index.IndexEngine.get_loc:161 │
│ │
│ in pandas._libs.index.Int64Engine._check_type:70 │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
KeyError: '00255'
The above exception was the direct cause of the following exception:
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ in fiducials_cli:2 │
│ ╭───────────────────────────────────── locals ─────────────────────────────────────╮ │
│ │ nominal_fiducial_diameter_nanometers = 10.0 │ │
│ │ output_directory = PosixPath('AlignTiltSeries') │ │
│ │ pipeline_control = PosixPath('AlignTiltSeries') │ │
│ │ tilt_series_star_file = PosixPath('CtfFind/tilt_series_ctf.star') │ │
│ │ tomogram_name = None │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /opt/conda/envs/relion-5.0/lib/python3.10/site-packages/tomography_python_programs/utils/relion │
│ .py:75 in pipeline_job │
│ │
│ 72 │ │ │ job_directory.mkdir(parents=True, exist_ok=True) │
│ 73 │ │ try: │
│ 74 │ │ │ pipeline_directory = kwargs.pop(PIPELINE_CONTROL_KEYWORD_ARGUMENT.name) │
│ ❱ 75 │ │ │ func(*args, **kwargs) │
│ 76 │ │ │ if job_directory is not None and pipeline_directory is not None: │
│ 77 │ │ │ │ write_job_success_file(job_directory) │
│ 78 │ │ except BaseException: │
│ │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │ args = () │ │
│ │ func = <function fiducials_cli at 0x7f0082679ea0> │ │
│ │ job_directory = PosixPath('AlignTiltSeries') │ │
│ │ kwargs = { │ │
│ │ │ 'tilt_series_star_file': PosixPath('CtfFind/tilt_series_ctf.star'), │ │
│ │ │ 'output_directory': PosixPath('AlignTiltSeries'), │ │
│ │ │ 'nominal_fiducial_diameter_nanometers': 10.0, │ │
│ │ │ 'tomogram_name': None │ │
│ │ } │ │
│ │ pipeline_directory = PosixPath('AlignTiltSeries') │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /opt/conda/envs/relion-5.0/lib/python3.10/site-packages/tomography_python_programs/align_tilt_se │
│ ries/imod/cli_fiducials.py:33 in fiducials_cli │
│ │
│ 30 │ tilt_series_set = RlnTiltSeriesSet.from_star_file( │
│ 31 │ │ filename=tilt_series_star_file, tilt_series_id=tomogram_name │
│ 32 │ ) │
│ ❱ 33 │ for global_data, tilt_series in tilt_series_set: │
│ 34 │ │ console.log(f'Aligning {tilt_series.name}...') │
│ 35 │ │ align_single_tilt_series( │
│ 36 │ │ │ tilt_series=tilt_series, │
│ │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │ nominal_fiducial_diameter_nanometers = 10.0 │ │
│ │ output_directory = PosixPath('AlignTiltSeries') │ │
│ │ tilt_series_set = RlnTiltSeriesSet( │ │
│ │ │ global_data= rlnTomoName rlnVoltage │ │
│ │ rlnSphericalAberration rlnAmplitudeContrast │ │
│ │ rlnMicrographOriginalPixelSize rlnTomoHand │ │
│ │ rlnTomoTiltSeriesPixelSize │ │
│ │ rlnTomoName │ │
│ │ 255 255 300.0 │ │
│ │ 2.7 0.07 │ │
│ │ 1.7005 1.0 1.7005 │ │
│ │ 256 256 300.0 │ │
│ │ 2.7 0.07 │ │
│ │ 1.7005 1.0 1.7005 │ │
│ │ 257 257 300.0 │ │
│ │ 2.7 0.07 │ │
│ │ 1.7005 1.0 1.7005, │ │
│ │ │ tilt_series=[ │ │
│ │ │ │ RlnTiltSeries( │ │
│ │ │ │ │ name='00255', │ │
│ │ │ │ │ data= rlnMicrographMovieName │ │
│ │ rlnTomoTiltMovieFrameCount │ │
│ │ rlnTomoNominalStageTiltAngle │ │
│ │ rlnTomoNominalTiltAxisAngle ... rlnCtfAstigmatism │ │
│ │ rlnDefocusAngle rlnCtfFigureOfMerit │ │
│ │ rlnCtfMaxResolution │ │
│ │ 0 data/data/TS_02_001-0.0.tif │ │
│ │ 12 -0.01135 │ │
│ │ 85.7 ... 275.392578 88.783653 │ │
│ │ 0.045798 8.291962 │ │
[...]
│ │ 40 data/data/TS_03_041_60.0.tif │ │
│ │ 12 59.98280 │ │
│ │ 85.7 ... 457.650391 -81.486980 │ │
│ │ 0.016579 14.756881 │ │
│ │ │ │
│ │ [41 rows x 19 columns] │ │
│ │ │ │ ), │ │
│ │ │ │ RlnTiltSeries( │ │
│ │ │ │ │ name='00257', │ │
│ │ │ │ │ data= rlnMicrographMovieName │ │
│ │ rlnTomoTiltMovieFrameCount │ │
│ │ rlnTomoNominalStageTiltAngle │ │
│ │ rlnTomoNominalTiltAxisAngle ... rlnCtfAstigmatism │ │
│ │ rlnDefocusAngle rlnCtfFigureOfMerit │ │
│ │ rlnCtfMaxResolution │ │
│ │ 0 data/data/TS_04_001_-0.0.tif │ │
│ │ 12 -0.01035 │ │
│ │ 85.7 ... 232.371094 -7.723870 │ │
│ │ 0.056624 7.195504 │ │
[...]
│ │ 40 data/data/TS_04_041_60.0.tif │ │
│ │ 12 59.98680 │ │
│ │ 85.7 ... 488.773438 -72.590830 │ │
│ │ 0.013987 9.262298 │ │
│ │ │ │
│ │ [41 rows x 19 columns] │ │
│ │ │ │ ) │ │
│ │ │ ] │ │
│ │ ) │ │
│ │ tilt_series_star_file = PosixPath('CtfFind/tilt_series_ctf.star') │ │
│ │ tomogram_name = None │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /opt/conda/envs/relion-5.0/lib/python3.10/site-packages/tomography_python_programs/_metadata_mod │
│ els/relion/tilt_series_set.py:95 in │
│ │
│ 92 │ │ return self.tilt_series[self.tilt_series_names.index(tilt_series_id)] │
│ 93 │ │
│ 94 │ def iter(self) -> Iterable[Tuple[pd.Series, RlnTiltSeries]]: │
│ ❱ 95 │ │ return ((self.global_data.loc[tilt_series.name], tilt_series) for │
│ 96 │ │ │ │ tilt_series in self.tilt_series) │
│ 97 │
│ │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │ .0 = <list_iterator object at 0x7f0082935540> │ │
│ │ self = RlnTiltSeriesSet( │ │
│ │ │ global_data= rlnTomoName rlnVoltage rlnSphericalAberration │ │
│ │ rlnAmplitudeContrast rlnMicrographOriginalPixelSize rlnTomoHand │ │
│ │ rlnTomoTiltSeriesPixelSize │ │
│ │ rlnTomoName │ │
│ │ 255 255 300.0 2.7 │ │
│ │ 0.07 1.7005 1.0 1.7005 │ │
│ │ 256 256 300.0 2.7 │ │
│ │ 0.07 1.7005 1.0 1.7005 │ │
│ │ 257 257 300.0 2.7 │ │
│ │ 0.07 1.7005 1.0 1.7005, │ │
│ │ │ tilt_series=[ │ │
│ │ │ │ RlnTiltSeries( │ │
│ │ │ │ │ name='00255', │ │
│ │ │ │ │ data= rlnMicrographMovieName rlnTomoTiltMovieFrameCount │ │
│ │ rlnTomoNominalStageTiltAngle rlnTomoNominalTiltAxisAngle ... │ │
│ │ rlnCtfAstigmatism rlnDefocusAngle rlnCtfFigureOfMerit rlnCtfMaxResolution │ │
│ │ 0 data/data/TS_02_001-0.0.tif 12 │ │
│ │ -0.01135 85.7 ... 275.392578 88.783653 │ │
[..]
│ │ -42.784290 0.019736 25.000000 │ │
│ │ 40 data/data/TS_02_041_60.0.tif 12 │ │
│ │ 59.98430 85.7 ... 569.316406 -58.727370 │ │
│ │ 0.013299 25.000000 │ │
│ │ │ │
│ │ [41 rows x 19 columns] │ │
│ │ │ │ ), │ │
│ │ │ │ RlnTiltSeries( │ │
│ │ │ │ │ name='00256', │ │
│ │ │ │ │ data= rlnMicrographMovieName rlnTomoTiltMovieFrameCount │ │
│ │ rlnTomoNominalStageTiltAngle rlnTomoNominalTiltAxisAngle ... │ │
│ │ rlnCtfAstigmatism rlnDefocusAngle rlnCtfFigureOfMerit rlnCtfMaxResolution │ │
│ │ 0 data/data/TS_03_001_-0.0.tif 12 │ │
│ │ -0.01535 85.7 ... 470.345703 -62.573570 │ │
│ │ 0.043402 5.213509 │ │
[..]
│ │ 40 data/data/TS_04_041_60.0.tif 12 │ │
│ │ 59.98680 85.7 ... 488.773438 -72.590830 │ │
│ │ 0.013987 9.262298 │ │
│ │ │ │
│ │ [41 rows x 19 columns] │ │
│ │ │ │ ) │ │
│ │ │ ] │ │
│ │ ) │ │
│ │ tilt_series = RlnTiltSeries( │ │
│ │ │ name='00255', │ │
│ │ │ data= rlnMicrographMovieName rlnTomoTiltMovieFrameCount │ │
│ │ rlnTomoNominalStageTiltAngle rlnTomoNominalTiltAxisAngle ... │ │
│ │ rlnCtfAstigmatism rlnDefocusAngle rlnCtfFigureOfMerit rlnCtfMaxResolution │ │
│ │ 0 data/data/TS_02_001_-0.0.tif 12 │ │
│ │ -0.01135 85.7 ... 275.392578 88.783653 │ │
│ │ 0.045798 8.291962 │ │
│ │ 1 data/data/TS_02_002_3.0.tif 12 │ │
│ │ 2.98478 85.7 ... 520.529297 -52.787060 │ │
│ │ 0.035346 6.263712 │ │
[...]
│ │ 40 data/data/TS_02_041_60.0.tif 12 │ │
│ │ 59.98430 85.7 ... 569.316406 -58.727370 │ │
│ │ 0.013299 25.000000 │ │
│ │ │ │
│ │ [41 rows x 19 columns] │ │
│ │ ) │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /opt/conda/envs/relion-5.0/lib/python3.10/site-packages/pandas/core/indexing.py:1192 in │
│ getitem │
│ │
│ 1189 │ │ │ │
│ 1190 │ │ │ maybe_callable = com.apply_if_callable(key, self.obj) │
│ 1191 │ │ │ maybe_callable = self._check_deprecated_callable_usage(key, maybe_callable) │
│ ❱ 1192 │ │ │ return self._getitem_axis(maybe_callable, axis=axis) │
│ 1193 │ │
│ 1194 │ def _is_scalar_access(self, key: tuple): │
│ 1195 │ │ raise NotImplementedError() │
│ │
│ ╭─────────────────────────────────── locals ───────────────────────────────────╮ │
│ │ axis = 0 │ │
│ │ key = '00255' │ │
│ │ maybe_callable = '00255' │ │
│ │ self = <pandas.core.indexing._LocIndexer object at 0x7f00826868e0> │ │
│ ╰──────────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /opt/conda/envs/relion-5.0/lib/python3.10/site-packages/pandas/core/indexing.py:1432 in │
│ _getitem_axis │
│ │
│ 1429 │ │ │
│ 1430 │ │ # fall thru to straight lookup │
│ 1431 │ │ self._validate_key(key, axis) │
│ ❱ 1432 │ │ return self._get_label(key, axis=axis) │
│ 1433 │ │
│ 1434 │ def _get_slice_axis(self, slice_obj: slice, axis: AxisInt): │
│ 1435 │ │ """ │
│ │
│ ╭─────────────────────────────── locals ───────────────────────────────╮ │
│ │ axis = 0 │ │
│ │ key = '00255' │ │
│ │ labels = Index([255, 256, 257], dtype='int64', name='rlnTomoName') │ │
│ │ self = <pandas.core.indexing._LocIndexer object at 0x7f00826868e0> │ │
│ ╰──────────────────────────────────────────────────────────────────────╯ │
│ │
│ /opt/conda/envs/relion-5.0/lib/python3.10/site-packages/pandas/core/indexing.py:1382 in │
│ _get_label │
│ │
│ 1379 │ │
│ 1380 │ def _get_label(self, label, axis: AxisInt): │
│ 1381 │ │ # GH#5567 this will fail if the label is not present in the axis. │
│ ❱ 1382 │ │ return self.obj.xs(label, axis=axis) │
│ 1383 │ │
│ 1384 │ def _handle_lowerdim_multi_index_axis0(self, tup: tuple): │
│ 1385 │ │ # we have an axis0 multi-index, handle or raise │
│ │
│ ╭────────────────────────────── locals ───────────────────────────────╮ │
│ │ axis = 0 │ │
│ │ label = '00255' │ │
│ │ self = <pandas.core.indexing.LocIndexer object at 0x7f00826868e0> │ │
│ ╰─────────────────────────────────────────────────────────────────────╯ │
│ │
│ /opt/conda/envs/relion-5.0/lib/python3.10/site-packages/pandas/core/generic.py:4295 in xs │
│ │
│ 4292 │ │ │ │ else: │
│ 4293 │ │ │ │ │ new_index = index[loc] │
│ 4294 │ │ else: │
│ ❱ 4295 │ │ │ loc = index.get_loc(key) │
│ 4296 │ │ │ │
│ 4297 │ │ │ if isinstance(loc, np.ndarray): │
│ 4298 │ │ │ │ if loc.dtype == np.bool: │
│ │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │ axis = 0 │ │
│ │ drop_level = True │ │
│ │ index = Index([255, 256, 257], dtype='int64', name='rlnTomoName') │ │
│ │ key = '00255' │ │
│ │ labels = Index([255, 256, 257], dtype='int64', name='rlnTomoName') │ │
│ │ level = None │ │
│ │ self = │ │ │ rlnTomoName rlnVoltage rlnSphericalAberration │ │
│ │ rlnAmplitudeContrast rlnMicrographOriginalPixelSize rlnTomoHand │ │
│ │ rlnTomoTiltSeriesPixelSize │ │
│ │ rlnTomoName │ │
│ │ 255 255 300.0 2.7 │ │
│ │ 0.07 1.7005 1.0 1.7005 │ │
│ │ 256 256 300.0 2.7 │ │
│ │ 0.07 1.7005 1.0 1.7005 │ │
│ │ 257 257 300.0 2.7 │ │
│ │ 0.07 1.7005 1.0 1.7005 │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /opt/conda/envs/relion-5.0/lib/python3.10/site-packages/pandas/core/indexes/base.py:3809 in │
│ get_loc │
│ │
│ 3806 │ │ │ │ and any(isinstance(x, slice) for x in casted_key) │
│ 3807 │ │ │ ): │
│ 3808 │ │ │ │ raise InvalidIndexError(key) │
│ ❱ 3809 │ │ │ raise KeyError(key) from err │
│ 3810 │ │ except TypeError: │
│ 3811 │ │ │ # If we have a listlike key, _check_indexing_error will raise │
│ 3812 │ │ │ # InvalidIndexError. Otherwise we fall through and re-raise │
│ │
│ ╭──────────────────────────────── locals ────────────────────────────────╮ │
│ │ casted_key = '00255' │ │
│ │ key = '00255' │ │
│ │ self = Index([255, 256, 257], dtype='int64', name='rlnTomoName') │ │
│ ╰────────────────────────────────────────────────────────────────────────╯ │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
KeyError: '00255'
---------------------------------- PYTHON ERROR ---------------------------------
Has RELION been provided a Python interpreter with the correct environment?
The interpreter can be passed to RELION either during Cmake configuration by
using the Cmake flag -DPYTHON_EXE_PATH=<path/to/python/interpreter>.
Using python executable: /opt/conda/envs/relion-5.0/bin/python
The text was updated successfully, but these errors were encountered: