[FIX] Data formats clarification #1720

nbeliy · 2024-03-05T13:31:28Z

closes Clarify data formats for imaging data (MRI, microscopy, ...), remove MRI-centric language from common principles #1713

Changes:

In common principles/image format, removed reference to Nifti, and removed Nifti specific content. Rephrased the rest.

In MRI, added Data format section with Nifti requirements.

To do:

To add your name, please edit our Contributors wiki and add your name with the type of contribution.

nbeliy · 2024-03-05T13:35:07Z

@bids-maintenance

Added explicit mentioning of:

incompatibility of ANALYZE 7.5 files with BIDS
Concatenation of multi-volume acquisitions (e.g. for 4D fmri)

I'm not sure that this is actual requirements from BIDS, can you confirm or dis-confirm?

codecov · 2024-03-05T13:35:17Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 87.93%. Comparing base (05f64ed) to head (7f15ae8).
Report is 70 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #1720   +/-   ##
=======================================
  Coverage   87.93%   87.93%           
=======================================
  Files          16       16           
  Lines        1351     1351           
=======================================
  Hits         1188     1188           
  Misses        163      163

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Remi-Gau · 2024-03-06T16:37:42Z

src/common-principles.md

@@ -409,20 +409,22 @@ datasets and non-compliant derivatives.

 ### Imaging files


Wondering if this heading should not be renamed now that this section has become more generic.

I used Imaging files as generic file, including for example EEG. I can't find a good name, data file will also incompass the tabular files. But I'm open for suggestions.

yeah I was more making a note also for other reviewers

src/common-principles.md

src/modality-specific-files/magnetic-resonance-imaging-data.md

Remi-Gau · 2024-03-06T16:44:36Z

src/modality-specific-files/magnetic-resonance-imaging-data.md

+of Nibabel package.
+
+Due to the important size of MRI imaging data, we RECOMMEND using compressed NIfTI files
+by [gzip](https://www.gzip.org/) algorithm (.nii.gz).


Suggested change

by [gzip](https://www.gzip.org/) algorithm (.nii.gz).

by [gzip](https://www.gzip.org/) algorithm.

Here I wanted to stress that compressed files should conserve their original extension and have .gz additional extension (in case if gzip can support several ones). But if it obvious, I agree with changes.

Good point.

Pinging @effigies and @sappelhoff to get second opinions.

Can briefly talk about this at the maintainers meeting tomorrow.

🤷 I think it's fine to mention .nii.gz in brackets here.

Remi-Gau

If you have the energy, implementing our soft rules would be nice:

mostly starting every new sentence on a new line

src/modality-specific-files/magnetic-resonance-imaging-data.md

nbeliy · 2024-03-08T14:25:12Z

@Remi-Gau , did you got news from maintainers meeting? Should I go with PET format too?

nbeliy · 2024-03-11T16:28:16Z

Added section for PET. Essentially a copy-paste of MRI, with a minimum-recommended version of dcm2niix, and additional sub-sections to better fit into structure of page.

I'll put specific remarks into a review of code.

nbeliy · 2024-03-11T16:29:31Z

src/modality-specific-files/positron-emission-tomography.md

+
+All PET imaging data MUST be stored using the
+[NIfTI-1 or NIFTI-2](https://nifti.nimh.nih.gov/) file format (`.nii`).
+The ANALYZE-7.5 file format (`.hdr`/`.img`), despite being very similar to NIFTI-1,


Not sure if it's still needed. Is .hdr/.img popular in PET?

No idea but I am starting to think we should NOT mention things that are NOT allowed, because it could get very long.

@mnoergaard might know :-)

For MRI is needed -- just saying NIFTI includes the .hdr/.img. And I know at least one attempt to bidsify such format.

src/modality-specific-files/positron-emission-tomography.md

.all-contributorsrc

nbeliy · 2024-03-11T16:33:22Z

src/modality-specific-files/positron-emission-tomography.md

 In addition to the imaging data (`*.nii`) a `_pet.json` sidecar file MUST be provided.
 The included metadata are divided into sections described below.
+-->


Commented this paragraph, as it appears in incorrect section (JSON must be present no matter if it's pure PET of PET-MRI dataset). Also, the same information appears directly below.

Remi-Gau · 2024-04-11T13:05:49Z

src/common-principles.md

+We RECOMMEND that the imaging file will be accompanied with a additional meta
+information extracted from source image and/or external sources in a sidecar
+JSON file.


For certain file the sidecar is REQUIRED so I would actually remove this sentence.

I would agree if JSON sidecars was properly introduced in common principles. It is mentioned in several places, but without a proper definition. This paragraph tried to provide such definition.

src/common-principles.md

Remi-Gau · 2024-04-11T13:10:31Z

src/modality-specific-files/positron-emission-tomography.md

+Dynamic (multi-volume) PET imaging data SHOULD be stored in 4D,
+in chronological order (the order they were acquired in).
+
+Due to the important size of PET imaging data, we RECOMMEND using compressed


we do not actually officialy recommend (meaning that the validator would throw a warning if people use .nii and not .nii.gz)

Maybe "we suggest" or "we enourage".

Recommendation was carried from common principles:

We RECOMMEND using compressed NIfTI files

Remi-Gau · 2024-04-11T13:11:16Z

src/modality-specific-files/magnetic-resonance-imaging-data.md

+volumes of functional MRI) MUST be concatenated into single image (3D or 4D,
+respectively).
+
+Due to the important size of MRI imaging data, we RECOMMEND using compressed


See comment about PET and recommending zipped data.

src/modality-specific-files/magnetic-resonance-imaging-data.md

nbeliy · 2024-05-14T08:43:56Z

Hi @Remi-Gau , just to know is there any developments on this merge request?

Frst draft: common principles and MRI

3c83f3e

fixing spelling

d2b33fe

nbeliy mentioned this pull request Mar 5, 2024

Clarify data formats for imaging data (MRI, microscopy, ...), remove MRI-centric language from common principles #1713

Open

Remi-Gau reviewed Mar 6, 2024

View reviewed changes

src/common-principles.md Outdated Show resolved Hide resolved

Remi-Gau reviewed Mar 6, 2024

View reviewed changes

src/common-principles.md Outdated Show resolved Hide resolved

Remi-Gau reviewed Mar 6, 2024

View reviewed changes

src/modality-specific-files/magnetic-resonance-imaging-data.md Outdated Show resolved Hide resolved

Remi-Gau reviewed Mar 6, 2024

View reviewed changes

src/modality-specific-files/magnetic-resonance-imaging-data.md Show resolved Hide resolved

Remi-Gau marked this pull request as ready for review March 6, 2024 17:57

Remi-Gau requested a review from DimitriPapadopoulos as a code owner March 6, 2024 17:57

Remi-Gau marked this pull request as draft March 6, 2024 17:57

Remi-Gau changed the title ~~[Fix] Data formats clarification (#1713)~~ [FIX] Data formats clarification Mar 7, 2024

Beliy Nikita added 3 commits March 8, 2024 14:45

Remi's suggestions

f8be29d

Implemented soft rules

f261fb3

added contribution

d5d2928

Beliy Nikita added 2 commits March 11, 2024 17:17

PET format draft

8943f54

Added sub-sections

f655623

nbeliy commented Mar 11, 2024

View reviewed changes

src/modality-specific-files/positron-emission-tomography.md Outdated Show resolved Hide resolved

Remi-Gau reviewed Mar 11, 2024

View reviewed changes

.all-contributorsrc Outdated Show resolved Hide resolved

nbeliy commented Mar 11, 2024

View reviewed changes

Beliy Nikita added 2 commits March 11, 2024 18:00

Removed from contributers(

838ed8e

Removed suggestions for data conversion in MRI and PET

55acae0

Remi-Gau reviewed Apr 11, 2024

View reviewed changes

src/common-principles.md Outdated Show resolved Hide resolved

Remi-Gau reviewed Apr 11, 2024

View reviewed changes

src/modality-specific-files/magnetic-resonance-imaging-data.md Show resolved Hide resolved

Removed compression requirement from common principle

7f15ae8

Remi-Gau marked this pull request as ready for review April 18, 2024 16:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FIX] Data formats clarification #1720

[FIX] Data formats clarification #1720

nbeliy commented Mar 5, 2024 •

edited by Remi-Gau

nbeliy commented Mar 5, 2024

codecov bot commented Mar 5, 2024 •

edited

Remi-Gau Mar 6, 2024

nbeliy Mar 6, 2024

Remi-Gau Mar 6, 2024

Remi-Gau Mar 6, 2024

nbeliy Mar 6, 2024

Remi-Gau Mar 6, 2024

sappelhoff Mar 11, 2024

Remi-Gau left a comment

nbeliy commented Mar 8, 2024

nbeliy commented Mar 11, 2024

nbeliy Mar 11, 2024

Remi-Gau Mar 11, 2024 •

edited

sappelhoff Mar 11, 2024

nbeliy Mar 11, 2024

nbeliy Mar 11, 2024

Remi-Gau Apr 11, 2024

nbeliy Apr 11, 2024

Remi-Gau Apr 11, 2024

nbeliy Apr 11, 2024

Remi-Gau Apr 11, 2024

nbeliy commented May 14, 2024

		@@ -409,20 +409,22 @@ datasets and non-compliant derivatives.

		### Imaging files

	by [gzip](https://www.gzip.org/) algorithm (.nii.gz).
	by [gzip](https://www.gzip.org/) algorithm.

[FIX] Data formats clarification #1720

Are you sure you want to change the base?

[FIX] Data formats clarification #1720

Conversation

nbeliy commented Mar 5, 2024 • edited by Remi-Gau

nbeliy commented Mar 5, 2024

codecov bot commented Mar 5, 2024 • edited

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Remi-Gau left a comment

Choose a reason for hiding this comment

nbeliy commented Mar 8, 2024

nbeliy commented Mar 11, 2024

Choose a reason for hiding this comment

Remi-Gau Mar 11, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nbeliy commented May 14, 2024

nbeliy commented Mar 5, 2024 •

edited by Remi-Gau

codecov bot commented Mar 5, 2024 •

edited

Remi-Gau Mar 11, 2024 •

edited