fmap issue #514

CPernet · 2023-01-30T09:58:03Z

under demos/fmap_failexemple 3 subjects for which all files do not populate -- have figured why
-- note my bad, this includes the 'old' suggested change about the trial type on the face rep data set

[FIX & ENH] filter transformers apply to all inputs & basic transformer can act on specific rows

[ENH] Improve reports

…l to 1

[FIX & ENH] coerce several elements of the bids stats models

…sts from lower levels to a higher level - fixes cpp-lln-lab/bidspm#616

…st whole layout

[ENH] improve bids.Description constructor

[FIX] fix printing of windows path

…ties

[ENH] Filter layout

for more information, see https://pre-commit.ci

Remi-Gau · 2023-01-30T10:11:55Z

I only see the participants.tsv.
Were by any chance the truncated files git ignored?

Remi-Gau · 2023-01-30T10:14:24Z

You may have to use git add --force given this gitignore

https://github.com/bids-standard/bids-matlab/blob/dev/demos/.gitignore

for more information, see https://pre-commit.ci

CPernet · 2023-01-30T10:49:44Z

OMG - yes, forced add and push, it's finally there

Remi-Gau · 2023-01-30T15:57:04Z

Will try to documment the debugging in the PR.

dir 'entity' is unknown for several fmaps

>> BIDS = bids.layout(pwd, 'verbose', true, 'use_schema', true)


Indexing dataset:
	/home/remi/github/bids/matlab/demos/fmap_failexemple
 Indexing subject: sub-55809 [.]
 Indexing subject: sub-57460 [.Warning:
Skipping file: sub-57460_ses-1wkfollowup_acq-gre_dir-AP_run-1_magnitude1.nii.gz.
 Unknown entities: 'dir'
Try using the parameter: " 'use_schema', false "

 
> In bids.internal.error_handling (line 57)
  In bids.internal.append_to_layout (line 112)
  In bids.layout>parse_using_schema (line 410)
  In bids.layout>parse_subject (line 351)
  In bids.layout (line 188)

Remi-Gau · 2023-01-30T15:58:06Z

According to the spec only pepolar fmap support the dir entity

https://bids-specification.readthedocs.io/en/latest/modality-specific-files/magnetic-resonance-imaging-data.html#types-of-fieldmaps

https://bids-specification.readthedocs.io/en/latest/modality-specific-files/magnetic-resonance-imaging-data.html#case-4-multiple-phase-encoded-directions-pepolar

Remi-Gau · 2023-01-30T16:01:17Z

Quick work around for now is to ask bids-matlab to index without relying on the schema.

>> BIDS = bids.layout(pwd, 'verbose', true, 'use_schema', false)


Indexing dataset:
	/home/remi/github/bids/matlab/demos/fmap_failexemple
 Indexing subject: sub-55809 [.]
 Indexing subject: sub-57460 [..]
 Indexing subject: sub-57507 [.]

BIDS = 

  struct with fields:

              pth: '/home/remi/github/bids/matlab/demos/fmap_failexemple'
      description: [1×1 struct]
         sessions: {}
     participants: [1×1 struct]
         subjects: [1×4 struct]
             root: [0×0 struct]
    is_datalad_ds: 0



>> bids.query(BIDS, 'data')

ans =

  18×1 cell array

    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-55809/ses-Baseline/fmap/sub-55809_ses-Baseline_acq-se_dir-AP_run-1_epi.nii.gz'              }
    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-55809/ses-Baseline/fmap/sub-55809_ses-Baseline_acq-se_dir-PA_run-1_epi.nii.gz'              }
    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-57460/ses-1wkfollowup/fmap/sub-57460_ses-1wkfollowup_acq-gre_dir-AP_run-1_magnitude1.nii.gz'}
    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-57460/ses-1wkfollowup/fmap/sub-57460_ses-1wkfollowup_acq-gre_dir-AP_run-1_magnitude2.nii.gz'}
    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-57460/ses-1wkfollowup/fmap/sub-57460_ses-1wkfollowup_acq-gre_dir-AP_run-1_phasediff.nii.gz' }
    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-57460/ses-1wkfollowup/fmap/sub-57460_ses-1wkfollowup_acq-gre_dir-LR_run-1_magnitude1.nii.gz'}
    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-57460/ses-1wkfollowup/fmap/sub-57460_ses-1wkfollowup_acq-gre_dir-LR_run-1_magnitude2.nii.gz'}
    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-57460/ses-1wkfollowup/fmap/sub-57460_ses-1wkfollowup_acq-gre_dir-LR_run-1_phasediff.nii.gz' }
    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-57460/ses-Baseline/fmap/sub-57460_ses-Baseline_acq-se_dir-AP_run-1_epi.nii.gz'              }
    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-57460/ses-Baseline/fmap/sub-57460_ses-Baseline_acq-se_dir-AP_run-2_epi.nii.gz'              }
    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-57460/ses-Baseline/fmap/sub-57460_ses-Baseline_acq-se_dir-PA_run-1_epi.nii.gz'              }
    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-57460/ses-Baseline/fmap/sub-57460_ses-Baseline_acq-se_dir-PA_run-2_epi.nii.gz'              }
    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-57507/ses-Baseline/fmap/sub-57507_ses-Baseline_acq-gre_dir-AP_run-1_magnitude1.nii.gz'      }
    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-57507/ses-Baseline/fmap/sub-57507_ses-Baseline_acq-gre_dir-AP_run-1_magnitude2.nii.gz'      }
    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-57507/ses-Baseline/fmap/sub-57507_ses-Baseline_acq-gre_dir-AP_run-1_phasediff.nii.gz'       }
    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-57507/ses-Baseline/fmap/sub-57507_ses-Baseline_acq-gre_dir-LR_run-1_magnitude1.nii.gz'      }
    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-57507/ses-Baseline/fmap/sub-57507_ses-Baseline_acq-gre_dir-LR_run-1_magnitude2.nii.gz'      }
    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-57507/ses-Baseline/fmap/sub-57507_ses-Baseline_acq-gre_dir-LR_run-1_phasediff.nii.gz'       }

Remi-Gau · 2023-01-30T16:05:38Z

other queries seem to work fine after that.

@CPernet can you cross check that if you do not use the schema, for indexing you can query the files you need?

>> BIDS = bids.layout(pwd, 'verbose', true, 'use_schema', false)


Indexing dataset:
	/home/remi/github/bids/matlab/demos/fmap_failexemple
 Indexing subject: sub-55809 [.]
 Indexing subject: sub-57460 [..]
 Indexing subject: sub-57507 [.]

BIDS = 

  struct with fields:

              pth: '/home/remi/github/bids/matlab/demos/fmap_failexemple'
      description: [1×1 struct]
         sessions: {}
     participants: [1×1 struct]
         subjects: [1×4 struct]
             root: [0×0 struct]
    is_datalad_ds: 0

>> bids.query(BIDS, 'data')

ans =

  18×1 cell array

    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-55809/ses-Baseline/fmap/sub-55809_ses-Baseline_acq-se_dir-AP_run-1_epi.nii.gz'              }
    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-55809/ses-Baseline/fmap/sub-55809_ses-Baseline_acq-se_dir-PA_run-1_epi.nii.gz'              }
    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-57460/ses-1wkfollowup/fmap/sub-57460_ses-1wkfollowup_acq-gre_dir-AP_run-1_magnitude1.nii.gz'}
    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-57460/ses-1wkfollowup/fmap/sub-57460_ses-1wkfollowup_acq-gre_dir-AP_run-1_magnitude2.nii.gz'}
    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-57460/ses-1wkfollowup/fmap/sub-57460_ses-1wkfollowup_acq-gre_dir-AP_run-1_phasediff.nii.gz' }
    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-57460/ses-1wkfollowup/fmap/sub-57460_ses-1wkfollowup_acq-gre_dir-LR_run-1_magnitude1.nii.gz'}
    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-57460/ses-1wkfollowup/fmap/sub-57460_ses-1wkfollowup_acq-gre_dir-LR_run-1_magnitude2.nii.gz'}
    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-57460/ses-1wkfollowup/fmap/sub-57460_ses-1wkfollowup_acq-gre_dir-LR_run-1_phasediff.nii.gz' }
    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-57460/ses-Baseline/fmap/sub-57460_ses-Baseline_acq-se_dir-AP_run-1_epi.nii.gz'              }
    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-57460/ses-Baseline/fmap/sub-57460_ses-Baseline_acq-se_dir-AP_run-2_epi.nii.gz'              }
    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-57460/ses-Baseline/fmap/sub-57460_ses-Baseline_acq-se_dir-PA_run-1_epi.nii.gz'              }
    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-57460/ses-Baseline/fmap/sub-57460_ses-Baseline_acq-se_dir-PA_run-2_epi.nii.gz'              }
    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-57507/ses-Baseline/fmap/sub-57507_ses-Baseline_acq-gre_dir-AP_run-1_magnitude1.nii.gz'      }
    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-57507/ses-Baseline/fmap/sub-57507_ses-Baseline_acq-gre_dir-AP_run-1_magnitude2.nii.gz'      }
    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-57507/ses-Baseline/fmap/sub-57507_ses-Baseline_acq-gre_dir-AP_run-1_phasediff.nii.gz'       }
    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-57507/ses-Baseline/fmap/sub-57507_ses-Baseline_acq-gre_dir-LR_run-1_magnitude1.nii.gz'      }
    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-57507/ses-Baseline/fmap/sub-57507_ses-Baseline_acq-gre_dir-LR_run-1_magnitude2.nii.gz'      }
    {'/home/remi/github/bids/matlab/demos/fmap_failexemple/sub-57507/ses-Baseline/fmap/sub-57507_ses-Baseline_acq-gre_dir-LR_run-1_phasediff.nii.gz'       }

Remi-Gau · 2023-01-30T16:06:26Z

If you want to stick to the schema, then this would require updating the spec by adding the dir entity for all fieldmaps.

CPernet · 2023-01-30T17:02:15Z

might relate to bids-standard/bids-specification#1391

CPernet · 2023-01-30T17:44:15Z

i see that, although dir- should really be allowed any where, donæt you think

Remi-Gau · 2023-01-30T17:50:59Z

Let's say I would not be shocked to have it added to many mri datatype instead of letting acq do the heavy lifting.

Remi-Gau · 2023-01-30T17:52:11Z

Will probably switch this PR to draft until we see things more clearly on this front, it that is ok with you?

CPernet · 2023-01-30T17:52:40Z

tagged you in the bids validator issue -- the entitiy says you can, and it silly to have acq-se_dir-AP valid but acq-gre_dir-AP invalid

CPernet · 2023-01-30T17:53:00Z

yes

CPernet · 2023-05-02T14:45:38Z

not AP-PA field maps are in SPM dev (testing it) so we should add them @gllmflndn

Remi-Gau added 30 commits May 29, 2022 11:30

doc - update doc

308d0fe

add badge to readme

8afb872

change python version for RTD

6bce06c

do not fail doc build on warning

72c8096

Merge pull request bids-standard#383 from bids-standard/fix_trans

94cc7a6

[FIX & ENH] filter transformers apply to all inputs & basic transformer can act on specific rows

fix bids report to actually include metadata for functional data

5e7871e

improve reports

50db7d9

Merge pull request bids-standard#384 from bids-standard/improve_reports

c1a2779

[ENH] Improve reports

small fix in diagnostic table plotting

a7a078c

fix bug 385 by coercing Nodes and Edges to always be cells

8381cf6

get_node function return cell is numel(output) > 1 but struct if equa…

0895e91

…l to 1

coerce contrasts into cells on loading

ab04030

add get_edge method

3dd6d69

add contrast validation method

c2b9079

improve get_edge method to handle edges with same source

c0722a8

add get_source_node method

fb7fca3

ensure written models are valid

dd97f02

Merge pull request bids-standard#386 from Remi-Gau/fix_model

e07f2db

[FIX & ENH] coerce several elements of the bids stats models

fix typo

1b53a46

clear variable that would otherwise lead to directly assigning contra…

3afc04b

…sts from lower levels to a higher level - fixes cpp-lln-lab/bidspm#616

Fix matlab test CI workflow

99eac7b

make sure tests are run in CI when workflow files are changed

ca2ec87

improve Description constructor to also accept filenames and not a ju…

15d3b6b

…st whole layout

fix bugs

7294ba1

allow empty subject directories when going schema less

0c0a4a8

Merge pull request bids-standard#388 from bids-standard/desc

20b9e2a

[ENH] improve bids.Description constructor

fix bids basic transformers row selection

9f3a94f

fix printing of windows path

e8f6153

fix printing of windows path

5bdacb4

Merge pull request bids-standard#389 from Remi-Gau/dev

89da6b5

[FIX] fix printing of windows path

Remi-Gau and others added 10 commits January 22, 2023 17:12

add filter to layout to only index certain subjects, sessions, modali…

e2029f4

…ties

use filtering of layout

0855b9a

fix when no filter

9e05c94

use regexp

bbfb6c8

fix when to use regex

d67d110

Merge pull request bids-standard#510 from bids-standard/filter_layout

3dfba0c

[ENH] Filter layout

compat

43c156f

all fmap not populated

fd5c4ab

merge remote

7eea138

[pre-commit.ci] auto fixes from pre-commit.com hooks

52928f2

for more information, see https://pre-commit.ci

CPernet and others added 2 commits January 30, 2023 11:48

with files

6bc24e6

[pre-commit.ci] auto fixes from pre-commit.com hooks

e916b0a

for more information, see https://pre-commit.ci

Remi-Gau marked this pull request as draft February 2, 2023 23:09

fix events

1fb21aa

Remi-Gau changed the base branch from dev to main November 20, 2023 19:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fmap issue #514

fmap issue #514

CPernet commented Jan 30, 2023

Remi-Gau commented Jan 30, 2023

Remi-Gau commented Jan 30, 2023

CPernet commented Jan 30, 2023

Remi-Gau commented Jan 30, 2023 •

edited

Remi-Gau commented Jan 30, 2023 •

edited

Remi-Gau commented Jan 30, 2023

Remi-Gau commented Jan 30, 2023

Remi-Gau commented Jan 30, 2023

CPernet commented Jan 30, 2023

CPernet commented Jan 30, 2023

Remi-Gau commented Jan 30, 2023

Remi-Gau commented Jan 30, 2023

CPernet commented Jan 30, 2023

CPernet commented Jan 30, 2023

CPernet commented May 2, 2023

fmap issue #514

Are you sure you want to change the base?

fmap issue #514

Conversation

CPernet commented Jan 30, 2023

Remi-Gau commented Jan 30, 2023

Remi-Gau commented Jan 30, 2023

CPernet commented Jan 30, 2023

Remi-Gau commented Jan 30, 2023 • edited

dir 'entity' is unknown for several fmaps

Remi-Gau commented Jan 30, 2023 • edited

Remi-Gau commented Jan 30, 2023

Remi-Gau commented Jan 30, 2023

Remi-Gau commented Jan 30, 2023

CPernet commented Jan 30, 2023

CPernet commented Jan 30, 2023

Remi-Gau commented Jan 30, 2023

Remi-Gau commented Jan 30, 2023

CPernet commented Jan 30, 2023

CPernet commented Jan 30, 2023

CPernet commented May 2, 2023

Remi-Gau commented Jan 30, 2023 •

edited

Remi-Gau commented Jan 30, 2023 •

edited