Finalize afni_proc.py settings

[tsalo]

@handwerkerd do you have any recommendations for running afni_proc.py on multi-echo data? Is there a way to ensure that the outputs are roughly similar to BIDS format?

Here's a copy from the help (feel free to edit as you wish):

#!/bin/tcsh

# --------------------------------------------------
# note fixed top-level directories
set data_root = /main/location/of/all/data

set input_root = $data_root/scanner_data
set output_root = $data_root/subject_analysis

# --------------------------------------------------
# get a list of subjects, or just use one (consider $argv)
cd $input root
set subjects = ( subj* )
cd -

# or perhaps just process one subject?
set subjects = ( subj_017 )

# --------------------------------------------------
# process all subjects
foreach subj_id ( $subjects )

  # --------------------------------------------------
  # note input and output directories
  set subj_indir = $input_root/$subj_id
  set subj_outdir = $output_root/$subj_id

  # --------------------------------------------------
  # if output dir exists, this subject has already been processed
  if ( -d $subj_outdir ) then
     echo "** results dir already exists, skipping subject $subj_id"
     continue
  endif

  # --------------------------------------------------
  # otherwise create the output directory, write an afni_proc.py
  # command to it, and fire it up
  mkdir -p $subj_outdir
  cd $subj_outdir

  # create a run.afni_proc script in this directory
  cat > run.afni_proc << EOF

  # notes:
  #   - consider different named inputs (rather than OutBrick)
  #   - verify how many time points to remove at start (using 5)
  #   - note which template space is preferable (using MNI)
  #   - consider non-linear alignment via -tlrc_NL_warp
  #   - choose blur size (using FWHM = 4 mm)
  #   - choose basis function (using BLOCK(2,1), for example)
  #   - assuming 4 CPUs for linear regression
  #   - afni_proc.py will actually run the proc script (-execute)
  afni_proc.py -subj_id $subj_id                          \
      -blocks tshift align tlrc volreg blur mask regress  \
      -copy_anat $subj_indir/anat+orig                    \
      -dsets                                              \
          $subj_indir/epi_r1+orig                         \
          $subj_indir/epi_r2+orig                         \
          $subj_indir/epi_r3+orig                         \
      -tcat_remove_first_trs 5                            \
      -align_opts_aea -cost lpc+ZZ                        \
      -tlrc_base MNI152_2009_template.nii.gz              \
      -tlrc_NL_warp                                       \
      -volreg_align_to MIN_OUTLIER                        \
      -volreg_align_e2a                                   \
      -volreg_tlrc_warp                                   \
      -blur_size 4.0                                      \
      -regress_motion_per_run                             \
      -regress_censor_motion 0.3                          \
      -regress_reml_exec -regress_3dD_stop                \
      -regress_stim_times                                 \
          $stim_dir/houses.txt                            \
          $stim_dir/faces.txt                             \
          $stim_dir/doughnuts.txt                         \
          $stim_dir/pizza.txt                             \
      -regress_stim_labels                                \
          house face nuts za                              \
      -regress_basis 'BLOCK(2,1)'                         \
      -regress_opts_3dD                                   \
          -jobs 4                                         \
          -gltsym 'SYM: house -face' -glt_label 1 H-F     \
          -gltsym 'SYM: nuts -za'    -glt_label 2 N-Z     \
      -regress_est_blur_errts                             \
      -blocks tshift align tlrc volreg mask combine blur scale regress \
      -dsets_me_echo   epi_run*_echo_01.nii                            \
      -dsets_me_echo   epi_run*_echo_02.nii                            \
      -dsets_me_echo   epi_run*_echo_03.nii                            \
      -echo_times      15 30.5 41                                      \
      -mask_epi_anat   yes                                             \
      -combine_method  OC
      -execute

EOF
# EOF terminates the 'cat > run.afni_proc' command, above
# (it must not be indented in the script)

  # now run the analysis (generate proc and execute)
  tcsh run.afni_proc

# end loop over subjects
end

[handwerkerd]

In addition to the section you copied from https://afni.nimh.nih.gov/pub/dist/doc/program_help/afni_proc.py.html examples 12-13 list multiple ways to use multi-echo with AFNI. The one you copied includes -combine_method OC which means it will proceed using just the optimally combined data. To work with tedana -combine_method should be one of the following options:

            m_tedana         : tedana from MEICA group (dn_ts_OC.nii.gz)
            m_tedana_OC      : tedana OC from MEICA group (ts_OC.nii.gz)
            m_tedana_m_tedort: tedana from MEICA group (dn_ts_OC.nii.gz)
                               "tedort" from MEICA group
                               (--tedort: "good" projected from "bad")

If any of those options are used, it will run whichever version of tedana is at the top of the user's path.

-combine_opts_tedana allows users to pass any additional options directly to tedana One issue, which I suspect is the core of your question is that I tried -combine_opts_tedana --convention bids and that didn't work because afni_proc was always passing --convention orig. This is necessary because AFNI has its own file naming conventions. It would be possible to have AFNI use the tedana file name registry to identify the correct files for the next steps, but AFNI would still continue to use its own file naming conventions for all later processing steps.

Last time I asked @mrneont and @afni-rickr they had no plans to have afni_proc natively use something similar to BIDS-derivatives filenaming, but afni_proc creates a file name registry. It should be very possible to take that registry and remap all files into a BIDS structure. I think they did some work on that, but I don't know the current status.

[tsalo]

To work with tedana -combine_method should be one of the following options

My goal would actually be to run tedana separately, since these datasets are meant to act as inputs to tedana. Can afni_proc be used to produce minimally-preprocessed echo-wise outputs?

It should be very possible to take that registry and remap all files into a BIDS structure. I think they did some work on that, but I don't know the current status.

That's a great idea! I'll try that.

[handwerkerd]

If you use you the example you copied with -combine_method OC then it would unnecessary calculate a T2* map and the optimally combined time series, but it would give you close to what you want.
For the -blocks option, you'd also want to remove everything after combine. That is replace -blocks tshift align tlrc volreg mask combine blur scale regress with -blocks tshift align tlrc volreg mask combine With the shorter version, some useful quality control outputs wouldn't be given, but that may or may not be an issue depending on how you plan to use these data. (Also there's something wrong with the command you copied above in that has two -blocks optiosn and both -dsets & -dsets_me_echo This is different from their docs so I'm not sure where you got it from)

[tsalo]

@handwerkerd sorry for the noise. I don't know where I copied the incorrect parameters from.

As far as a good repo for a template script goes, what about the Multi-echo masking test dataset (https://openneuro.org/datasets/ds002156)? There's only one subject, so it's small.

The inputs can be:

sub-23638/ses-01/anat/sub-23638_ses-01_acq-mpragePURE_run-1_T1w.nii.gz
sub-23638/ses-01/func/sub-23638_ses-01_task-rest_run-1_echo-1_bold.nii.gz
sub-23638/ses-01/func/sub-23638_ses-01_task-rest_run-1_echo-2_bold.nii.gz
sub-23638/ses-01/func/sub-23638_ses-01_task-rest_run-1_echo-3_bold.nii.gz
sub-23638/ses-01/func/sub-23638_ses-01_task-rest_run-1_echo-4_bold.nii.gz

The echo times are 15.1, 28.7, 42.3, and 55.9.

I'm hoping that afni_proc can be used to produce (1) minimally-preprocessed echo-wise data to run tedana on, (2) a brain mask in the echo-wise images' space for tedana, and (3) optimally-combined data in standard space, which I could denoise using the tedana results. What do you think?

[mrneont]

Howdy-

We actually do have a mapping of afni_proc.py outputs to current BIDS-derivatives naming; we just haven't had time to implement that as an option yet. Hopefully soon after the Bootcamp...

For multiecho FMRI data, we do have a fair number of afni_proc.py examples in the APMULTI demo, which is downloadable for resting state FMRI with:

@Install_APMULTI_Demo1_rest

https://afni.nimh.nih.gov/pub/dist/OHBM2022/OHBM2022_tayloretal_apmulti.pdf

@tsalo : it sounds like you want just the datasets output by using OC or tedana, then? Or with further regression processing? Are the pbcombine datasets what you would like, at present?

--pt

[handwerkerd]

@tsalo Do you want the data aligned to a template space or in native space? If you want a template, which one?

[tsalo]

@mrneont that's awesome! I mainly want these AFNI derivatives to be useful for testing tedana, so the primary output I'm interested in is minimally-preprocessed individual echoes, along with some way to either transform outputs from running tedana separately on those echoes to standard space or optimally combined data in standard space that can be denoised using the results from running tedana separately. I'd love to have regressors that could be used for regression (e.g., motion parameters), but not denoised data.

@handwerkerd if possible, I'd love to have the echo-wise outputs in native space (per our recommendations in the tedana docs) and optimally combined outputs in standard space. Maybe MNI152NLin6Asym, since that's the one I'll use for fMRIPrep?

[handwerkerd]

When afni_proc aligns to a template it does a single spatial transformation for both motion correction and alignment. (Probably time to revisit that recommendation ME-ICA/tedana#990 ).

Do get what you want, I can write one afni_proc with and one without spatial alignment, or I can just run with spatial alignment and use a stand-along AFNI command to create volumes for each echo that aren't aligned to the template.

AFNI includes a bunch of MNI templates, but I'm not sure which one matches MNI152NLin6Asym. Can you point me to the template volumes fMRIPrep uses so I can make sure to align to the exact same one?

[tsalo]

Do get what you want, I can write one afni_proc with and one without spatial alignment, or I can just run with spatial alignment and use a stand-along AFNI command to create volumes for each echo that aren't aligned to the template.

That would be great, thank you!

AFNI includes a bunch of MNI templates, but I'm not sure which one matches MNI152NLin6Asym. Can you point me to the template volumes fMRIPrep uses so I can make sure to align to the exact same one?

fMRIPrep grabs its templates from templateflow (https://www.templateflow.org/browse/). The template is under the tpl-MNI152NLin6Asym folder. It's the file tpl-MNI152NLin6Asym_res-02_T1w.nii.gz.

[afni-rickr]

AFNI does not need to have a template in the distribution to use it for template alignment. If there is one you like, just download and give it to afni_proc.py with the -tlrc_base option. Yes, you can give afni_proc.py the echoes and just do the steps prior to 'combine', but really, it does not hurt to add 'combine' and 'regress' blocks. That would lead to a much more detailed QC report, and it won't change the datasets that you would want to use without those blocks. The cost is just extra disk space. But whatever you prefer is fine.

…

-rick

________________________________ From: Taylor Salo ***@***.***> Sent: Tuesday, October 24, 2023 2:38 PM To: ME-ICA/open-multi-echo-data ***@***.***> Cc: Reynolds, Richard C. (NIH/NIMH) [E] ***@***.***>; Mention ***@***.***> Subject: [EXTERNAL] Re: [ME-ICA/open-multi-echo-data] Finalize `afni_proc.py` settings (Issue #29) Do get what you want, I can write one afni_proc with and one without spatial alignment, or I can just run with spatial alignment and use a stand-along AFNI command to create volumes for each echo that aren't aligned to the template. That would be great, thank you! AFNI includes a bunch of MNI templates, but I'm not sure which one matches MNI152NLin6Asym. Can you point me to the template volumes fMRIPrep uses so I can make sure to align to the exact same one? fMRIPrep grabs its templates from templateflow (https://www.templateflow.org/browse/). The template is under the tpl-MNI152NLin6Asym folder. It's the file tpl-MNI152NLin6Asym_res-02_T1w.nii.gz. — Reply to this email directly, view it on GitHub<#29 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACKLXXNOUENLMS2KLAXGB23YBADKLAVCNFSM6AAAAAA6KHEHB6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONZXHAYTIMJYGA>. You are receiving this because you were mentioned.Message ID: ***@***.***> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and are confident the content is safe.

[handwerkerd]

Here is a sample script for the above dataset. I have a version that runs regress to get a bunch of quality control outputs. This really might be useful, but, for this run something was going wrong with the MNI alignment. My best guess is that either the template image or the original data has something that diverges from the NIFTI standard and makes AFNI unhappy. I can talk more to @afni-rickr to check/confirm the problem isn't on AFNI's side, but the following should work fine for your current plans.

sbjID=sub-23638
ses=ses-01
task=rest_run-1
echo_times=(15.1 28.7 42.3 55.9)
rootdir=~/code/meica/data_testing/Multi-echo_masking_test_dataset/${sbjID}/${ses}
anat=anat/${sbjID}_${ses}_acq-mpragePURE_run-1_T1w.nii.gz

# The data had several non-steady state volumes at the beginning.
# This will be used to remove the first four
remove_first_trs=4

# template from here: https://www.templateflow.org/browse/ in tpl-MNI152NLin6Asym
# Using the T1 that was already skull stripped
template=../../template/tpl-MNI152NLin6Asym_res-02_desc-brain_T1w.nii.gz

# You probably want to put files elsewhere, but fine for this demo
cd $rootdir

# Runs AFNI proc in native space
# The pre-processed runs to input into tedana are called pb03.${sbjID}.r01.e0?.volreg+orig.HEAD
afni_proc.py -subj_id ${sbjID} \
   -blocks despike tshift align volreg mask combine tlrc \
   -dsets_me_run  ./func/${sbjID}_${ses}_task-${task}_echo-1_bold.nii.gz \
       ./func/${sbjID}_${ses}_task-${task}_echo-2_bold.nii.gz \
       ./func/${sbjID}_${ses}_task-${task}_echo-3_bold.nii.gz \
       ./func/${sbjID}_${ses}_task-${task}_echo-4_bold.nii.gz \
   -tlrc_base $template \
   -echo_times $echo_times \
   -reg_echo 2 \
   -copy_anat $anat \
   -anat_has_skull yes \
   -mask_epi_anat yes \
   -align_unifize_epi local \
   -volreg_align_e2a \
   -tcat_remove_first_trs $remove_first_trs \
   -execute

# The above afni_proc calculates an alignment to template space, but doesn't use it
# The following will apply the transform to write out the echo data in template space
# The new files are called pb03a.${sbjID}.r01.e01.volreg+tlrc.HEAD (lousy naming, acknowledged) 
cd ${rootdir}/${sbjID}.results
echo_list=(01 02 03 04)
for eind in ${echo_list[@]}; do
    3dAllineate -base ../$template    \
                   -input pb03.${sbjID}.r01.e${eind}.volreg+orig.HEAD                 \
                   -1Dmatrix_apply warp.anat.Xat.1D                    \
                   -mast_dxyz 3.5                                             \
                   -prefix pb03a.${sbjID}.r01.e${eind}.volreg+tlrc.HEAD
done

[afni-rickr]

Hi @handwerkerd. To get the EPI into standard space (without the extra resampling at the end), use:
-blocks despike tshift align tlrc volreg mask combine regress
It needs the tlrc block before volreg in order to concatenate the transformations in the volreg block. And I just threw the 'regress' block in there for QC.
I will try to get this data and try the command locally.

[afni-rickr]

Hi @handwerkerd. The script worked, at least at a quick glance. I did use the blocks from above
-blocks despike tshift align tlrc volreg mask combine regress
and to actually warp the EPI into standard space requires the -volreg_tlrc_warp option.

[tsalo]

Thanks so much @afni-rickr @handwerkerd @mrneont! Our server is down at the moment, but I will try the updated command on one of the example datasets soon.