concurrent.futures error

[madeleinebarter]

What happened?

I ran mriqc using the docker version on one participant and received the error code: exception calling callback for <Future at 0x2aaac0288e50 state=finished raised BrokenProcessPool>

I am running the latest docker version on a Mac Studio with an M2 chip and macOS Sonoma

What command did you use?

docker run --platform linux/amd64 -it --rm -v /Volumes/clippdata/IDP/bids2:/data:ro -v /Volumes/clippdata/IDP/mriqc:/out nipreps/mriqc:latest /data /out participant --participant_label 001

What version of the software are you running?

24.1.0

How are you running this software?

Docker

Is your data BIDS valid?

Yes

Are you reusing any previously computed results?

No

Please copy and paste any relevant log output.

No response

Additional information / screenshots

[oesteban]

Hi @madeleinebarter, this looks like a memory error. Although I'm actively working on making this more reliable, in the case of having many subjects you may want to play a little with the options --omp-nthreads and --nprocs.

Can you elaborate more on the characteristics of your images (number of images, modalities, etc.) and your machine (number of cpus, memory available, etc.)?

It would be also helpful to run mriqc a bit more verbose (e.g., set -vv or -vvv).

Finally, copying and pasting the last config file generated at <work-directory>/config-<timestamp>-<unique-id>.toml would also be of help.

[Armos05]

Hey @oesteban , I also have had a similar error for some time now, " ERROR | concurrent.futures | exception calling callback for <Future at 0x7f622a65b8d0 state=finished raised BrokenProcessPool>"

I also tried implementing some solutions to get around this problem.

• Tried limiting memory usage using --mem. Same error.
• Tried using --use-plugin options. "Got a strange "File Not Found Error" for the attached plugin.yml file.
• Currently using the following command docker run -it --rm -v /home/test_data/Nifti:/data:ro -v /home/test_data/mriqc2:/out nipreps/mriqc:latest /data /out participant --participant_label 01 --no-sub --omp-nthreads 1 --nprocs 1 --mem 4

System Information:
Running MRIQC version 24.1.0, on Linux (8GB) and MAC OS(16 GB), latest Docker Version, platforms, but got the same error in both cases.

I have attached the log files (linux and MAC OS) for your reference.
crash-20240507-073505-root-UploadMetrics.a0-4ce7c5f2-2f4c-4dc8-a8d3-b684affa8fd9.txt
mriqc-20240507-083540_d7ea3439-f37f-449c-9158-9ae617df4d5f.log
MACOS_mriqc-20240507-081646_441bbab0-acb9-43c4-9304-c6a9e99cff5a.log

Thanks in advance

[oesteban]

@Armos05 - by looking at your crashfile, it seems our server was down when you ran MRIQC.

Can you try again with the same arguments? If it still crashes, can you run MRIQC while adding the argument --no-sub?

[Armos05]

@oesteban, Thanks for looking into it.
MRIQC works now!!

Although, I was unable to perform group analysis, MRIQC can calculate all the parameters into the json file but there are no visual renders of those values into the graphs to look at in the final HTML file.

Here the config file:
[environment]
cpu_count = 8
exec_env = "singularity"
free_mem = 13.2
freesurfer_home = "/opt/freesurfer"
overcommit_policy = "heuristic"
overcommit_limit = "50%"
nipype_version = "1.8.6"
synthstrip_path = "PosixPath('/opt/freesurfer/models/synthstrip.1.pt')"
templateflow_version = "24.2.0"
total_memory = 15.365371704101562
version = "24.1.0.dev0+g3fe90466.d20240417"

[execution]
ants_float = false
bids_dir = "/data"
bids_dir_datalad = false
bids_database_dir = "/out/.bids_db"
bids_database_wipe = false
cwd = "/tmp"
datalad_get = true
debug = false
dry_run = false
dsname = ""
float32 = true
layout = "BIDS Layout: /data"
log_dir = "/out/logs"
log_level = 25
modalities = [ "T1w", "T2w", "bold", "dwi",]
no_sub = true
notrack = false
output_dir = "/out"
participant_label = [ "01", "32", "33",]
pdb = false
reports_only = false
resource_monitor = false
run_uuid = "20240521-083912_05591ae9-5501-48d5-9296-462c064a37a7"
templateflow_home = "/templateflow"
upload_strict = false
verbose_reports = false
webapi_url = "https://mriqc.nimh.nih.gov:443/api/v1"
work_dir = "/tmp/work"
write_graph = false

[workflow]
analysis_level = [ "group",]
biggest_file_gb = 1.1229652797928313e-10
deoblique = false
despike = false
fd_thres = 0.2
fd_radius = 50
fft_spikes_detector = false
min_len_dwi = 7
min_len_bold = 5
species = "human"
template_id = "MNI152NLin2009cAsym"

[nipype]
crashfile_format = "txt"
get_linked_libs = false
local_hash_check = true
nprocs = 2
omp_nthreads = 2
plugin = "MultiProc"
remove_node_directories = false
resource_monitor = false
stop_on_first_crash = true

[settings]
file_path = "/out/logs/config-20240521-083912_05591ae9-5501-48d5-9296-462c064a37a7.toml"
start_time = 1716280752.2356057

[execution.bids_filters]

[workflow.inputs]
t1w = [ "/data/sub-01/anat/sub-01_T1w.nii.gz", "/data/sub-32/anat/sub-32_T1w.nii.gz", "/data/sub-33/anat/sub-33_T1w.nii.gz",]
bold = [ "/data/sub-01/func/sub-01_task-rest_bold.nii.gz", "/data/sub-32/func/sub-32_task-rest_bold.nii.gz", "/data/sub-33/func/sub-33_task-rest_bold.nii.gz",]

[nipype.plugin_args]
maxtasksperchild = 1
raise_insufficient = false