Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FDR multiple comparison correction in ft_sourcestatistics corrects for out of head voxels too #2263

Open
RAJKoster opened this issue Jun 23, 2023 · 1 comment

Comments

@RAJKoster
Copy link

Short version:
The number of comparisons the fdr.m script corrects for is defined by the elements in its input. However, with ft_sourcestatistics, its input is all the voxels in the MRI volume, not just those inside the brain. Hence, it is too stringent and effectively corrects for more comparisons than it should.

Longer version:
The use of false discovery rate correction, as per fdr.m, in ft_sourcestatistics is to correct for the multiple comparison (i.e., a statistical test for each voxel). Its input is stat.prob, representing p-values for every voxel in the MRI volume, whose number of elements describes the number of multiple comparisons it corrects for.
However, most voxels in this volume are not within the head. Logically tests on these voxels are not interpreted and should be discarded (also, they are NaN values), but they are currently counted towards the number of comparisons to correct for. I believe this to be a mistake. As a result, it currently effectively uses a critical alpha-threshold that is smaller (and can be MUCH smaller) than it should be.

Suggested solution:

  1. Input only the p-values of the voxels that are actually considered into the function, each time the function is called upon.

stat.mask = fdr(stat.prob, cfg.alpha);

should be:
stat.mask(stat.inside) = fdr(stat.prob(stat.inside), cfg.alpha);

  1. Correct only for the non-NaN p-values (the voxels that are actually tested) in the fdr.m function.

% count the number of voxels
V = length(p);
% compute the threshold probability for each voxel
pi = ((1:V)/V) * q / c(V);

should be:
% count the number of voxels
V = length(p);
m = sum(~isnan(p)); % count the number of tests to correct for
% compute the threshold probability for each voxel
pi = ((1:V)/m) * q / c(m);

fieldtrip version: from Mar 2021

@robertoostenveld
Copy link
Contributor

fieldtrip version: from Mar 2021

Could you first check whether your still applies to the latest release? I don't want to look into a >2 year old version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants