Multiple testing correction over contrasts for brain imaging

Abstract

The multiple testing problem appears in brain imaging in the context of the general linear model in two forms: a statistical test is performed at each voxel or vertex of the image, and multiple contrasts are tested over the same data. The first has been greatly studied and various different procedures have been proposed, e.g., Bonferroni, random field theory, non-parametric approaches and false discovery rate. The second arises when there are various hypotheses (contrasts) about the same model, or different models are analyzed using the same data. If left uncontrolled, such multiplicity can lead to an undesirably high false positive rate, and spurious effects can be interpreted as real. Even though a number of methods have been proposed for contrast correction in non-imaging fields, most of these have seen little use in brain imaging, and often the brain analyses are reported without such correction. Thus, in this work, I discuss and compare the correction performance from Bonferroni, Dunn–Šidák, Fisher’s lsd, Tukey, Scheffé, Fisher–Hayter, Wang–Cui and Westfall–Young permutation method using both simulated data and real data from imaging studies of the brain. Although some procedures had good performance in some simulation scenarios, permutation method was the most suitable method to correct for multiple testing: it can be used to correct across both contrasts and voxels (or vertices), showed a strong control of the fwer, can be used with balanced and unbalanced models, and held one of the highest power independently of the number of contrasts tested or their dependency structure. I also confirmed that Fisher’s lsd presents a weak control of the fwer for more than three groups and, therefore, is invalid. Using magnetic resonance images of the brain, I showed how these methods can be applied to different types of analysis, such that they can be used as guideline for anyone working with neuroimaging.