TY - JOUR
T1 - How ignoring physiological noise can bias the conclusions from fMRI simulation results
AU - Welvaert, M.
AU - Rosseel, Yves
PY - 2012/10/15
Y1 - 2012/10/15
N2 - Neuroimaging researchers use simulation studies to validate their statistical methods because it is acknowledged that this is the most feasible way to know the ground truth of the data. The noise model used in these studies typically varies from a simple Gaussian distribution to an estimate of the noise distribution from real data. However, although several studies point out the presence of physiological noise in fMRI data, this noise source is currently lacking in simulation studies. Therefore, we explored the impact of adding physiological noise to the simulated data. For several experimental designs, fMRI data were generated under different noise models while the signal-to-noise ratio was kept constant. The sensitivity and specificity of a standard statistical parametric mapping (SPM) analysis were determined by comparing the known activation with the detected activation. We show that by including physiological noise in the data generation process, the simulation results in terms of sensitivity and specificity drop dramatically. Additionally, we used the new proposed simulation model to compare a standard SPM analysis against the method proposed by Cabella et al. (2009). The results indicate that the analysis of data containing no physiological noise yields a better performance of the SPM analysis. However, if physiological noise is included in the data, the sensitivity and specificity of the Cabella method are higher compared to the SPM analysis. Based on these results, we argue that the results of current simulation studies are likely to be biased, especially when analysis methods are compared using ROC curves. (C) 2012 Elsevier B.V. All rights reserved
AB - Neuroimaging researchers use simulation studies to validate their statistical methods because it is acknowledged that this is the most feasible way to know the ground truth of the data. The noise model used in these studies typically varies from a simple Gaussian distribution to an estimate of the noise distribution from real data. However, although several studies point out the presence of physiological noise in fMRI data, this noise source is currently lacking in simulation studies. Therefore, we explored the impact of adding physiological noise to the simulated data. For several experimental designs, fMRI data were generated under different noise models while the signal-to-noise ratio was kept constant. The sensitivity and specificity of a standard statistical parametric mapping (SPM) analysis were determined by comparing the known activation with the detected activation. We show that by including physiological noise in the data generation process, the simulation results in terms of sensitivity and specificity drop dramatically. Additionally, we used the new proposed simulation model to compare a standard SPM analysis against the method proposed by Cabella et al. (2009). The results indicate that the analysis of data containing no physiological noise yields a better performance of the SPM analysis. However, if physiological noise is included in the data, the sensitivity and specificity of the Cabella method are higher compared to the SPM analysis. Based on these results, we argue that the results of current simulation studies are likely to be biased, especially when analysis methods are compared using ROC curves. (C) 2012 Elsevier B.V. All rights reserved
KW - fMRI
KW - Simulation
KW - Physiological noise
KW - ROC analysis
KW - Model validation
U2 - 10.1016/j.jneumeth.2012.08.022
DO - 10.1016/j.jneumeth.2012.08.022
M3 - Article
SN - 0165-0270
VL - 211
SP - 125
EP - 132
JO - Journal of Neuroscience Methods
JF - Journal of Neuroscience Methods
IS - 1
ER -