How ignoring physiological noise can bias the conclusions from fMRI simulation results

M. Welvaert, Yves Rosseel

    Research output: Contribution to journalArticle

    9 Citations (Scopus)

    Abstract

    Neuroimaging researchers use simulation studies to validate their statistical methods because it is acknowledged that this is the most feasible way to know the ground truth of the data. The noise model used in these studies typically varies from a simple Gaussian distribution to an estimate of the noise distribution from real data. However, although several studies point out the presence of physiological noise in fMRI data, this noise source is currently lacking in simulation studies. Therefore, we explored the impact of adding physiological noise to the simulated data. For several experimental designs, fMRI data were generated under different noise models while the signal-to-noise ratio was kept constant. The sensitivity and specificity of a standard statistical parametric mapping (SPM) analysis were determined by comparing the known activation with the detected activation. We show that by including physiological noise in the data generation process, the simulation results in terms of sensitivity and specificity drop dramatically. Additionally, we used the new proposed simulation model to compare a standard SPM analysis against the method proposed by Cabella et al. (2009). The results indicate that the analysis of data containing no physiological noise yields a better performance of the SPM analysis. However, if physiological noise is included in the data, the sensitivity and specificity of the Cabella method are higher compared to the SPM analysis. Based on these results, we argue that the results of current simulation studies are likely to be biased, especially when analysis methods are compared using ROC curves. (C) 2012 Elsevier B.V. All rights reserved
    Original languageEnglish
    Pages (from-to)125-132
    JournalJournal of Neuroscience Methods
    Volume211
    Issue number1
    DOIs
    Publication statusPublished - 15 Oct 2012

    Cite this

    @article{90a9af94227549919acf2993f412e186,
    title = "How ignoring physiological noise can bias the conclusions from fMRI simulation results",
    abstract = "Neuroimaging researchers use simulation studies to validate their statistical methods because it is acknowledged that this is the most feasible way to know the ground truth of the data. The noise model used in these studies typically varies from a simple Gaussian distribution to an estimate of the noise distribution from real data. However, although several studies point out the presence of physiological noise in fMRI data, this noise source is currently lacking in simulation studies. Therefore, we explored the impact of adding physiological noise to the simulated data. For several experimental designs, fMRI data were generated under different noise models while the signal-to-noise ratio was kept constant. The sensitivity and specificity of a standard statistical parametric mapping (SPM) analysis were determined by comparing the known activation with the detected activation. We show that by including physiological noise in the data generation process, the simulation results in terms of sensitivity and specificity drop dramatically. Additionally, we used the new proposed simulation model to compare a standard SPM analysis against the method proposed by Cabella et al. (2009). The results indicate that the analysis of data containing no physiological noise yields a better performance of the SPM analysis. However, if physiological noise is included in the data, the sensitivity and specificity of the Cabella method are higher compared to the SPM analysis. Based on these results, we argue that the results of current simulation studies are likely to be biased, especially when analysis methods are compared using ROC curves. (C) 2012 Elsevier B.V. All rights reserved",
    keywords = "fMRI, Simulation, Physiological noise, ROC analysis, Model validation",
    author = "M. Welvaert and Yves Rosseel",
    year = "2012",
    month = "10",
    day = "15",
    doi = "10.1016/j.jneumeth.2012.08.022",
    language = "English",
    volume = "211",
    pages = "125--132",
    journal = "Journal of Neuroscience Methods",
    issn = "0165-0270",
    publisher = "Elsevier",
    number = "1",

    }

    How ignoring physiological noise can bias the conclusions from fMRI simulation results. / Welvaert, M.; Rosseel, Yves.

    In: Journal of Neuroscience Methods, Vol. 211, No. 1, 15.10.2012, p. 125-132.

    Research output: Contribution to journalArticle

    TY - JOUR

    T1 - How ignoring physiological noise can bias the conclusions from fMRI simulation results

    AU - Welvaert, M.

    AU - Rosseel, Yves

    PY - 2012/10/15

    Y1 - 2012/10/15

    N2 - Neuroimaging researchers use simulation studies to validate their statistical methods because it is acknowledged that this is the most feasible way to know the ground truth of the data. The noise model used in these studies typically varies from a simple Gaussian distribution to an estimate of the noise distribution from real data. However, although several studies point out the presence of physiological noise in fMRI data, this noise source is currently lacking in simulation studies. Therefore, we explored the impact of adding physiological noise to the simulated data. For several experimental designs, fMRI data were generated under different noise models while the signal-to-noise ratio was kept constant. The sensitivity and specificity of a standard statistical parametric mapping (SPM) analysis were determined by comparing the known activation with the detected activation. We show that by including physiological noise in the data generation process, the simulation results in terms of sensitivity and specificity drop dramatically. Additionally, we used the new proposed simulation model to compare a standard SPM analysis against the method proposed by Cabella et al. (2009). The results indicate that the analysis of data containing no physiological noise yields a better performance of the SPM analysis. However, if physiological noise is included in the data, the sensitivity and specificity of the Cabella method are higher compared to the SPM analysis. Based on these results, we argue that the results of current simulation studies are likely to be biased, especially when analysis methods are compared using ROC curves. (C) 2012 Elsevier B.V. All rights reserved

    AB - Neuroimaging researchers use simulation studies to validate their statistical methods because it is acknowledged that this is the most feasible way to know the ground truth of the data. The noise model used in these studies typically varies from a simple Gaussian distribution to an estimate of the noise distribution from real data. However, although several studies point out the presence of physiological noise in fMRI data, this noise source is currently lacking in simulation studies. Therefore, we explored the impact of adding physiological noise to the simulated data. For several experimental designs, fMRI data were generated under different noise models while the signal-to-noise ratio was kept constant. The sensitivity and specificity of a standard statistical parametric mapping (SPM) analysis were determined by comparing the known activation with the detected activation. We show that by including physiological noise in the data generation process, the simulation results in terms of sensitivity and specificity drop dramatically. Additionally, we used the new proposed simulation model to compare a standard SPM analysis against the method proposed by Cabella et al. (2009). The results indicate that the analysis of data containing no physiological noise yields a better performance of the SPM analysis. However, if physiological noise is included in the data, the sensitivity and specificity of the Cabella method are higher compared to the SPM analysis. Based on these results, we argue that the results of current simulation studies are likely to be biased, especially when analysis methods are compared using ROC curves. (C) 2012 Elsevier B.V. All rights reserved

    KW - fMRI

    KW - Simulation

    KW - Physiological noise

    KW - ROC analysis

    KW - Model validation

    U2 - 10.1016/j.jneumeth.2012.08.022

    DO - 10.1016/j.jneumeth.2012.08.022

    M3 - Article

    VL - 211

    SP - 125

    EP - 132

    JO - Journal of Neuroscience Methods

    JF - Journal of Neuroscience Methods

    SN - 0165-0270

    IS - 1

    ER -