Power, effects, confidence, and significance: An investigation of statistical practices in nursing research

CADEYRN GASKIN, Brenda HAPPELL

Research output: Contribution to journalArticle

24 Citations (Scopus)

Abstract

Objectives: To (a) assess the statistical power of nursing research to detect small, medium, and large effect sizes; (b) estimate the experiment-wise Type I error rate in these studies; and (c) assess the extent to which (i) a priori power analyses, (ii) effect sizes (and interpretations thereof), and (iii) confidence intervals were reported. Design: Statistical review. Data sources: Papers published in the 2011 volumes of the 10 highest ranked nursing journals, based on their 5-year impact factors. Review methods: Papers were assessed for statistical power, control of experiment-wise Type I error, reporting of a priori power analyses, reporting and interpretation of effect sizes, and reporting of confidence intervals. The analyses were based on 333 papers, from which 10,337 inferential statistics were identified. Results: The median power to detect small, medium, and large effect sizes was .40 (interquartile range [. IQR]. = .24-.71), .98 (IQR= .85-1.00), and 1.00 (IQR= 1.00-1.00), respectively. The median experiment-wise Type I error rate was .54 (IQR= .26-.80). A priori power analyses were reported in 28% of papers. Effect sizes were routinely reported for Spearman's rank correlations (100% of papers in which this test was used), Poisson regressions (100%), odds ratios (100%), Kendall's tau correlations (100%), Pearson's correlations (99%), logistic regressions (98%), structural equation modelling/confirmatory factor analyses/path analyses (97%), and linear regressions (83%), but were reported less often for two-proportion z tests (50%), analyses of variance/analyses of covariance/multivariate analyses of variance (18%), t tests (8%), Wilcoxon's tests (8%), Chi-squared tests (8%), and Fisher's exact tests (7%), and not reported for sign tests, Friedman's tests, McNemar's tests, multi-level models, and Kruskal-Wallis tests. Effect sizes were infrequently interpreted. Confidence intervals were reported in 28% of papers. Conclusion: The use, reporting, and interpretation of inferential statistics in nursing research need substantial improvement. Most importantly, researchers should abandon the misleading practice of interpreting the results from inferential tests based solely on whether they are statistically significant (or not) and, instead, focus on reporting and interpreting effect sizes, confidence intervals, and significance levels. Nursing researchers also need to conduct and report a priori power analyses, and to address the issue of Type I experiment-wise error inflation in their studies.

Original languageEnglish
Pages (from-to)795-806
Number of pages12
JournalInternational Journal of Nursing Studies
Volume51
Issue number5
DOIs
Publication statusPublished - 2014

Fingerprint

Nursing Research
Confidence Intervals
Analysis of Variance
Nursing
Research Personnel
Information Storage and Retrieval
Economic Inflation
Statistical Factor Analysis
Linear Models
Multivariate Analysis
Logistic Models
Odds Ratio

Cite this

@article{888e99af535a4aa7889038b46933e585,
title = "Power, effects, confidence, and significance: An investigation of statistical practices in nursing research",
abstract = "Objectives: To (a) assess the statistical power of nursing research to detect small, medium, and large effect sizes; (b) estimate the experiment-wise Type I error rate in these studies; and (c) assess the extent to which (i) a priori power analyses, (ii) effect sizes (and interpretations thereof), and (iii) confidence intervals were reported. Design: Statistical review. Data sources: Papers published in the 2011 volumes of the 10 highest ranked nursing journals, based on their 5-year impact factors. Review methods: Papers were assessed for statistical power, control of experiment-wise Type I error, reporting of a priori power analyses, reporting and interpretation of effect sizes, and reporting of confidence intervals. The analyses were based on 333 papers, from which 10,337 inferential statistics were identified. Results: The median power to detect small, medium, and large effect sizes was .40 (interquartile range [. IQR]. = .24-.71), .98 (IQR= .85-1.00), and 1.00 (IQR= 1.00-1.00), respectively. The median experiment-wise Type I error rate was .54 (IQR= .26-.80). A priori power analyses were reported in 28{\%} of papers. Effect sizes were routinely reported for Spearman's rank correlations (100{\%} of papers in which this test was used), Poisson regressions (100{\%}), odds ratios (100{\%}), Kendall's tau correlations (100{\%}), Pearson's correlations (99{\%}), logistic regressions (98{\%}), structural equation modelling/confirmatory factor analyses/path analyses (97{\%}), and linear regressions (83{\%}), but were reported less often for two-proportion z tests (50{\%}), analyses of variance/analyses of covariance/multivariate analyses of variance (18{\%}), t tests (8{\%}), Wilcoxon's tests (8{\%}), Chi-squared tests (8{\%}), and Fisher's exact tests (7{\%}), and not reported for sign tests, Friedman's tests, McNemar's tests, multi-level models, and Kruskal-Wallis tests. Effect sizes were infrequently interpreted. Confidence intervals were reported in 28{\%} of papers. Conclusion: The use, reporting, and interpretation of inferential statistics in nursing research need substantial improvement. Most importantly, researchers should abandon the misleading practice of interpreting the results from inferential tests based solely on whether they are statistically significant (or not) and, instead, focus on reporting and interpreting effect sizes, confidence intervals, and significance levels. Nursing researchers also need to conduct and report a priori power analyses, and to address the issue of Type I experiment-wise error inflation in their studies.",
keywords = "Nursing research, Research design, Review literature as topic, Statistics, Statistics as topic",
author = "CADEYRN GASKIN and Brenda HAPPELL",
year = "2014",
doi = "10.1016/j.ijnurstu.2013.09.014",
language = "English",
volume = "51",
pages = "795--806",
journal = "International Journal of Nursing Studies",
issn = "0020-7489",
publisher = "Elsevier Limited",
number = "5",

}

Power, effects, confidence, and significance: An investigation of statistical practices in nursing research. / GASKIN, CADEYRN; HAPPELL, Brenda.

In: International Journal of Nursing Studies, Vol. 51, No. 5, 2014, p. 795-806.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Power, effects, confidence, and significance: An investigation of statistical practices in nursing research

AU - GASKIN, CADEYRN

AU - HAPPELL, Brenda

PY - 2014

Y1 - 2014

N2 - Objectives: To (a) assess the statistical power of nursing research to detect small, medium, and large effect sizes; (b) estimate the experiment-wise Type I error rate in these studies; and (c) assess the extent to which (i) a priori power analyses, (ii) effect sizes (and interpretations thereof), and (iii) confidence intervals were reported. Design: Statistical review. Data sources: Papers published in the 2011 volumes of the 10 highest ranked nursing journals, based on their 5-year impact factors. Review methods: Papers were assessed for statistical power, control of experiment-wise Type I error, reporting of a priori power analyses, reporting and interpretation of effect sizes, and reporting of confidence intervals. The analyses were based on 333 papers, from which 10,337 inferential statistics were identified. Results: The median power to detect small, medium, and large effect sizes was .40 (interquartile range [. IQR]. = .24-.71), .98 (IQR= .85-1.00), and 1.00 (IQR= 1.00-1.00), respectively. The median experiment-wise Type I error rate was .54 (IQR= .26-.80). A priori power analyses were reported in 28% of papers. Effect sizes were routinely reported for Spearman's rank correlations (100% of papers in which this test was used), Poisson regressions (100%), odds ratios (100%), Kendall's tau correlations (100%), Pearson's correlations (99%), logistic regressions (98%), structural equation modelling/confirmatory factor analyses/path analyses (97%), and linear regressions (83%), but were reported less often for two-proportion z tests (50%), analyses of variance/analyses of covariance/multivariate analyses of variance (18%), t tests (8%), Wilcoxon's tests (8%), Chi-squared tests (8%), and Fisher's exact tests (7%), and not reported for sign tests, Friedman's tests, McNemar's tests, multi-level models, and Kruskal-Wallis tests. Effect sizes were infrequently interpreted. Confidence intervals were reported in 28% of papers. Conclusion: The use, reporting, and interpretation of inferential statistics in nursing research need substantial improvement. Most importantly, researchers should abandon the misleading practice of interpreting the results from inferential tests based solely on whether they are statistically significant (or not) and, instead, focus on reporting and interpreting effect sizes, confidence intervals, and significance levels. Nursing researchers also need to conduct and report a priori power analyses, and to address the issue of Type I experiment-wise error inflation in their studies.

AB - Objectives: To (a) assess the statistical power of nursing research to detect small, medium, and large effect sizes; (b) estimate the experiment-wise Type I error rate in these studies; and (c) assess the extent to which (i) a priori power analyses, (ii) effect sizes (and interpretations thereof), and (iii) confidence intervals were reported. Design: Statistical review. Data sources: Papers published in the 2011 volumes of the 10 highest ranked nursing journals, based on their 5-year impact factors. Review methods: Papers were assessed for statistical power, control of experiment-wise Type I error, reporting of a priori power analyses, reporting and interpretation of effect sizes, and reporting of confidence intervals. The analyses were based on 333 papers, from which 10,337 inferential statistics were identified. Results: The median power to detect small, medium, and large effect sizes was .40 (interquartile range [. IQR]. = .24-.71), .98 (IQR= .85-1.00), and 1.00 (IQR= 1.00-1.00), respectively. The median experiment-wise Type I error rate was .54 (IQR= .26-.80). A priori power analyses were reported in 28% of papers. Effect sizes were routinely reported for Spearman's rank correlations (100% of papers in which this test was used), Poisson regressions (100%), odds ratios (100%), Kendall's tau correlations (100%), Pearson's correlations (99%), logistic regressions (98%), structural equation modelling/confirmatory factor analyses/path analyses (97%), and linear regressions (83%), but were reported less often for two-proportion z tests (50%), analyses of variance/analyses of covariance/multivariate analyses of variance (18%), t tests (8%), Wilcoxon's tests (8%), Chi-squared tests (8%), and Fisher's exact tests (7%), and not reported for sign tests, Friedman's tests, McNemar's tests, multi-level models, and Kruskal-Wallis tests. Effect sizes were infrequently interpreted. Confidence intervals were reported in 28% of papers. Conclusion: The use, reporting, and interpretation of inferential statistics in nursing research need substantial improvement. Most importantly, researchers should abandon the misleading practice of interpreting the results from inferential tests based solely on whether they are statistically significant (or not) and, instead, focus on reporting and interpreting effect sizes, confidence intervals, and significance levels. Nursing researchers also need to conduct and report a priori power analyses, and to address the issue of Type I experiment-wise error inflation in their studies.

KW - Nursing research

KW - Research design

KW - Review literature as topic

KW - Statistics

KW - Statistics as topic

UR - http://www.scopus.com/inward/record.url?scp=84897056879&partnerID=8YFLogxK

UR - http://www.mendeley.com/research/power-effects-confidence-significance-investigation-statistical-practices-nursing-research

U2 - 10.1016/j.ijnurstu.2013.09.014

DO - 10.1016/j.ijnurstu.2013.09.014

M3 - Article

VL - 51

SP - 795

EP - 806

JO - International Journal of Nursing Studies

JF - International Journal of Nursing Studies

SN - 0020-7489

IS - 5

ER -