Identification of Real and Artifactual Moderators of Effect Size in Meta-Analysis

Mark Collins, Timothy Carey

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

This article argues that while meta-analytic studies are widely used in psychological literature, heterogeneity and the potential for confounding remain major problems in the interpretation of meta-analytic study results. The article demonstrates the use of exploratory analysis including graphical methods prior to meta-analysis, and introduces a methodology to screen for artifactual effects. These procedures are illustrated on effect size data comparing depression treatment outcome from psychotherapy versus pharmacotherapy. Results support prior findings of a nonsignificant difference in effect size between the two treatments. They also support findings that treatment type accounts for only a very small proportion of outcome variance. However, the results indicate that some previously reported covariates of depression treatment outcome may be artifactual.
Original languageEnglish
Pages (from-to)109-125
Number of pages17
JournalMultivariate Behavioral Research
Volume50
Issue number1
DOIs
Publication statusPublished - 2015

Fingerprint

Effect Size
Meta-Analysis
Depression
Psychotherapy
Psychology
Exploratory Analysis
Drug Therapy
Graphical Methods
Confounding
Therapeutics
Covariates
Proportion
Meta-analysis
Treatment Outcome
Methodology
Demonstrate
Pharmacotherapy
Psychological Literature

Cite this

Collins, Mark ; Carey, Timothy. / Identification of Real and Artifactual Moderators of Effect Size in Meta-Analysis. In: Multivariate Behavioral Research. 2015 ; Vol. 50, No. 1. pp. 109-125.
@article{b120f63a493645d5bac1db4914a29ff6,
title = "Identification of Real and Artifactual Moderators of Effect Size in Meta-Analysis",
abstract = "This article argues that while meta-analytic studies are widely used in psychological literature, heterogeneity and the potential for confounding remain major problems in the interpretation of meta-analytic study results. The article demonstrates the use of exploratory analysis including graphical methods prior to meta-analysis, and introduces a methodology to screen for artifactual effects. These procedures are illustrated on effect size data comparing depression treatment outcome from psychotherapy versus pharmacotherapy. Results support prior findings of a nonsignificant difference in effect size between the two treatments. They also support findings that treatment type accounts for only a very small proportion of outcome variance. However, the results indicate that some previously reported covariates of depression treatment outcome may be artifactual.",
author = "Mark Collins and Timothy Carey",
year = "2015",
doi = "10.1080/00273171.2014.963193",
language = "English",
volume = "50",
pages = "109--125",
journal = "Multivariate Behavioral Research",
issn = "0027-3171",
publisher = "Psychology Press Ltd",
number = "1",

}

Identification of Real and Artifactual Moderators of Effect Size in Meta-Analysis. / Collins, Mark; Carey, Timothy.

In: Multivariate Behavioral Research, Vol. 50, No. 1, 2015, p. 109-125.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Identification of Real and Artifactual Moderators of Effect Size in Meta-Analysis

AU - Collins, Mark

AU - Carey, Timothy

PY - 2015

Y1 - 2015

N2 - This article argues that while meta-analytic studies are widely used in psychological literature, heterogeneity and the potential for confounding remain major problems in the interpretation of meta-analytic study results. The article demonstrates the use of exploratory analysis including graphical methods prior to meta-analysis, and introduces a methodology to screen for artifactual effects. These procedures are illustrated on effect size data comparing depression treatment outcome from psychotherapy versus pharmacotherapy. Results support prior findings of a nonsignificant difference in effect size between the two treatments. They also support findings that treatment type accounts for only a very small proportion of outcome variance. However, the results indicate that some previously reported covariates of depression treatment outcome may be artifactual.

AB - This article argues that while meta-analytic studies are widely used in psychological literature, heterogeneity and the potential for confounding remain major problems in the interpretation of meta-analytic study results. The article demonstrates the use of exploratory analysis including graphical methods prior to meta-analysis, and introduces a methodology to screen for artifactual effects. These procedures are illustrated on effect size data comparing depression treatment outcome from psychotherapy versus pharmacotherapy. Results support prior findings of a nonsignificant difference in effect size between the two treatments. They also support findings that treatment type accounts for only a very small proportion of outcome variance. However, the results indicate that some previously reported covariates of depression treatment outcome may be artifactual.

UR - http://www.scopus.com/inward/record.url?scp=84924067710&partnerID=8YFLogxK

UR - http://www.mendeley.com/research/identification-real-artifactual-moderators-effect-size-metaanalysis

U2 - 10.1080/00273171.2014.963193

DO - 10.1080/00273171.2014.963193

M3 - Article

VL - 50

SP - 109

EP - 125

JO - Multivariate Behavioral Research

JF - Multivariate Behavioral Research

SN - 0027-3171

IS - 1

ER -