This article argues that while meta-analytic studies are widely used in psychological literature, heterogeneity and the potential for confounding remain major problems in the interpretation of meta-analytic study results. The article demonstrates the use of exploratory analysis including graphical methods prior to meta-analysis, and introduces a methodology to screen for artifactual effects. These procedures are illustrated on effect size data comparing depression treatment outcome from psychotherapy versus pharmacotherapy. Results support prior findings of a nonsignificant difference in effect size between the two treatments. They also support findings that treatment type accounts for only a very small proportion of outcome variance. However, the results indicate that some previously reported covariates of depression treatment outcome may be artifactual.
Collins, M., & Carey, T. (2015). Identification of Real and Artifactual Moderators of Effect Size in Meta-Analysis. Multivariate Behavioral Research, 50(1), 109-125. https://doi.org/10.1080/00273171.2014.963193