Analyzing cause and effect in environmental assessments: using weighted evidence from the literature

Richard Norris, J. Webb, Susan Nichols, Michael Stewardson, Evan Harrison

    Research output: Contribution to journalArticle

    57 Citations (Scopus)

    Abstract

    Sound decision making in environmental research and management requires an understanding of causal relationships between stressors and ecological responses. However, demonstrating causeâ¿¿effect relationships in natural systems is challenging because of difficulties with natural variability, performing experiments, lack of replication, and the presence of confounding influences. Thus, even the best-designed study may not establish causality. We describe a method that uses evidence available in the extensive published ecological literature to assess support for causeâ¿¿effect hypotheses in environmental investigations. Our method, called Eco Evidence, is a form of causal criteria analysisâ¿¿a technique developed by epidemiologists in the 1960sâ¿¿ who faced similar difficulties in attributing causation. The Eco Evidence method is an 8-step process in which the user conducts a systematic review of the evidence for one or more causeâ¿¿effect hypotheses to assess the level of support for an overall question. In contrast to causal criteria analyses in epidemiology, users of Eco Evidence use a subset of criteriamost relevant to environmental investigations and weight each piece of evidence according to its study design. Stronger studies contribute more to the assessment of causality, but weaker evidence is not discarded. This feature is important because environmental evidence is often scarce. The outputs of the analysis are a guide to the strength of evidence for or against the causeâ¿¿effect hypotheses. They strengthen confidence in the conclusions drawn from that evidence, but cannot ever prove causality. They also indicate situations where knowledge gaps signify insufficient evidence to reach a conclusion. The method is supported by the freely available Eco Evidence software package, which produces a standard report, maximizing the transparency and repeatability of any assessment. Environmental science has lagged behind other disciplines in systematic assessment of evidence to improve research and management. Using the Eco Evidence method, environmental scientists can better use the extensive published literature to guide evidence-based decisions and undertake transparent assessments of ecological cause and effect.
    Original languageEnglish
    Pages (from-to)5-21
    Number of pages17
    JournalFreshwater Science
    Volume31
    Issue number1
    DOIs
    Publication statusPublished - 2012

    Fingerprint

    environmental assessment
    methodology
    environmental research
    environmental science
    epidemiology
    transparency
    environmental management
    systematic review
    repeatability
    decision making
    method
    effect
    software
    experimental design
    experiment

    Cite this

    Norris, Richard ; Webb, J. ; Nichols, Susan ; Stewardson, Michael ; Harrison, Evan. / Analyzing cause and effect in environmental assessments: using weighted evidence from the literature. In: Freshwater Science. 2012 ; Vol. 31, No. 1. pp. 5-21.
    @article{def3419410794d71ad50e575fb7768c5,
    title = "Analyzing cause and effect in environmental assessments: using weighted evidence from the literature",
    abstract = "Sound decision making in environmental research and management requires an understanding of causal relationships between stressors and ecological responses. However, demonstrating cause{\^a}¿¿effect relationships in natural systems is challenging because of difficulties with natural variability, performing experiments, lack of replication, and the presence of confounding influences. Thus, even the best-designed study may not establish causality. We describe a method that uses evidence available in the extensive published ecological literature to assess support for cause{\^a}¿¿effect hypotheses in environmental investigations. Our method, called Eco Evidence, is a form of causal criteria analysis{\^a}¿¿a technique developed by epidemiologists in the 1960s{\^a}¿¿ who faced similar difficulties in attributing causation. The Eco Evidence method is an 8-step process in which the user conducts a systematic review of the evidence for one or more cause{\^a}¿¿effect hypotheses to assess the level of support for an overall question. In contrast to causal criteria analyses in epidemiology, users of Eco Evidence use a subset of criteriamost relevant to environmental investigations and weight each piece of evidence according to its study design. Stronger studies contribute more to the assessment of causality, but weaker evidence is not discarded. This feature is important because environmental evidence is often scarce. The outputs of the analysis are a guide to the strength of evidence for or against the cause{\^a}¿¿effect hypotheses. They strengthen confidence in the conclusions drawn from that evidence, but cannot ever prove causality. They also indicate situations where knowledge gaps signify insufficient evidence to reach a conclusion. The method is supported by the freely available Eco Evidence software package, which produces a standard report, maximizing the transparency and repeatability of any assessment. Environmental science has lagged behind other disciplines in systematic assessment of evidence to improve research and management. Using the Eco Evidence method, environmental scientists can better use the extensive published literature to guide evidence-based decisions and undertake transparent assessments of ecological cause and effect.",
    keywords = "causal criteria analysis, causality, causal inference, weak inference, Eco Evidence, systematic, review, environmental assessment.",
    author = "Richard Norris and J. Webb and Susan Nichols and Michael Stewardson and Evan Harrison",
    year = "2012",
    doi = "10.1899/11-027.1",
    language = "English",
    volume = "31",
    pages = "5--21",
    journal = "Journal of the North American Benthological Society",
    issn = "2161-9565",
    publisher = "The Society for Freshwater Science",
    number = "1",

    }

    Analyzing cause and effect in environmental assessments: using weighted evidence from the literature. / Norris, Richard; Webb, J.; Nichols, Susan; Stewardson, Michael; Harrison, Evan.

    In: Freshwater Science, Vol. 31, No. 1, 2012, p. 5-21.

    Research output: Contribution to journalArticle

    TY - JOUR

    T1 - Analyzing cause and effect in environmental assessments: using weighted evidence from the literature

    AU - Norris, Richard

    AU - Webb, J.

    AU - Nichols, Susan

    AU - Stewardson, Michael

    AU - Harrison, Evan

    PY - 2012

    Y1 - 2012

    N2 - Sound decision making in environmental research and management requires an understanding of causal relationships between stressors and ecological responses. However, demonstrating causeâ¿¿effect relationships in natural systems is challenging because of difficulties with natural variability, performing experiments, lack of replication, and the presence of confounding influences. Thus, even the best-designed study may not establish causality. We describe a method that uses evidence available in the extensive published ecological literature to assess support for causeâ¿¿effect hypotheses in environmental investigations. Our method, called Eco Evidence, is a form of causal criteria analysisâ¿¿a technique developed by epidemiologists in the 1960sâ¿¿ who faced similar difficulties in attributing causation. The Eco Evidence method is an 8-step process in which the user conducts a systematic review of the evidence for one or more causeâ¿¿effect hypotheses to assess the level of support for an overall question. In contrast to causal criteria analyses in epidemiology, users of Eco Evidence use a subset of criteriamost relevant to environmental investigations and weight each piece of evidence according to its study design. Stronger studies contribute more to the assessment of causality, but weaker evidence is not discarded. This feature is important because environmental evidence is often scarce. The outputs of the analysis are a guide to the strength of evidence for or against the causeâ¿¿effect hypotheses. They strengthen confidence in the conclusions drawn from that evidence, but cannot ever prove causality. They also indicate situations where knowledge gaps signify insufficient evidence to reach a conclusion. The method is supported by the freely available Eco Evidence software package, which produces a standard report, maximizing the transparency and repeatability of any assessment. Environmental science has lagged behind other disciplines in systematic assessment of evidence to improve research and management. Using the Eco Evidence method, environmental scientists can better use the extensive published literature to guide evidence-based decisions and undertake transparent assessments of ecological cause and effect.

    AB - Sound decision making in environmental research and management requires an understanding of causal relationships between stressors and ecological responses. However, demonstrating causeâ¿¿effect relationships in natural systems is challenging because of difficulties with natural variability, performing experiments, lack of replication, and the presence of confounding influences. Thus, even the best-designed study may not establish causality. We describe a method that uses evidence available in the extensive published ecological literature to assess support for causeâ¿¿effect hypotheses in environmental investigations. Our method, called Eco Evidence, is a form of causal criteria analysisâ¿¿a technique developed by epidemiologists in the 1960sâ¿¿ who faced similar difficulties in attributing causation. The Eco Evidence method is an 8-step process in which the user conducts a systematic review of the evidence for one or more causeâ¿¿effect hypotheses to assess the level of support for an overall question. In contrast to causal criteria analyses in epidemiology, users of Eco Evidence use a subset of criteriamost relevant to environmental investigations and weight each piece of evidence according to its study design. Stronger studies contribute more to the assessment of causality, but weaker evidence is not discarded. This feature is important because environmental evidence is often scarce. The outputs of the analysis are a guide to the strength of evidence for or against the causeâ¿¿effect hypotheses. They strengthen confidence in the conclusions drawn from that evidence, but cannot ever prove causality. They also indicate situations where knowledge gaps signify insufficient evidence to reach a conclusion. The method is supported by the freely available Eco Evidence software package, which produces a standard report, maximizing the transparency and repeatability of any assessment. Environmental science has lagged behind other disciplines in systematic assessment of evidence to improve research and management. Using the Eco Evidence method, environmental scientists can better use the extensive published literature to guide evidence-based decisions and undertake transparent assessments of ecological cause and effect.

    KW - causal criteria analysis

    KW - causality

    KW - causal inference

    KW - weak inference

    KW - Eco Evidence

    KW - systematic

    KW - review

    KW - environmental assessment.

    U2 - 10.1899/11-027.1

    DO - 10.1899/11-027.1

    M3 - Article

    VL - 31

    SP - 5

    EP - 21

    JO - Journal of the North American Benthological Society

    JF - Journal of the North American Benthological Society

    SN - 2161-9565

    IS - 1

    ER -