Assessing community readiness online: A concurrent validation study

Iordan Kostadinov, Mark DANIEL, Linda Stanley, Margaret CARGO

Research output: Contribution to journalArticle

3 Citations (Scopus)
2 Downloads (Pure)

Abstract

Background:Community readiness for facilitation and uptake of interventions can impact the success of community-based prevention efforts. As currently practiced, measuring community readiness can be a resource intensive process, compromising its use in evaluating multisite community-based prevention efforts. The purpose of this study was to develop, test and validate a more efficient online version of an existing community readiness tool and identify potential problems in completing assessments. This study was conducted in the context of a complex community-based childhood obesity prevention program in South Australia.Methods:Following pre-testing, an online version of the community readiness tool was created, wherein respondents, with detailed knowledge of their community and prevention efforts, rated their communities on five anchored rating scales (Knowledge of Efforts, Leadership, Knowledge of the Issue, Community Climate, and Resources). Respondents completed the standard, over-the-phone community readiness interview (“gold standard”) and the new online survey. Paired t-test, St. Laurent’s correlation coefficient and intra-class correlation (ICC) were used to determine the validity of the online tool. Contact summary forms were completed after each interview to capture interview quality.Results: Twenty-five respondents completed both assessments. There was a statistically significant difference in the overall community readiness scores between the two methods (paired t-test p = 0.03); online scores were consistently higher than interview scores. St. Laurent’s correlation of 0.58 (95 % CI 0.42–0.73) was moderate; the ICC of 0.65 (95 % CI 0.35–0.83) was good. Only for the leadership and resources dimensions was there no statistically significant difference between the scores from the two methods (p = 0.61, p = 0.08 respectively). St Laurent’s correlation (r = 0.83, 95 % CI 0.71–0.92) and the ICC (0.78, 95 % CI 0.57–0.90) were excellent for leadership. Qualitative results from the standard interview method suggest that some respondents felt reluctant to answer questions on behalf of other community members. This may have impacted their self-selected ratings and/or responses to questions during the interview. Conclusions: Concurrent validity for the online method was supported for the Leadership dimension only. However, the online method holds promise as it reduces time and resource burden, allowing for a quicker return of results to the community to inform program planning, implementation and evaluations to improve community health
Original languageEnglish
Pages (from-to)1-6
Number of pages6
JournalBMC Public Health
Volume15
Issue number1
DOIs
Publication statusPublished - 2015
Externally publishedYes

Fingerprint

Validation Studies
Cohort Studies
Interviews
South Australia
Pediatric Obesity
Climate

Cite this

Kostadinov, Iordan ; DANIEL, Mark ; Stanley, Linda ; CARGO, Margaret. / Assessing community readiness online: A concurrent validation study. In: BMC Public Health. 2015 ; Vol. 15, No. 1. pp. 1-6.
@article{46add351a9aa487289fab2ece21e3eb7,
title = "Assessing community readiness online: A concurrent validation study",
abstract = "Background:Community readiness for facilitation and uptake of interventions can impact the success of community-based prevention efforts. As currently practiced, measuring community readiness can be a resource intensive process, compromising its use in evaluating multisite community-based prevention efforts. The purpose of this study was to develop, test and validate a more efficient online version of an existing community readiness tool and identify potential problems in completing assessments. This study was conducted in the context of a complex community-based childhood obesity prevention program in South Australia.Methods:Following pre-testing, an online version of the community readiness tool was created, wherein respondents, with detailed knowledge of their community and prevention efforts, rated their communities on five anchored rating scales (Knowledge of Efforts, Leadership, Knowledge of the Issue, Community Climate, and Resources). Respondents completed the standard, over-the-phone community readiness interview (“gold standard”) and the new online survey. Paired t-test, St. Laurent’s correlation coefficient and intra-class correlation (ICC) were used to determine the validity of the online tool. Contact summary forms were completed after each interview to capture interview quality.Results: Twenty-five respondents completed both assessments. There was a statistically significant difference in the overall community readiness scores between the two methods (paired t-test p = 0.03); online scores were consistently higher than interview scores. St. Laurent’s correlation of 0.58 (95 {\%} CI 0.42–0.73) was moderate; the ICC of 0.65 (95 {\%} CI 0.35–0.83) was good. Only for the leadership and resources dimensions was there no statistically significant difference between the scores from the two methods (p = 0.61, p = 0.08 respectively). St Laurent’s correlation (r = 0.83, 95 {\%} CI 0.71–0.92) and the ICC (0.78, 95 {\%} CI 0.57–0.90) were excellent for leadership. Qualitative results from the standard interview method suggest that some respondents felt reluctant to answer questions on behalf of other community members. This may have impacted their self-selected ratings and/or responses to questions during the interview. Conclusions: Concurrent validity for the online method was supported for the Leadership dimension only. However, the online method holds promise as it reduces time and resource burden, allowing for a quicker return of results to the community to inform program planning, implementation and evaluations to improve community health",
author = "Iordan Kostadinov and Mark DANIEL and Linda Stanley and Margaret CARGO",
year = "2015",
doi = "10.1186/s12889-015-1953-5",
language = "English",
volume = "15",
pages = "1--6",
journal = "BMC Public Health",
issn = "1471-2458",
publisher = "BioMed Central",
number = "1",

}

Assessing community readiness online: A concurrent validation study. / Kostadinov, Iordan; DANIEL, Mark; Stanley, Linda; CARGO, Margaret.

In: BMC Public Health, Vol. 15, No. 1, 2015, p. 1-6.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Assessing community readiness online: A concurrent validation study

AU - Kostadinov, Iordan

AU - DANIEL, Mark

AU - Stanley, Linda

AU - CARGO, Margaret

PY - 2015

Y1 - 2015

N2 - Background:Community readiness for facilitation and uptake of interventions can impact the success of community-based prevention efforts. As currently practiced, measuring community readiness can be a resource intensive process, compromising its use in evaluating multisite community-based prevention efforts. The purpose of this study was to develop, test and validate a more efficient online version of an existing community readiness tool and identify potential problems in completing assessments. This study was conducted in the context of a complex community-based childhood obesity prevention program in South Australia.Methods:Following pre-testing, an online version of the community readiness tool was created, wherein respondents, with detailed knowledge of their community and prevention efforts, rated their communities on five anchored rating scales (Knowledge of Efforts, Leadership, Knowledge of the Issue, Community Climate, and Resources). Respondents completed the standard, over-the-phone community readiness interview (“gold standard”) and the new online survey. Paired t-test, St. Laurent’s correlation coefficient and intra-class correlation (ICC) were used to determine the validity of the online tool. Contact summary forms were completed after each interview to capture interview quality.Results: Twenty-five respondents completed both assessments. There was a statistically significant difference in the overall community readiness scores between the two methods (paired t-test p = 0.03); online scores were consistently higher than interview scores. St. Laurent’s correlation of 0.58 (95 % CI 0.42–0.73) was moderate; the ICC of 0.65 (95 % CI 0.35–0.83) was good. Only for the leadership and resources dimensions was there no statistically significant difference between the scores from the two methods (p = 0.61, p = 0.08 respectively). St Laurent’s correlation (r = 0.83, 95 % CI 0.71–0.92) and the ICC (0.78, 95 % CI 0.57–0.90) were excellent for leadership. Qualitative results from the standard interview method suggest that some respondents felt reluctant to answer questions on behalf of other community members. This may have impacted their self-selected ratings and/or responses to questions during the interview. Conclusions: Concurrent validity for the online method was supported for the Leadership dimension only. However, the online method holds promise as it reduces time and resource burden, allowing for a quicker return of results to the community to inform program planning, implementation and evaluations to improve community health

AB - Background:Community readiness for facilitation and uptake of interventions can impact the success of community-based prevention efforts. As currently practiced, measuring community readiness can be a resource intensive process, compromising its use in evaluating multisite community-based prevention efforts. The purpose of this study was to develop, test and validate a more efficient online version of an existing community readiness tool and identify potential problems in completing assessments. This study was conducted in the context of a complex community-based childhood obesity prevention program in South Australia.Methods:Following pre-testing, an online version of the community readiness tool was created, wherein respondents, with detailed knowledge of their community and prevention efforts, rated their communities on five anchored rating scales (Knowledge of Efforts, Leadership, Knowledge of the Issue, Community Climate, and Resources). Respondents completed the standard, over-the-phone community readiness interview (“gold standard”) and the new online survey. Paired t-test, St. Laurent’s correlation coefficient and intra-class correlation (ICC) were used to determine the validity of the online tool. Contact summary forms were completed after each interview to capture interview quality.Results: Twenty-five respondents completed both assessments. There was a statistically significant difference in the overall community readiness scores between the two methods (paired t-test p = 0.03); online scores were consistently higher than interview scores. St. Laurent’s correlation of 0.58 (95 % CI 0.42–0.73) was moderate; the ICC of 0.65 (95 % CI 0.35–0.83) was good. Only for the leadership and resources dimensions was there no statistically significant difference between the scores from the two methods (p = 0.61, p = 0.08 respectively). St Laurent’s correlation (r = 0.83, 95 % CI 0.71–0.92) and the ICC (0.78, 95 % CI 0.57–0.90) were excellent for leadership. Qualitative results from the standard interview method suggest that some respondents felt reluctant to answer questions on behalf of other community members. This may have impacted their self-selected ratings and/or responses to questions during the interview. Conclusions: Concurrent validity for the online method was supported for the Leadership dimension only. However, the online method holds promise as it reduces time and resource burden, allowing for a quicker return of results to the community to inform program planning, implementation and evaluations to improve community health

U2 - 10.1186/s12889-015-1953-5

DO - 10.1186/s12889-015-1953-5

M3 - Article

VL - 15

SP - 1

EP - 6

JO - BMC Public Health

JF - BMC Public Health

SN - 1471-2458

IS - 1

ER -