TY - CONF
T1 - Strategies to improve competency-based assessment
AU - BACON, Rachel
AU - HOLMES, Kay
AU - Palermo, Claire
PY - 2016
Y1 - 2016
N2 - Within the dietetics profession, there are inconsistencies in the assessments made of student performances during practical placements. The aim of this research was to measure variations in assessor's judgements of a student dietitian's performance and to explore the influence of a group discussion on their judgements. The assessments of 26 dietitians of a student's performance, as observed from a video recording of an authentic student–client consultation, were measured pre and post a group discussion using a mixed‐methods questionnaire. This instrument included a validated 7‐point VAS (Visual Analogue Scale) rating scale (1 = novice; 7 = competent), a qualitative global description of performance and an assessor's confidence rating (1 = not at all confident; 10 = extremely confident). No consensus in assessors’ rating was found in either round (Round 1 md = 4, R = 6; Round 2 md = 4, R = 5); however, the discussion influenced the ratings (35%) and/or confidence (58%) of most (78%) participants. Three themes emerged from the thematic analysis of the participants’ global descriptions of performance: (1) discourse supports assessors to justify their judgements, identify assumptions and learn from the observations of others; (2) discourse leads assessors to more holistic judgements; and (3) multiple sources of evidence and student reflections are necessary for credible judgement. This research questions the notion that ‘actual’ performance can be objectively measured, and rather, considers assessment as ‘interpretations’. An ‘interpretive community of assessors’ and multiple sources of evidence support a more credible and defensible approach to competency‐based assessment.
AB - Within the dietetics profession, there are inconsistencies in the assessments made of student performances during practical placements. The aim of this research was to measure variations in assessor's judgements of a student dietitian's performance and to explore the influence of a group discussion on their judgements. The assessments of 26 dietitians of a student's performance, as observed from a video recording of an authentic student–client consultation, were measured pre and post a group discussion using a mixed‐methods questionnaire. This instrument included a validated 7‐point VAS (Visual Analogue Scale) rating scale (1 = novice; 7 = competent), a qualitative global description of performance and an assessor's confidence rating (1 = not at all confident; 10 = extremely confident). No consensus in assessors’ rating was found in either round (Round 1 md = 4, R = 6; Round 2 md = 4, R = 5); however, the discussion influenced the ratings (35%) and/or confidence (58%) of most (78%) participants. Three themes emerged from the thematic analysis of the participants’ global descriptions of performance: (1) discourse supports assessors to justify their judgements, identify assumptions and learn from the observations of others; (2) discourse leads assessors to more holistic judgements; and (3) multiple sources of evidence and student reflections are necessary for credible judgement. This research questions the notion that ‘actual’ performance can be objectively measured, and rather, considers assessment as ‘interpretations’. An ‘interpretive community of assessors’ and multiple sources of evidence support a more credible and defensible approach to competency‐based assessment.
UR - http://www.mendeley.com/research/concurrent-session-papers-program-order-3
U2 - 10.1111/1747-0080.12774
DO - 10.1111/1747-0080.12774
M3 - Abstract
SP - 49
ER -