Strategies to improve competency-based assessment

Rachel BACON, Kay HOLMES, Claire Palermo

Research output: Contribution to conference (non-published works)Abstract

Abstract

Within the dietetics profession, there are inconsistencies in the assessments made of student performances during practical placements. The aim of this research was to measure variations in assessor's judgements of a student dietitian's performance and to explore the influence of a group discussion on their judgements. The assessments of 26 dietitians of a student's performance, as observed from a video recording of an authentic student–client consultation, were measured pre and post a group discussion using a mixed‐methods questionnaire. This instrument included a validated 7‐point VAS (Visual Analogue Scale) rating scale (1 = novice; 7 = competent), a qualitative global description of performance and an assessor's confidence rating (1 = not at all confident; 10 = extremely confident). No consensus in assessors’ rating was found in either round (Round 1 md = 4, R = 6; Round 2 md = 4, R = 5); however, the discussion influenced the ratings (35%) and/or confidence (58%) of most (78%) participants. Three themes emerged from the thematic analysis of the participants’ global descriptions of performance: (1) discourse supports assessors to justify their judgements, identify assumptions and learn from the observations of others; (2) discourse leads assessors to more holistic judgements; and (3) multiple sources of evidence and student reflections are necessary for credible judgement. This research questions the notion that ‘actual’ performance can be objectively measured, and rather, considers assessment as ‘interpretations’. An ‘interpretive community of assessors’ and multiple sources of evidence support a more credible and defensible approach to competency‐based assessment.
Original languageEnglish
Pages49
Number of pages1
DOIs
Publication statusPublished - 2016

Fingerprint

Students
Nutritionists
Video Recording
Dietetics
Visual Analog Scale
Research
Consensus
Referral and Consultation
Surveys and Questionnaires

Cite this

BACON, Rachel ; HOLMES, Kay ; Palermo, Claire. / Strategies to improve competency-based assessment. 1 p.
@conference{25d8fe950cbd4adc93da631a9e977725,
title = "Strategies to improve competency-based assessment",
abstract = "Within the dietetics profession, there are inconsistencies in the assessments made of student performances during practical placements. The aim of this research was to measure variations in assessor's judgements of a student dietitian's performance and to explore the influence of a group discussion on their judgements. The assessments of 26 dietitians of a student's performance, as observed from a video recording of an authentic student–client consultation, were measured pre and post a group discussion using a mixed‐methods questionnaire. This instrument included a validated 7‐point VAS (Visual Analogue Scale) rating scale (1 = novice; 7 = competent), a qualitative global description of performance and an assessor's confidence rating (1 = not at all confident; 10 = extremely confident). No consensus in assessors’ rating was found in either round (Round 1 md = 4, R = 6; Round 2 md = 4, R = 5); however, the discussion influenced the ratings (35{\%}) and/or confidence (58{\%}) of most (78{\%}) participants. Three themes emerged from the thematic analysis of the participants’ global descriptions of performance: (1) discourse supports assessors to justify their judgements, identify assumptions and learn from the observations of others; (2) discourse leads assessors to more holistic judgements; and (3) multiple sources of evidence and student reflections are necessary for credible judgement. This research questions the notion that ‘actual’ performance can be objectively measured, and rather, considers assessment as ‘interpretations’. An ‘interpretive community of assessors’ and multiple sources of evidence support a more credible and defensible approach to competency‐based assessment.",
author = "Rachel BACON and Kay HOLMES and Claire Palermo",
year = "2016",
doi = "10.1111/1747-0080.12774",
language = "English",
pages = "49",

}

Strategies to improve competency-based assessment. / BACON, Rachel; HOLMES, Kay; Palermo, Claire.

2016. 49.

Research output: Contribution to conference (non-published works)Abstract

TY - CONF

T1 - Strategies to improve competency-based assessment

AU - BACON, Rachel

AU - HOLMES, Kay

AU - Palermo, Claire

PY - 2016

Y1 - 2016

N2 - Within the dietetics profession, there are inconsistencies in the assessments made of student performances during practical placements. The aim of this research was to measure variations in assessor's judgements of a student dietitian's performance and to explore the influence of a group discussion on their judgements. The assessments of 26 dietitians of a student's performance, as observed from a video recording of an authentic student–client consultation, were measured pre and post a group discussion using a mixed‐methods questionnaire. This instrument included a validated 7‐point VAS (Visual Analogue Scale) rating scale (1 = novice; 7 = competent), a qualitative global description of performance and an assessor's confidence rating (1 = not at all confident; 10 = extremely confident). No consensus in assessors’ rating was found in either round (Round 1 md = 4, R = 6; Round 2 md = 4, R = 5); however, the discussion influenced the ratings (35%) and/or confidence (58%) of most (78%) participants. Three themes emerged from the thematic analysis of the participants’ global descriptions of performance: (1) discourse supports assessors to justify their judgements, identify assumptions and learn from the observations of others; (2) discourse leads assessors to more holistic judgements; and (3) multiple sources of evidence and student reflections are necessary for credible judgement. This research questions the notion that ‘actual’ performance can be objectively measured, and rather, considers assessment as ‘interpretations’. An ‘interpretive community of assessors’ and multiple sources of evidence support a more credible and defensible approach to competency‐based assessment.

AB - Within the dietetics profession, there are inconsistencies in the assessments made of student performances during practical placements. The aim of this research was to measure variations in assessor's judgements of a student dietitian's performance and to explore the influence of a group discussion on their judgements. The assessments of 26 dietitians of a student's performance, as observed from a video recording of an authentic student–client consultation, were measured pre and post a group discussion using a mixed‐methods questionnaire. This instrument included a validated 7‐point VAS (Visual Analogue Scale) rating scale (1 = novice; 7 = competent), a qualitative global description of performance and an assessor's confidence rating (1 = not at all confident; 10 = extremely confident). No consensus in assessors’ rating was found in either round (Round 1 md = 4, R = 6; Round 2 md = 4, R = 5); however, the discussion influenced the ratings (35%) and/or confidence (58%) of most (78%) participants. Three themes emerged from the thematic analysis of the participants’ global descriptions of performance: (1) discourse supports assessors to justify their judgements, identify assumptions and learn from the observations of others; (2) discourse leads assessors to more holistic judgements; and (3) multiple sources of evidence and student reflections are necessary for credible judgement. This research questions the notion that ‘actual’ performance can be objectively measured, and rather, considers assessment as ‘interpretations’. An ‘interpretive community of assessors’ and multiple sources of evidence support a more credible and defensible approach to competency‐based assessment.

UR - http://www.mendeley.com/research/concurrent-session-papers-program-order-3

U2 - 10.1111/1747-0080.12774

DO - 10.1111/1747-0080.12774

M3 - Abstract

SP - 49

ER -