Outputs (24)

A realist evaluation of how, why and when Objective Structured Clinical Exams (OSCEs) are experienced as an authentic assessment of clinical preparedness (2024)
Journal Article

Introduction: Whilst rarely researched, the authenticity with which Objective Structured Clinical Exams (OSCEs) simulate practice is arguably critical to making valid judgements about candidates’ preparedness to progress in their training. We studied... Read More about A realist evaluation of how, why and when Objective Structured Clinical Exams (OSCEs) are experienced as an authentic assessment of clinical preparedness.

Using video-based examiner score comparison and adjustment (VESCA) to compare the influence of examiners at different sites in a distributed objective structured clinical exam (OSCE) (2023)
Journal Article

Purpose: Ensuring equivalence of examiners’ judgements within distributed objective structured clinical exams (OSCEs) is key to both fairness and validity but is hampered by lack of cross-over in the performances which different groups of examiners o... Read More about Using video-based examiner score comparison and adjustment (VESCA) to compare the influence of examiners at different sites in a distributed objective structured clinical exam (OSCE).

VESCA’s variable precision: Determining the accuracy of adjustment for examiner differences in distributed OSCEs (2023)
Other

Enhancing authenticity, diagnosticity and equivalence (AD-Equiv) in multicentre OSCE exams in health professionals education: protocol for a complex intervention study. (2022)
Journal Article

INTRODUCTION: Objective structured clinical exams (OSCEs) are a cornerstone of assessing the competence of trainee healthcare professionals, but have been criticised for (1) lacking authenticity, (2) variability in examiners' judgements which can cha... Read More about Enhancing authenticity, diagnosticity and equivalence (AD-Equiv) in multicentre OSCE exams in health professionals education: protocol for a complex intervention study..

Technology enhanced assessment: Ottawa consensus statement and recommendations. (2022)
Journal Article

INTRODUCTION: In 2011, a consensus report was produced on technology-enhanced assessment (TEA), its good practices, and future perspectives. Since then, technological advances have enabled innovative practices and tools that have revolutionised how l... Read More about Technology enhanced assessment: Ottawa consensus statement and recommendations..

In Reply to Anto et al (2022)
Journal Article

Determining the influence of different linking patterns on the stability of students' score adjustments produced using Video-based Examiner Score Comparison and Adjustment (VESCA). (2022)
Journal Article

BACKGROUND: Ensuring equivalence of examiners' judgements across different groups of examiners is a priority for large scale performance assessments in clinical education, both to enhance fairness and reassure the public. This study extends insight i... Read More about Determining the influence of different linking patterns on the stability of students' score adjustments produced using Video-based Examiner Score Comparison and Adjustment (VESCA)..

Memory, credibility and insight: How video-based feedback promotes deeper reflection and learning in objective structured clinical exams (2022)
Journal Article

INTRODUCTION: Providing high-quality feedback from Objective Structured Clinical Exams (OSCEs) is important but challenging. Whilst prior research suggests that video-based feedback (VbF), where students review their own performances alongside usual... Read More about Memory, credibility and insight: How video-based feedback promotes deeper reflection and learning in objective structured clinical exams.

Determining influence, interaction and causality of Contrast and Sequence effects in OSCEs. (2021)
Journal Article

INTRODUCTION: Differential rater function over time (DRIFT) and contrast effects (examiners' scores biased away from the standard of preceding performances) both challenge the fairness of scoring in Objective Structured Clinical Exams (OSCEs). This i... Read More about Determining influence, interaction and causality of Contrast and Sequence effects in OSCEs..

Understanding patient involvement in judging students' communication skills in OSCEs. (2021)
Journal Article

INTRODUCTION: Communication skills are assessed by medically-enculturated examiners using consensus frameworks which were developed with limited patient involvement. Assessments consequently risk rewarding performance which incompletely serves patien... Read More about Understanding patient involvement in judging students' communication skills in OSCEs..

Measuring the Effect of Examiner Variability in a Multiple-Circuit Objective Structured Clinical Examination (OSCE) (2021)
Journal Article

Purpose Ensuring that examiners in different parallel circuits of objective structured clinical examinations (OSCEs) judge to the same standard is critical to the chain of validity. Recent work suggests examiner-cohort (i.e., the particular group of... Read More about Measuring the Effect of Examiner Variability in a Multiple-Circuit Objective Structured Clinical Examination (OSCE).

The influence of candidates' physical attributes on assessors' ratings in clinical practice. (2021)
Journal Article

BACKGROUND: Assessments of physician competence in the work-place are common and often contribute to high-stakes assessments. Previous research suggests that assessors' judgements can be influenced by candidates' physical attributes. We investigated... Read More about The influence of candidates' physical attributes on assessors' ratings in clinical practice..

Understanding and developing procedures for video-based assessment in medical education (2020)
Journal Article

Introduction Novel uses of video aim to enhance assessment in health-professionals education. Whilst these uses presume equivalence between video and live scoring, some research suggests that poorly understood variations could challenge validity. We... Read More about Understanding and developing procedures for video-based assessment in medical education.

Comparing the influence of ‘describing findings to the examiner’ or ‘examining as in usual practice’ on the students’ performance and assessors’ judgements during physical examination skills assessment (2020)
Journal Article

Background: Within assessment of physical examination skills, two approaches are common: “Describing Findings” (students comment throughout); and examining as “Usual Practice” (students only report findings at the end). Despite numerous potential... Read More about Comparing the influence of ‘describing findings to the examiner’ or ‘examining as in usual practice’ on the students’ performance and assessors’ judgements during physical examination skills assessment.

The mental workload of conducting research in assessor cognition. (2019)
Journal Article

Commentary paper.

Perils, pitfalls and potential for the use of reporting guidelines in experimental research in medical education. (2019)
Journal Article

Commentary.

Exploring differences in individual and group judgements in standard setting (2019)
Journal Article

Background: Standard setting is critically important to assessment decisions in medical education. Recent research has demonstrated variations between medical schools in the standards set for shared items. Despite the centrality of judgement to crite... Read More about Exploring differences in individual and group judgements in standard setting.

Developing a video-based method to compare and adjust examiner effects in fully nested OSCEs (2018)
Journal Article

Background: Whilst averaging across multiple examiners judgements reduces unwanted overall score variability in Objective Structured Clinical Examinations (OSCE), designs involving several parallel circuits of the OSCE require that different examin... Read More about Developing a video-based method to compare and adjust examiner effects in fully nested OSCEs.

Comparatively salient: examining the influence of preceding performances on assessors' focus and interpretations in written assessment comments. (2018)
Journal Article

Recent literature places more emphasis on assessment comments rather than relying solely on scores. Both are variable, however, emanating from assessment judgements. One established source of variability is "contrast effects": scores are shifted away... Read More about Comparatively salient: examining the influence of preceding performances on assessors' focus and interpretations in written assessment comments..

Exploring the Relationship between Examiners’ Memories for performances, domain separation and score variability. (2018)
Journal Article

Background: OSCE examiners' scores are variable and may discriminate domains of performance poorly. Examiners must hold their observations of OSCE performances in "episodic memory" until performances end. We investigated whether examiners vary in the... Read More about Exploring the Relationship between Examiners’ Memories for performances, domain separation and score variability..

A randomised trial of the influence of racial stereotype bias on examiners’ scores, feedback and recollections in undergraduate clinical exams (2017)
Journal Article

Asian medical students and doctors receive lower scores on average than their white counterparts in examinations in the UK and internationally (a phenomenon known as “differential attainment”). This could be due to examiner bias or to social, psychol... Read More about A randomised trial of the influence of racial stereotype bias on examiners’ scores, feedback and recollections in undergraduate clinical exams.

Hawks, Doves and Rasch decisions: Understanding the influence of different cycles of an OSCE on students' scores using Many Facet Rasch Modeling (2016)
Journal Article

INTRODUCTION: OSCEs are commonly conducted in multiple cycles (different circuits, times, and locations), yet the potential for students' allocation to different OSCE cycles is rarely considered as a source of variance-perhaps in part because convent... Read More about Hawks, Doves and Rasch decisions: Understanding the influence of different cycles of an OSCE on students' scores using Many Facet Rasch Modeling.

Relatively speaking: contrast effects influence assessors’ scores and narrative feedback (2015)
Journal Article

Context In prior research, the scores assessors assign can be biased away from the standard of preceding performances (i.e. ‘contrast effects’ occur). Objectives This study examines the mechanism and robustness of these findings to advance under... Read More about Relatively speaking: contrast effects influence assessors’ scores and narrative feedback.

Are Examiners’ Judgments in OSCE-Style Assessments Influenced by Contrast Effects? (2015)
Journal Article

Purpose Laboratory studies have shown that performance assessment judgments can be biased by “contrast effects.” Assessors’ scores become more positive, for example, when the assessed performance is preceded by relatively weak candidates. The autho... Read More about Are Examiners’ Judgments in OSCE-Style Assessments Influenced by Contrast Effects?.