Skip to main content

Research Repository

Advanced Search

All Outputs (27)

Inter-school variations in the standard of examiners’ graduation-level OSCE judgements (2024)
Journal Article
Yeates, P., Maluf, A., McCray, G., Kinston, R., Cope, N., Cullen, K., …McKinley, R. (2024). Inter-school variations in the standard of examiners’ graduation-level OSCE judgements. Medical Teacher, 1-9. https://doi.org/10.1080/0142159X.2024.2372087

Introduction
Ensuring equivalence in high-stakes performance exams is important for patient safety and candidate fairness. We compared inter-school examiner differences within a shared OSCE and resulting impact on students’ pass/fail categorisation.... Read More about Inter-school variations in the standard of examiners’ graduation-level OSCE judgements.

A realist evaluation of how, why and when Objective Structured Clinical Exams (OSCEs) are experienced as an authentic assessment of clinical preparedness (2024)
Journal Article
Yeates, P., Maluf, A., Kinston, R., Cope, N., Cullen, K., Cole, A., …Wong, G. (2024). A realist evaluation of how, why and when Objective Structured Clinical Exams (OSCEs) are experienced as an authentic assessment of clinical preparedness. Medical Teacher, 1-9. https://doi.org/10.1080/0142159X.2024.2339413

Introduction: Whilst rarely researched, the authenticity with which Objective Structured Clinical Exams (OSCEs) simulate practice is arguably critical to making valid judgements about candidates’ preparedness to progress in their training. We studied... Read More about A realist evaluation of how, why and when Objective Structured Clinical Exams (OSCEs) are experienced as an authentic assessment of clinical preparedness.

Using video-based examiner score comparison and adjustment (VESCA) to compare the influence of examiners at different sites in a distributed objective structured clinical exam (OSCE) (2023)
Journal Article
Yeates, P., Maluf, A., Cope, N., McCray, G., McBain, S., Beardow, D., …McKinley, R. B. (2023). Using video-based examiner score comparison and adjustment (VESCA) to compare the influence of examiners at different sites in a distributed objective structured clinical exam (OSCE). BMC Medical Education, 23(1), Article 803. https://doi.org/10.1186/s12909-023-04774-4

Purpose: Ensuring equivalence of examiners’ judgements within distributed objective structured clinical exams (OSCEs) is key to both fairness and validity but is hampered by lack of cross-over in the performances which different groups of examiners o... Read More about Using video-based examiner score comparison and adjustment (VESCA) to compare the influence of examiners at different sites in a distributed objective structured clinical exam (OSCE).

Enhancing authenticity, diagnosticity and equivalence (AD-Equiv) in multicentre OSCE exams in health professionals education: protocol for a complex intervention study. (2022)
Journal Article
Yeates, P., Maluf, A., Kinston, R., Cope, N., McCray, G., Cullen, K., …Wong, G. (2022). Enhancing authenticity, diagnosticity and equivalence (AD-Equiv) in multicentre OSCE exams in health professionals education: protocol for a complex intervention study. BMJ Open, 12(12), Article e064387. https://doi.org/10.1136/bmjopen-2022-064387

INTRODUCTION: Objective structured clinical exams (OSCEs) are a cornerstone of assessing the competence of trainee healthcare professionals, but have been criticised for (1) lacking authenticity, (2) variability in examiners' judgements which can cha... Read More about Enhancing authenticity, diagnosticity and equivalence (AD-Equiv) in multicentre OSCE exams in health professionals education: protocol for a complex intervention study..

Technology enhanced assessment: Ottawa consensus statement and recommendations. (2022)
Journal Article
Fuller, R., Goddard, V. C., Nadarajah, V. D., Treasure-Jones, T., Yeates, P., Scott, K., …Pyorala, E. (2022). Technology enhanced assessment: Ottawa consensus statement and recommendations. Medical Teacher, 1 - 15. https://doi.org/10.1080/0142159X.2022.2083489

INTRODUCTION: In 2011, a consensus report was produced on technology-enhanced assessment (TEA), its good practices, and future perspectives. Since then, technological advances have enabled innovative practices and tools that have revolutionised how l... Read More about Technology enhanced assessment: Ottawa consensus statement and recommendations..

Determining the influence of different linking patterns on the stability of students' score adjustments produced using Video-based Examiner Score Comparison and Adjustment (VESCA). (2022)
Journal Article
Yeates, P., McCray, G., Moult, A., Cope, N., Fuller, R., & McKinley, R. (2022). Determining the influence of different linking patterns on the stability of students' score adjustments produced using Video-based Examiner Score Comparison and Adjustment (VESCA). BMC Medical Education, 22, Article 41. https://doi.org/10.1186/s12909-022-03115-1

BACKGROUND: Ensuring equivalence of examiners' judgements across different groups of examiners is a priority for large scale performance assessments in clinical education, both to enhance fairness and reassure the public. This study extends insight i... Read More about Determining the influence of different linking patterns on the stability of students' score adjustments produced using Video-based Examiner Score Comparison and Adjustment (VESCA)..

Memory, credibility and insight: How video-based feedback promotes deeper reflection and learning in objective structured clinical exams (2022)
Journal Article
Makrides, A., & Yeates, P. (2022). Memory, credibility and insight: How video-based feedback promotes deeper reflection and learning in objective structured clinical exams. Medical Teacher, 44(6), 664-671. https://doi.org/10.1080/0142159x.2021.2020232

INTRODUCTION: Providing high-quality feedback from Objective Structured Clinical Exams (OSCEs) is important but challenging. Whilst prior research suggests that video-based feedback (VbF), where students review their own performances alongside usual... Read More about Memory, credibility and insight: How video-based feedback promotes deeper reflection and learning in objective structured clinical exams.

Determining influence, interaction and causality of Contrast and Sequence effects in OSCEs. (2021)
Journal Article
Yeates, P., Moult, A., Cope, N., McCray, G., Fuller, R., & McKinley, R. (2022). Determining influence, interaction and causality of Contrast and Sequence effects in OSCEs. Medical Education, 56(3), 292-302. https://doi.org/10.1111/medu.14713

INTRODUCTION: Differential rater function over time (DRIFT) and contrast effects (examiners' scores biased away from the standard of preceding performances) both challenge the fairness of scoring in Objective Structured Clinical Exams (OSCEs). This i... Read More about Determining influence, interaction and causality of Contrast and Sequence effects in OSCEs..

Understanding patient involvement in judging students' communication skills in OSCEs (2021)
Journal Article
Moult, A., McKinley, R. K., & Yeates, P. (2021). Understanding patient involvement in judging students' communication skills in OSCEs. Medical Teacher, 43(9), 1070 - 1078. https://doi.org/10.1080/0142159X.2021.1915467

INTRODUCTION: Communication skills are assessed by medically-enculturated examiners using consensus frameworks which were developed with limited patient involvement. Assessments consequently risk rewarding performance which incompletely serves patien... Read More about Understanding patient involvement in judging students' communication skills in OSCEs.

Measuring the Effect of Examiner Variability in a Multiple-Circuit Objective Structured Clinical Examination (OSCE) (2021)
Journal Article
Yeates, P., Moult, A., Cope, N., McCray, G., Xilas, E., Lovelock, T., …McKinley, R. K. (. (2021). Measuring the Effect of Examiner Variability in a Multiple-Circuit Objective Structured Clinical Examination (OSCE). Academic Medicine, 96(8), 1189-1196. https://doi.org/10.1097/acm.0000000000004028

Purpose Ensuring that examiners in different parallel circuits of objective structured clinical examinations (OSCEs) judge to the same standard is critical to the chain of validity. Recent work suggests examiner-cohort (i.e., the particular group of... Read More about Measuring the Effect of Examiner Variability in a Multiple-Circuit Objective Structured Clinical Examination (OSCE).

The influence of candidates' physical attributes on assessors' ratings in clinical practice (2021)
Journal Article
Sam, A., Reid, M., Thakerar, V., Gurnell, M., Westacott, R., Yeates, P., …Brown, C. (2021). The influence of candidates' physical attributes on assessors' ratings in clinical practice. Medical Teacher, 43(5), 554-559. https://doi.org/10.1080/0142159X.2021.1877268

BACKGROUND: Assessments of physician competence in the work-place are common and often contribute to high-stakes assessments. Previous research suggests that assessors' judgements can be influenced by candidates' physical attributes. We investigated... Read More about The influence of candidates' physical attributes on assessors' ratings in clinical practice.

Understanding and developing procedures for video-based assessment in medical education (2020)
Journal Article
Yeates, P., Moult, A., Lefroy, J., Walsh-House, J., Clews, L., McKinley, R., & Fuller, R. (2020). Understanding and developing procedures for video-based assessment in medical education. Medical Teacher, 42(11), https://doi.org/10.1080/0142159X.2020.1801997

Introduction
Novel uses of video aim to enhance assessment in health-professionals education. Whilst these uses presume equivalence between video and live scoring, some research suggests that poorly understood variations could challenge validity. We... Read More about Understanding and developing procedures for video-based assessment in medical education.

Comparing the influence of ‘describing findings to the examiner’ or ‘examining as in usual practice’ on the students’ performance and assessors’ judgements during physical examination skills assessment (2020)
Journal Article
Stephenson, C., Yeates, P., & Lefroy, J. (2020). Comparing the influence of ‘describing findings to the examiner’ or ‘examining as in usual practice’ on the students’ performance and assessors’ judgements during physical examination skills assessment. MedEdPublish, 9(18), https://doi.org/10.15694/mep.2020.000018.1

Background:

Within assessment of physical examination skills, two approaches are common: “Describing Findings” (students comment throughout); and examining as “Usual Practice” (students only report findings at the end). Despite numerous potential... Read More about Comparing the influence of ‘describing findings to the examiner’ or ‘examining as in usual practice’ on the students’ performance and assessors’ judgements during physical examination skills assessment.

Exploring differences in individual and group judgements in standard setting (2019)
Journal Article
Yeates, P., Cope, N., Luksaite, E., Hassell, A., & Dikomitis, L. (2019). Exploring differences in individual and group judgements in standard setting. Medical Education, https://doi.org/10.1111/medu.13915

Background: Standard setting is critically important to assessment decisions in medical education. Recent research has demonstrated variations between medical schools in the standards set for shared items. Despite the centrality of judgement to crite... Read More about Exploring differences in individual and group judgements in standard setting.

Developing a video-based method to compare and adjust examiner effects in fully nested OSCEs (2018)
Journal Article
Yeates, P., Cope, N., Hawarden, A., Bradshaw, H., McCray, G., & Homer, M. (2019). Developing a video-based method to compare and adjust examiner effects in fully nested OSCEs. Medical Education, 53(3), 250-263. https://doi.org/10.1111/medu.13783

Background:
Whilst averaging across multiple examiners judgements reduces unwanted overall score variability in Objective Structured Clinical Examinations (OSCE), designs involving several parallel circuits of the OSCE require that different examin... Read More about Developing a video-based method to compare and adjust examiner effects in fully nested OSCEs.

Comparatively salient: examining the influence of preceding performances on assessors' focus and interpretations in written assessment comments. (2018)
Journal Article
Yeates. (2018). Comparatively salient: examining the influence of preceding performances on assessors' focus and interpretations in written assessment comments. Advances in Health Sciences Education, 937-959. https://doi.org/10.1007/s10459-018-9841-2

Recent literature places more emphasis on assessment comments rather than relying solely on scores. Both are variable, however, emanating from assessment judgements. One established source of variability is "contrast effects": scores are shifted away... Read More about Comparatively salient: examining the influence of preceding performances on assessors' focus and interpretations in written assessment comments..

Exploring the Relationship between Examiners’ Memories for performances, domain separation and score variability. (2018)
Journal Article
Yeates, & McCray. (2018). Exploring the Relationship between Examiners’ Memories for performances, domain separation and score variability. Medical Teacher, 1159-1165. https://doi.org/10.1080/0142159X.2018.1463088

Background: OSCE examiners' scores are variable and may discriminate domains of performance poorly. Examiners must hold their observations of OSCE performances in "episodic memory" until performances end. We investigated whether examiners vary in the... Read More about Exploring the Relationship between Examiners’ Memories for performances, domain separation and score variability..