Skip to main content

Research Repository

Advanced Search

Enhancing authenticity, diagnosticity and equivalence (AD-Equiv) in multicentre OSCE exams in health professionals education: protocol for a complex intervention study. (2022)
Journal Article
Yeates, P., Maluf, A., Kinston, R., Cope, N., McCray, G., Cullen, K., …Wong, G. (2022). Enhancing authenticity, diagnosticity and equivalence (AD-Equiv) in multicentre OSCE exams in health professionals education: protocol for a complex intervention study. BMJ Open, 12(12), Article e064387. https://doi.org/10.1136/bmjopen-2022-064387

INTRODUCTION: Objective structured clinical exams (OSCEs) are a cornerstone of assessing the competence of trainee healthcare professionals, but have been criticised for (1) lacking authenticity, (2) variability in examiners' judgements which can cha... Read More about Enhancing authenticity, diagnosticity and equivalence (AD-Equiv) in multicentre OSCE exams in health professionals education: protocol for a complex intervention study..

Technology enhanced assessment: Ottawa consensus statement and recommendations. (2022)
Journal Article
Fuller, R., Goddard, V. C., Nadarajah, V. D., Treasure-Jones, T., Yeates, P., Scott, K., …Pyorala, E. (2022). Technology enhanced assessment: Ottawa consensus statement and recommendations. Medical Teacher, 1 - 15. https://doi.org/10.1080/0142159X.2022.2083489

INTRODUCTION: In 2011, a consensus report was produced on technology-enhanced assessment (TEA), its good practices, and future perspectives. Since then, technological advances have enabled innovative practices and tools that have revolutionised how l... Read More about Technology enhanced assessment: Ottawa consensus statement and recommendations..

Determining influence, interaction and causality of Contrast and Sequence effects in OSCEs. (2021)
Journal Article
Yeates, P., Moult, A., Cope, N., McCray, G., Fuller, R., & McKinley, R. (2022). Determining influence, interaction and causality of Contrast and Sequence effects in OSCEs. Medical Education, 56(3), 292-302. https://doi.org/10.1111/medu.14713

INTRODUCTION: Differential rater function over time (DRIFT) and contrast effects (examiners' scores biased away from the standard of preceding performances) both challenge the fairness of scoring in Objective Structured Clinical Exams (OSCEs). This i... Read More about Determining influence, interaction and causality of Contrast and Sequence effects in OSCEs..

Measuring the Effect of Examiner Variability in a Multiple-Circuit Objective Structured Clinical Examination (OSCE) (2021)
Journal Article
Yeates, P., Moult, A., Cope, N., McCray, G., Xilas, E., Lovelock, T., …McKinley, R. K. (. (2021). Measuring the Effect of Examiner Variability in a Multiple-Circuit Objective Structured Clinical Examination (OSCE). Academic Medicine, 96(8), 1189-1196. https://doi.org/10.1097/acm.0000000000004028

Purpose Ensuring that examiners in different parallel circuits of objective structured clinical examinations (OSCEs) judge to the same standard is critical to the chain of validity. Recent work suggests examiner-cohort (i.e., the particular group of... Read More about Measuring the Effect of Examiner Variability in a Multiple-Circuit Objective Structured Clinical Examination (OSCE).

Understanding and developing procedures for video-based assessment in medical education (2020)
Journal Article
Yeates, P., Moult, A., Lefroy, J., Walsh-House, J., Clews, L., McKinley, R., & Fuller, R. (2020). Understanding and developing procedures for video-based assessment in medical education. Medical Teacher, 42(11), https://doi.org/10.1080/0142159X.2020.1801997

Introduction
Novel uses of video aim to enhance assessment in health-professionals education. Whilst these uses presume equivalence between video and live scoring, some research suggests that poorly understood variations could challenge validity. We... Read More about Understanding and developing procedures for video-based assessment in medical education.

Comparing the influence of ‘describing findings to the examiner’ or ‘examining as in usual practice’ on the students’ performance and assessors’ judgements during physical examination skills assessment (2020)
Journal Article
Stephenson, C., Yeates, P., & Lefroy, J. (2020). Comparing the influence of ‘describing findings to the examiner’ or ‘examining as in usual practice’ on the students’ performance and assessors’ judgements during physical examination skills assessment. MedEdPublish, 9(18), https://doi.org/10.15694/mep.2020.000018.1

Background:

Within assessment of physical examination skills, two approaches are common: “Describing Findings” (students comment throughout); and examining as “Usual Practice” (students only report findings at the end). Despite numerous potential... Read More about Comparing the influence of ‘describing findings to the examiner’ or ‘examining as in usual practice’ on the students’ performance and assessors’ judgements during physical examination skills assessment.

Exploring differences in individual and group judgements in standard setting (2019)
Journal Article
Yeates, P., Cope, N., Luksaite, E., Hassell, A., & Dikomitis, L. (2019). Exploring differences in individual and group judgements in standard setting. Medical Education, https://doi.org/10.1111/medu.13915

Background: Standard setting is critically important to assessment decisions in medical education. Recent research has demonstrated variations between medical schools in the standards set for shared items. Despite the centrality of judgement to crite... Read More about Exploring differences in individual and group judgements in standard setting.

Developing a video-based method to compare and adjust examiner effects in fully nested OSCEs (2018)
Journal Article
Yeates, P., Cope, N., Hawarden, A., Bradshaw, H., McCray, G., & Homer, M. (2019). Developing a video-based method to compare and adjust examiner effects in fully nested OSCEs. Medical Education, 53(3), 250-263. https://doi.org/10.1111/medu.13783

Background:
Whilst averaging across multiple examiners judgements reduces unwanted overall score variability in Objective Structured Clinical Examinations (OSCE), designs involving several parallel circuits of the OSCE require that different examin... Read More about Developing a video-based method to compare and adjust examiner effects in fully nested OSCEs.

Comparatively salient: examining the influence of preceding performances on assessors' focus and interpretations in written assessment comments. (2018)
Journal Article
Yeates. (2018). Comparatively salient: examining the influence of preceding performances on assessors' focus and interpretations in written assessment comments. Advances in Health Sciences Education, 937-959. https://doi.org/10.1007/s10459-018-9841-2

Recent literature places more emphasis on assessment comments rather than relying solely on scores. Both are variable, however, emanating from assessment judgements. One established source of variability is "contrast effects": scores are shifted away... Read More about Comparatively salient: examining the influence of preceding performances on assessors' focus and interpretations in written assessment comments..

Exploring the Relationship between Examiners’ Memories for performances, domain separation and score variability. (2018)
Journal Article
Yeates, & McCray. (2018). Exploring the Relationship between Examiners’ Memories for performances, domain separation and score variability. Medical Teacher, 1159-1165. https://doi.org/10.1080/0142159X.2018.1463088

Background: OSCE examiners' scores are variable and may discriminate domains of performance poorly. Examiners must hold their observations of OSCE performances in "episodic memory" until performances end. We investigated whether examiners vary in the... Read More about Exploring the Relationship between Examiners’ Memories for performances, domain separation and score variability..

A randomised trial of the influence of racial stereotype bias on examiners’ scores, feedback and recollections in undergraduate clinical exams (2017)
Journal Article
Yeates. (2017). A randomised trial of the influence of racial stereotype bias on examiners’ scores, feedback and recollections in undergraduate clinical exams. BMC Medicine, 179 -?. https://doi.org/10.1186/s12916-017-0943-0

Asian medical students and doctors receive lower scores on average than their white counterparts in examinations in the UK and internationally (a phenomenon known as “differential attainment”). This could be due to examiner bias or to social, psychol... Read More about A randomised trial of the influence of racial stereotype bias on examiners’ scores, feedback and recollections in undergraduate clinical exams.