| Title: |
Determining the influence of video-based benchmarking (VBB) on examiner variability in objective structured clinical exams (OSCE): The Align study |
| Authors: |
Yeates, Peter; Edwards, Rebecca Jane; Narain, Aditya; McInley, Robert; Lefroy, Janet; McCray, Gareth; Roberts, Giles; Hammond, Ellie; McBain, Stu; Blythe, Andrew; Cullen, Kathy; Napier, Craig; Sims, Laura; Thampy, Harish; Vince, Tushar; Ensaff, Sue; Goodfellow, Rhian; Harrison, Christopher; Chung, Ching-Wa; Capey, Steven; Roberts, Chris; Vallender, Rebecca |
| Publisher Information: |
Informa UK Limited |
| Publication Year: |
2026 |
| Collection: |
Cardiff University: ORCA (Online Research @ Cardiff) |
| Description: |
IntroductionReducing examiner variability in Objective Structured Clinical Exams (OSCEs) is a priority within clinical performance assessment. In contrast to typical OSCE examiner training, video-based benchmarking (VBB) involves examiners scoring videos a/from their specific station b/shortly before the OSCE and then reflecting on and discussing scores/justifications agreed by an expert panel. Whilst realist evaluation has described mechanisms and contexts by which VBB may operate, VBB’s overall efficacy is unknown.MethodsWe performed a multi-centre (12 UK medical schools) stratified randomised controlled trial of VBB versus control to determine the influence of VBB on examiners’ score variability and other score characteristics. Secondarily, we compared the average scores allocated by examiners from different schools.Results171 medically qualified, trained OSCE examiners participated in the study. VBB showed no significant effect on overall examiner variability. In pre-specified analyses, VBB reduced variability from group mean of initially ‘outlying’ examiners on the borderline performance (VBB mean variability 3.02 out of 27 (IQR1.98-4.98), control 4.70 (3.91–5.70), p < 0.016) and made examiners more likely to correctly fail a minimally failing performance (p < 0.03, OR = 2.133[95% CI 1.081–4.208]). VBB caused a small increase in confidence. There were no significant differences in average scores by school.ConclusionsVBB may enhance trust in OSCEs through more accurate classification of borderline performances and aligning outlying examiners scoring. |
| Document Type: |
article in journal/newspaper |
| File Description: |
application/pdf |
| Language: |
English |
| Relation: |
https://orca.cardiff.ac.uk/id/eprint/185614/1/Determining%20the%20influence%20of%20video-based%20benchmarking%20VBB%20on%20examiner%20variability%20in%20objective%20structured%20clinical%20exams%20OSCE%20The%20Align%20study.pdf; Yeates, Peter, Edwards, Rebecca Jane, Narain, Aditya, McInley, Robert, Lefroy, Janet, McCray, Gareth, Roberts, Giles, Hammond, Ellie, McBain, Stu, Blythe, Andrew, Cullen, Kathy, Napier, Craig, Sims, Laura, Thampy, Harish, Vince, Tushar, Ensaff, Sue https://orca.cardiff.ac.uk/view/cardiffauthors/A0316736.html, Goodfellow, Rhian https://orca.cardiff.ac.uk/view/cardiffauthors/A0160731.html, Harrison, Christopher, Chung, Ching-Wa, Capey, Steven, Roberts, Chris and Vallender, Rebecca https://orca.cardiff.ac.uk/view/cardiffauthors/A903858T.html 2026. Determining the influence of video-based benchmarking (VBB) on examiner variability in objective structured clinical exams (OSCE): The Align study. Medical Teacher 10.1080/0142159x.2026.2631743 https://doi.org/10.1080/0142159x.2026.2631743 file https://orca.cardiff.ac.uk/id/eprint/185614/1/Determining%20the%20influence%20of%20video-based%20benchmarking%20VBB%20on%20examiner%20variability%20in%20objective%20structured%20clinical%20exams%20OSCE%20The%20Align%20study.pdf |
| DOI: |
10.1080/0142159x.2026.2631743 |
| Availability: |
https://orca.cardiff.ac.uk/id/eprint/185614/; https://doi.org/10.1080/0142159x.2026.2631743 |
| Rights: |
cc_by_4_0 |
| Accession Number: |
edsbas.A939FC33 |
| Database: |
BASE |