Effect of examiner calibration on OSCE pass mark when borderline regression method is used

Research output: Contribution to conferencePoster

158 Downloads (Pure)


The conduct of the OSCE exams in the Division of Dentistry, The University of Manchester underwent major changes in 2014 in the sense that global marking was introduced to allow setting the pass score using borderline regression analysis (BRA). Due to the nature of the changes, training and calibration of the OSCE examiners became paramount; however, due to the time-strain and occupational commitments of the OSCE examiners, an online training and calibration package was developed and launched and the examiners were advised to use it prior to the exam.

Since the latter was a “desirable” exercise, a number of the examiners chose to do the training while some chose not to. This gave us a unique opportunity to have a large dataset of two groups: calibrated (Cal) and non-calibrated (nCal) examiners. What we did not know was that if the exposure to online training had an effect on the pass mark of the stations set by BRA.

A panel of six determined the pass score for each station using the modified Angoff (mAngoff) method. This score was used as a point of reference for the future analyses. The objective and global score data for the Cal (study) and nCal (control) groups were collected. Using BRA, the pass mark for each station was calculated. To take the station difficulty into consideration, the difference between the BRA and mAngoff pass marks were calculated for each examiner.

The difference between the pass marks set by Cal and nCal groups was not statistically different when the BRA was used.

Although training and calibration of OSCE examiners is considered good practice; however, when the BRA method of standard setting is used, we can potentially rely on the professional judgement of the examiners, regardless of their training history.
Original languageEnglish
Publication statusPublished - 23 Aug 2017
EventAssociation of Dental Education in Europe - Vilnius University, Vilnius, Lithuania
Duration: 23 Aug 201725 Aug 2017


ConferenceAssociation of Dental Education in Europe
Abbreviated titleADEE
Internet address


Dive into the research topics of 'Effect of examiner calibration on OSCE pass mark when borderline regression method is used'. Together they form a unique fingerprint.

Cite this