Fixing the Rubber Band: Calibration of Communication Skills items in OSCE checklists

Background. Communication skills are commonly measured through ‘communication-items’ in OSCE station-checklists. Our aim is to calibrate the communication component of OSCE station checklists according to the MAAS-Global. ... [ view full abstract ]

Background. Communication skills are commonly measured through ‘communication-items’ in OSCE station-checklists. Our aim is to calibrate the communication component of OSCE station checklists according to the MAAS-Global.

Method. The term ‘calibration' is used to rate how close items in station checklists correspond to an accepted standard. Three raters independently compared checklists of 4 departments with 17 items of the MAAS-Global. G-theory was used to analyse the reliability of the calibration procedure.

Results. G-Kappa was 0.8. For two raters G-Kappa is 0.72 and fell to 0.57 for one rater. 46% of the checklist items correspond to section three of the MAAS-Global (i.e. medical content of the consultation), whilst 12% correspond to section two (i.e. general communication skills), and 8.2% to section one (i.e. communication skills for each separate phase of consultation). 34% of the items were not considered to be communication skills.

Conclusion. This research confirms a reliable and valid procedure for calibrating OSCE CS item checklists using the MAAS-Global. Such calibration will enable comparison of results of CS assessments between and across students. We strongly suggest that such a procedure is more widely employed to arrive at a stable judgment of the communication component of the doctors’ behaviors.