Reliability of a rubric for performance evaluation in clinical simulation

Safe and effective care for all individuals is an integral component of health service delivery both internationally and within Ireland. Ensuring that adult patients at risk of clinical deterioration are promptly identified... [ view full abstract ]

Safe and effective care for all individuals is an integral component of health service delivery both internationally and within Ireland. Ensuring that adult patients at risk of clinical deterioration are promptly identified and managed according to their clinical need is a priority for HSE (2012). The DARE2 (Detect Deterioration, Accurate Assessment, Rapid Response, Effective Escalation) patient safety rubric was developed to evaluate final year nursing students’ detection of patient deterioration in clinical simulation. This rubric contains four domains of competency; systematic patient assessment, clinical response, clinical-psychomotor skills and communication proficiency.
Aim
The aim of this research was to investigate the inter-rater reliability of data from the DARE2-Patient Safety Rubric.
Method
Archived recorded performances of students (n=34) were independently evaluated by nurse educators (n=4), using the DARE2 –patient safety rubric. Students were individually evaluated over the course of thirty–six minutes as they managed a deteriorating clinical situation.
Results
Percentage agreement for the overall rubric was 59% highlighting the difference between consensus and consistency estimates of inter-rater reliability. Internal consistency was also good achieving a Cronbachs alpha of 0.82. Correlation Coefficients were greater than 0.70 for three of the domains of practice and 0.58 for the fourth (clinical-psychomotor skills). An ICCr of 0.75 for the overall rubric score is indicative of excellent reliability.
Conclusion
The DARE2-Patient Safety Rubric was developed to ensure objectivity and consistency in the evaluation of nursing students’ management of deteriorating patients in the simulated environment. Analysis of data from the DARE2-Patient Safety Rubric indicates excellent consistency estimates and average consensus estimates of inter-rater reliability.
Summary
The DARE2-patient safety rubric was specifically developed to evaluate performance outcomes in the management of deteriorating patients which is integral to National Clinical Guidelines. It is imperative that nurse educators use a reliable instrument to evaluate student performance in assessment of patient deterioration.