Objective: With increasing medical student graduation rates, there is a clear need for proper evaluation of the provided resident education. Research currently presents a range of evaluation methods, from faculty... [ view full abstract ]
Objective: With increasing medical student graduation rates, there is a clear need for proper evaluation of the provided resident education. Research currently presents a range of evaluation methods, from faculty self-assessment to lengthy standardized surveys. The present study aims to examine a mixed method assessment of clinical education performance among attending physicians at a large academic medical center.
Methods: Data assessing 67 physicians across 12 surgical divisions was compiled from four separate evaluation tools between the Spring 2015 and Spring 2016 residency semesters. The tools included blinded resident ratings, resident group evaluations, resident two-word evaluations, and the surgical division directors’ ratings; with all measuring the attending as an educator. Group, two-word, and division director results were then correlated with the standardized blinded residents’ ratings. Results’ level of agreement was measured using Cohen’s kappa.
Results: All of the supplementary methods showed significant positive correlations with the blinded resident ratings; resident group (r=0.71, p=<0.0001), resident two-word (r=0.41, p=0.002), and division directors (r=0.42, p=0.005). Little agreement was observed between blinded ratings and group ratings (K=-0.003, p=0.717). Significant agreement was found between the director’s ratings and group ratings (K=-0.155, p=0.006) and positive word rates (K=0.107, p=0.017).
Conclusion: Results of this analysis show that with some variation in agreement all survey methods present as valid tools for assessment. Results do not support the idea of a single gold-standard method, like blinded resident evaluations. Instead results suggest the benefit of this mixed methods approach, creating a composite perspective of an individual’s capabilities as an educator.
