Milla Kviatkovsky
Oregon Health and Science University
Dr. Kviatkovsky is a third year resident in Internal Medicine at OHSU in Portland, Oregon. She has an MPH from Maastricht University which she completed in the Netherlands prior to her medical school education. She has interests in ultrasound, and plans to pursue fellowship in either Cardiology or Critical care after some hospitalist time.
Introduction:
The use of bedside ultrasound is becoming increasingly frequent amongst providers across specialties (1-4), including internal medicine (IM) (5-9) where some authors have referred to bedside ultrasound as “the stethoscope of the 21st century” (10). The skills required for appropriate use are now being taught at many IM programs globally, however a universal approach to assessing skills is not yet established. The two main components of bedside ultrasound are image acquisition and image interpretation. We aim to assess the effect of framing effect bias in image acquisition and interpretation of novice ultrasound learners, with hopes to determine how much clinical information impacts accurate assessment of image interpretation skill, and to a more limited extent, acquisition. We hope that this information will be useful in the creation of accurate ultrasound competency assessment in learners.
Methods:
Thirty internal medicine resident interns underwent a 1.5 day ultrasound training course at our urban, academic center residency program. Interns were taught basic ultrasound skills including vascular, cardiac, pulmonary and abdominal exams using both simulation equipment and patient models. These interns were then given a 12-question test using SonoSim®Ultrasound Training equipment, which includes acquiring images on a simulated computer model. After acquiring images interns had recorded their interpretation by answering a short answer question. Half of questions were paired with relevant clinical data, and half were paired with no clinical data at all. One question with clinical data included discordant data, the rest contained concordant data for the demonstrated ultrasound finding. Comparison of performance between questions with clinical data and those without were analyzed. Separate analysis was performed to evaluate affect of discordant data on performance. Statistical analysis was performed using STATA.
Results:
A total of 30 internal medicine interns participated in the study, and each intern answered all twelve questions. The average number of questions answered correctly in the clinical data group (history) was 3.6 vs. 1.9 in the group without clinical data (image only): a statistically significant mean difference of 1.7 questions (p<0.01). When questions were analyzed by group, 58.7% of questions in the history group were answered correctly vs. 30.7 in the image only group with a mean difference of 28% (p=0.09). A single question with clinical data was considered discordant. When this question was included in analysis for the image only group, mean difference between correctly answered questions in the history vs image only group was 41.7% (p<0.01). The lowest proportion of correctly answered questions in the entire group was that with discordant data, of which only 7% of participants answered correctly.
Discussion:
Our study suggests that novice ultrasound learners are more likely to correctly interpret ultrasound images when evaluated within clinical context. In our case, this is demonstrated by availability of clinical history, with further evidence that without this information, appropriate ultrasound use may be limited. This suggests that accurate assessment of isolated ultrasound interpretation skills may be limited if clinical information is available during evaluation, allowing participants to compensate for lack of skill via clinical reasoning. Although knowledge of clinical information has utility in a practice setting, true assessment of raw ultrasound skills may require absence of clinical data.
We also demonstrate the susceptibility of novice learners to framing effect, as is represented by highest frequency of error in the discordant data question. Studies suggest that knowledgeable people are less susceptible to bias (in this study, basic anchoring effects)(11), which speaks to importance of continuous training to improve objectivity of image interpretation and removal of bias.
Although problem based learning is paramount in medical education, when learning a skill and assessing competency in this skill, limiting bias provides most accurate information. Once providers are sufficiently skilled at raw image interpretation, these skills should then be considered appropriate for bedside use. We can not extrapolate if the aforementioned is true for more experienced users, thereby future studies should explore these effects on providers with more training in ultrasound, with hopes that bias is less frequent in these cases.
Use of ultrasound in Graduate Medical and Continuing Education , Point of Care ultrasound in general clinical practice
