Evaluation of Resident Presentations Using a DEI Checklist

Background: In 2021, the Indiana University School of Medicine Department of Emergency Medicine developed a DEI checklist for residents presenting to their colleagues during weekly didactic sessions. The purpose of this present study is to evaluate the implementation of this DEI checklist as determined by its impact on resident presentations. Methods: A Qualtrics survey based on the DEI checklist was produced to collect data from a Kaltura database containing all resident lectures from the years 2018-19 through 2023-24. Ten lectures were analyzed from each year for 50 total lectures. The year 2020-21 was excluded. Data collected was subjected to quantitative statistics using content analysis. Transcripts for each lecture were generated using Otter.ai and analyzed qualitatively to draw conclusions from how residents discussed DEI topics. Results: Each year was assigned an average overall composite score according to the checklist items, with a score of zero indicating complete adherence to the checklist. The results were as follows: 2018-19: 1.8; 2019-20: 2.3; 2021-22: 2.3; 2022-23: 1.2; 2023-24: 0.8. Residents discussed how their topic may affect someone of a diverse background in 70% of lectures in 2018-2019; 70% in 2019-20; 50% in 2021-22; 80% in 2022-23; and 80% in 2023-24. The percentage of presentations with a dedicated DEI/health equity slide gradually increased from 20% of lectures in 2018-19 to 80% in 2023-24. Transcript review revealed residents transitioned from indirectly discussing health equity in 2018 to directly addressing DEI from 2022-24. Conclusion: Our analysis indicates a general trend towards increasing attention to health equity among these residents. Residents showed more intentionality in approaching DEI topics by directly addressing them in later years. Impact and Implications: Current literature provides no widely agreed upon methodology for determining the impact of DEI training in graduate medical education. Our study provides a novel technique for operationalizing this impact.