Automating medical student case logging: An innovative method to capture required clinical experiences

Background:  In U.S. clinical clerkships, medical students must manually log patient care encounters, known as required clinical experiences (RCEs), to demonstrate competency in specific skills and meet institutional standards. The potential downsides of the current manual process include student administrative burden, reporting inaccuracy, delays in data entry, and decreased incentives to log after reaching requisite minimums. Automation of RCE capture using electronic health record (EHR) data could address these challenges. Summary of Work:  We sought to determine the feasibility of using EHR data to automate RCE capture for medical students on an inpatient medicine rotation. We compared diagnoses captured via EHR data to those logged manually by students over a fixed timeframe. EHR diagnoses (ICD–10 codes) were mapped to the relevant RCEs using Systematized Nomenclature of Medicine Clinical Terms concepts. To assess accuracy, we queried students weekly to verify patient list accuracy, confirm top diagnoses, and identify missing patients or encounters. Summary of Results:  During the 16-month study period, 2,276 diagnoses (119 per month) were manually logged by students vs. 29,224 diagnoses (1,911 per month) obtained via the automated system, an approximately 13–fold increase. Students’ median diagnoses logged per month increased from 17.2 (manual) to 55.3 (automated). During the accuracy analysis period, 21 students were interviewed; 90% of students confirmed all patients seen were captured by the automated system, and 95% agreed the top three diagnoses were accurate. Discussion and Conclusion:  The automated system significantly augmented the quantity and scope of RCEs logged on an inpatient medicine clerkship. Scaling of this process may provide comprehensive centralized tracking across multiple clerkships and enable real-time learner coaching opportunities by highlighting gaps in experience. Limitations include reliance on potentially inaccurate or incomplete EHR diagnoses and limited integration across multiple EHR systems. One potential area for future exploration is using large language models to extract diagnoses directly from EHR documentation, reducing reliance on existing diagnostic ontologies. Take-home Message(s):  Automated capture of RCEs via EHR data can significantly increase the breadth of diagnoses captured longitudinally across the entire medical school experience, decrease students’ administrative burden, and permit near-real-time adjustments to learners’ clinical experiences to meet requisite competencies.