Structured Codes and Free-Text Notes: Measuring Information Complementarity in Electronic Health Records

ABSTRACT Background Electronic health records (EHRs) consist of both structured data (e.g., diagnostic codes) and unstructured data (e.g., clinical notes). It’s commonly believed that unstructured clinical narratives provide more comprehensive information. However, this assumption often lacks large-scale validation and direct validation methods. Objective This study aims to quantitatively compare the information in structured and unstructured EHR data and directly validate whether unstructured data offers more extensive information across a patient population. Methods We analyzed both structured and unstructured data from patient records and visits in a large Dutch primary care EHR database between January 2021 and January 2024. Clinical concepts were identified from free-text notes using an extraction framework tailored for Dutch and compared with concepts from structured data. Concept embeddings were generated to measure semantic similarity between structured and extracted concepts through cosine similarity. A similarity threshold was systematically determined via annotated matches and minimized weighted Gini impurity. We then quantified the concept overlap between structured and unstructured data across various concept domains and patient populations. Results In a population of 1.8 million patients, 42% of structured concepts in patient records and 25% in individual visits had similar matches in unstructured data. Conversely, only 13% of extracted concepts from records and 7% from visits had similar structured counterparts. Condition concepts had the highest overlap, followed by measurements and drug concepts. Subpopulation visits, such as those with chronic conditions or psychological disorders, showed different proportions of data overlap, indicating varied reliance on structured versus unstructured data across clinical contexts. Conclusions Our study demonstrates the feasibility of quantifying the information difference between structured and unstructured data, showing that the unstructured data provides important additional information in the studied database and populations. Despite some limitations, our proposed methodology proves versatile, and its application can lead to more robust and insightful observational clinical research.