Electronic Health Records (EHRs) provide a rich, longitudinal view of patient health, but their high dimensionality, irregular temporal structure, and heterogeneous data types present significant challenges for machine learning. Transformer-based foundation models offer a promising approach by representing patient histories as sequences of clinical events. However, key design decisions, such as how to represent clinical concepts, encode continuous values, and structure temporal information, remain largely underexplored.
This presentation systematically evaluates how different ways of organizing and representing EHR data affect model performance on downstream clinical tasks. Patient histories are converted into sequences of events (diagnoses, medications, lab results, etc.) that a model can process similarly to language. This study compares several key design choices, including how detailed diagnosis codes should be, how to represent numerical values such as lab results, and how to best capture the timing and structure of healthcare encounters.
By quantifying the impact of each design decision, this presentation provides practical guidance for constructing effective EHR foundation models and highlights representation design as a key driver of performance in healthcare AI.
