A Masked language model for multi-source EHR trajectories contextual representation learning
This work addresses a specific bottleneck in EHR analysis for healthcare applications, but it is incremental as it builds on existing transformer methods.
The paper tackled the challenge of modeling interactions between diseases and interventions in electronic health records by masking one source and training a transformer to predict it using other sources, resulting in a method for multi-source contextual representation learning.
Using electronic health records data and machine learning to guide future decisions needs to address challenges, including 1) long/short-term dependencies and 2) interactions between diseases and interventions. Bidirectional transformers have effectively addressed the first challenge. Here we tackled the latter challenge by masking one source (e.g., ICD10 codes) and training the transformer to predict it using other sources (e.g., ATC codes).