DynImmune-BERT: Dynamic Immune Repertoire Modeling with Neural ODE Driven Continuous Transformers
This work addresses the need for temporal modeling in immune repertoire analysis, offering a method that captures clonal dynamics for patient-level prediction, but the gains are incremental over strong static baselines and limited by small external validation cohorts.
DynImmune-BERT models longitudinal T cell receptor repertoires for patient-level immune status prediction by combining neural ODE dynamics with transformer attention, achieving improved temporal modeling over static baselines. The method demonstrates that event-aware temporal modeling complements static encoders when longitudinal data is available, though small external cohorts require cautious interpretation.
Longitudinal T cell receptor repertoires contain signals of clonal expansion, contraction, disappearance, and reappearance after immune perturbation. Static repertoire language models usually summarize a sample as a bag of sequences, so the sampling interval, sequencing depth, and clone presence pattern are only weakly represented. This paper presents DynImmune-BERT, a continuous time repertoire model for patient level immune status prediction. The method combines depth adaptive centered log ratio initialization, clone presence gated Neural ordinary differential equation dynamics, bounded neighborhood self attention, event based state restart, and a hybrid transport objective that supervises dominant and rare clone mass. A low rank meta adapter initializes reappearing clonotypes while keeping the parameter count independent of the number of observed clones. The evaluation separates literature reported baselines from internally controlled temporal comparisons, reports uncertainty for small external cohorts, adds calibration and threshold diagnostics, and visualizes latent clone trajectories and attention neighborhoods. The results indicate that event aware temporal modeling can complement strong static encoders when longitudinal repertoire structure is available, while small external cohorts and protocol differences require cautious interpretation.