Enforcing Predictive Invariance across Structured Biomedical Domains
This work addresses the problem of domain generalization in structured biomedical data for researchers and practitioners, representing an incremental improvement over existing invariant risk minimization methods.
The paper tackled the challenge of generalizing predictive models across unseen or sparsely sampled structured biomedical domains, such as molecular scaffolds or protein families, by proposing a regret minimization algorithm that adapts invariant risk minimization to structured environments. The method significantly outperformed previous state-of-the-art baselines in applications like molecular property prediction and protein homology and stability prediction.
Many biochemical applications such as molecular property prediction require models to generalize beyond their training domains (environments). Moreover, natural environments in these tasks are structured, defined by complex descriptors such as molecular scaffolds or protein families. Therefore, most environments are either never seen during training, or contain only a single training example. To address these challenges, we propose a new regret minimization (RGM) algorithm and its extension for structured environments. RGM builds from invariant risk minimization (IRM) by recasting simultaneous optimality condition in terms of predictive regret, finding a representation that enables the predictor to compete against an oracle with hindsight access to held-out environments. The structured extension adaptively highlights variation due to complex environments via specialized domain perturbations. We evaluate our method on multiple applications: molecular property prediction, protein homology and stability prediction and show that RGM significantly outperforms previous state-of-the-art baselines.