FOSSIL: Regret-minimizing weighting for robust learning under imbalance and small data
This addresses challenges in domains such as rare disease imaging and genomics, where labeled samples are scarce, offering a practical solution with theoretical guarantees.
The paper tackles the problem of robust learning under imbalanced and small data regimes, introducing FOSSIL, a unified weighting framework that integrates multiple aspects like class imbalance correction and difficulty-aware curricula, achieving consistent empirical gains over baselines on synthetic and real-world datasets.
Imbalanced and small data regimes are pervasive in domains such as rare disease imaging, genomics, and disaster response, where labeled samples are scarce and naive augmentation often introduces artifacts. Existing solutions such as oversampling, focal loss, or meta-weighting address isolated aspects of this challenge but remain fragile or complex. We introduce FOSSIL (Flexible Optimization via Sample Sensitive Importance Learning), a unified weighting framework that seamlessly integrates class imbalance correction, difficulty-aware curricula, augmentation penalties, and warmup dynamics into a single interpretable formula. Unlike prior heuristics, the proposed framework provides regret-based theoretical guarantees and achieves consistent empirical gains over ERM, curriculum, and meta-weighting baselines on synthetic and real-world datasets, while requiring no architectural changes.