Uncertainty-Guided Model Selection for Tabular Foundation Models in Biomolecule Efficacy Prediction
This addresses the challenge of optimizing predictions for biomolecule efficacy, such as siRNA knockdown, using uncertainty as a label-free heuristic, representing an incremental improvement in model selection for domain-specific applications.
The study tackled the problem of selecting the best models for ensembling in biomolecule efficacy prediction without ground truth labels by using an uncertainty-guided strategy, resulting in the OligoICP method achieving superior performance compared to naive ensembling or single models.
In-context learners like TabPFN are promising for biomolecule efficacy prediction, where established molecular feature sets and relevant experimental results can serve as powerful contextual examples. However, their performance is highly sensitive to the provided context, making strategies like post-hoc ensembling of models trained on different data subsets a viable approach. An open question is how to select the best models for the ensemble without access to ground truth labels. In this study, we investigate an uncertainty-guided strategy for model selection. We demonstrate on an siRNA knockdown efficacy task that a TabPFN model using straightforward sequence-based features can surpass specialized state-of-the-art predictors. We also show that the model's predicted inter-quantile range (IQR), a measure of its uncertainty, has a negative correlation with true prediction error. We developed the OligoICP method, which selects and averages an ensemble of models with the lowest mean IQR for siRNA efficacy prediction, achieving superior performance compared to naive ensembling or using a single model trained on all available data. This finding highlights model uncertainty as a powerful, label-free heuristic for optimizing biomolecule efficacy predictions.