Correcting Class Imbalance in Prior-Data Fitted Networks for Tabular Classification
For practitioners using PFNs on imbalanced tabular data, this work provides practical, effective adaptations of classical methods, though it is incremental in nature.
This paper adapts classical class imbalance techniques to Prior-Data Fitted Networks (PFNs) for tabular classification, finding that thresholding and downsampling effectively mitigate imbalance, with thresholding leveraging PFN calibration and downsampling reducing inference cost.
Prior-data fitted networks (PFNs) have achieved exceptional performance on tabular classification tasks. However, like other classifiers, their performance can suffer under the effect of class imbalance, resulting in poor performance for rare classes. Several techniques exist which attempt to mitigate the deleterious effect of class imbalance on classification performance, but the in-context learning (ICL) dynamic of PFNs means that loss-based strategies are impossible, and other techniques are unproven. We have adapted several classical techniques addressing class imbalance and analyzed their performance on PFN classification. We observe that thresholding performs exceptionally well because of the calibration characteristics of PFNs, and downsampling performs comparably because of PFNs exceptional limited-data performance, with the additional benefit of reduced computation cost for inference.