Beyond Imbalance Ratio: Data Characteristics as Critical Moderators of Oversampling Method Selection
This work addresses a practical problem for machine learning practitioners by challenging a common paradigm in imbalanced data handling, offering evidence-based criteria, though it is incremental in refining existing oversampling approaches.
The study tackled the assumption that imbalance ratio (IR) correlates with oversampling effectiveness by conducting controlled experiments, finding IR had a weak to moderate negative correlation while class separability was a stronger moderator, and proposed a framework for method selection.
The prevailing IR-threshold paradigm posits a positive correlation between imbalance ratio (IR) and oversampling effectiveness, yet this assumption remains empirically unsubstantiated through controlled experimentation. We conducted 12 controlled experiments (N > 100 dataset variants) that systematically manipulated IR while holding data characteristics (class separability, cluster structure) constant via algorithmic generation of Gaussian mixture datasets. Two additional validation experiments examined ceiling effects and metric-dependence. All methods were evaluated on 17 real-world datasets from OpenML. Upon controlling for confounding variables, IR exhibited a weak to moderate negative correlation with oversampling benefits. Class separability emerged as a substantially stronger moderator, accounting for significantly more variance in method effectiveness than IR alone. We propose a 'Context Matters' framework that integrates IR, class separability, and cluster structure to provide evidence-based selection criteria for practitioners.