Long-tailed Recognition with Model Rebalancing
This addresses the challenge of skewed class distributions in deep learning, which hinders model generalization to tail classes, offering a plug-and-play solution that is incremental but effective in complementing existing methods.
The paper tackles the problem of long-tailed recognition in deep learning by proposing Model Rebalancing (MORE), a framework that rebalances the model's parameter space to improve generalization, especially for tail classes, achieving significant gains across diverse benchmarks without increasing model complexity or inference costs.
Long-tailed recognition is ubiquitous and challenging in deep learning and even in the downstream finetuning of foundation models, since the skew class distribution generally prevents the model generalization to the tail classes. Despite the promise of previous methods from the perspectives of data augmentation, loss rebalancing and decoupled training etc., consistent improvement in the broad scenarios like multi-label long-tailed recognition is difficult. In this study, we dive into the essential model capacity impact under long-tailed context, and propose a novel framework, Model Rebalancing (MORE), which mitigates imbalance by directly rebalancing the model's parameter space. Specifically, MORE introduces a low-rank parameter component to mediate the parameter space allocation guided by a tailored loss and sinusoidal reweighting schedule, but without increasing the overall model complexity or inference costs. Extensive experiments on diverse long-tailed benchmarks, spanning multi-class and multi-label tasks, demonstrate that MORE significantly improves generalization, particularly for tail classes, and effectively complements existing imbalance mitigation methods. These results highlight MORE's potential as a robust plug-and-play module in long-tailed settings.