$φ$-DPO: Fairness Direct Preference Optimization Approach to Continual Learning in Large Multimodal Models
This paper addresses the underexplored challenge of fairness in continual learning for Large Multimodal Models, which is important for practitioners deploying LMMs in real-world scenarios with imbalanced data.
This paper tackles the problem of fairness in continual learning for Large Multimodal Models (LMMs) when faced with imbalanced data distributions, which can lead to biased model updates and suboptimal performance. They propose a novel Fairness Direct Preference Optimization ($φ$-DPO) framework that mitigates catastrophic forgetting and explicitly addresses distributional biases, achieving State-of-the-Art performance across multiple benchmarks.
Fairness in Continual Learning for Large Multimodal Models (LMMs) is an emerging yet underexplored challenge, particularly in the presence of imbalanced data distributions that can lead to biased model updates and suboptimal performance across tasks. While recent continual learning studies have made progress in addressing catastrophic forgetting, the problem of fairness caused the imbalanced data remains largely underexplored. This paper presents a novel Fairness Direct Preference Optimization (FaiDPO or $φ$-DPO) framework for continual learning in LMMs. In particular, we first propose a new continual learning paradigm based on Direct Preference Optimization (DPO) to mitigate catastrophic forgetting by aligning learning with pairwise preference signals. Then, we identify the limitations of conventional DPO in imbalanced data and present a new $φ$-DPO loss that explicitly addresses distributional biases. We provide a comprehensive theoretical analysis demonstrating that our approach addresses both forgetting and data imbalance. Additionally, to enable $φ$-DPO-based continual learning, we construct pairwise preference annotations for existing benchmarks in the context of continual learning. Extensive experiments and ablation studies show the proposed $φ$-DPO achieves State-of-the-Art performance across multiple benchmarks, outperforming prior continual learning methods of LMMs.