Dual-LoRA and Quality-Enhanced Pseudo Replay for Multimodal Continual Food Learning
This addresses the problem of costly retraining for health-related food analysis tasks, though it appears incremental as it builds on existing continual learning methods.
The paper tackles catastrophic forgetting in large multimodal models for food analysis by proposing a continual learning framework with Dual-LoRA and Quality-Enhanced Pseudo Replay, achieving superior performance in mitigating forgetting on the Uni-Food dataset.
Food analysis has become increasingly critical for health-related tasks such as personalized nutrition and chronic disease prevention. However, existing large multimodal models (LMMs) in food analysis suffer from catastrophic forgetting when learning new tasks, requiring costly retraining from scratch. To address this, we propose a novel continual learning framework for multimodal food learning, integrating a Dual-LoRA architecture with Quality-Enhanced Pseudo Replay. We introduce two complementary low-rank adapters for each task: a specialized LoRA that learns task-specific knowledge with orthogonal constraints to previous tasks' subspaces, and a cooperative LoRA that consolidates shared knowledge across tasks via pseudo replay. To improve the reliability of replay data, our Quality-Enhanced Pseudo Replay strategy leverages self-consistency and semantic similarity to reduce hallucinations in generated samples. Experiments on the comprehensive Uni-Food dataset show superior performance in mitigating forgetting, representing the first effective continual learning approach for complex food tasks.