Better Matching, Less Forgetting: A Quality-Guided Matcher for Transformer-based Incremental Object Detection
This addresses forgetting in incremental learning for object detection, particularly in transformer-based models, with incremental improvements over prior methods.
The paper tackled catastrophic forgetting in incremental object detection for DETR-like architectures by identifying background foregrounding as a novel source of forgetting, and proposed a Quality-guided Min-Cost Max-Flow matcher that eliminated harmful supervision, resulting in consistent outperformance of state-of-the-art methods on the COCO dataset.
Incremental Object Detection (IOD) aims to continuously learn new object classes without forgetting previously learned ones. A persistent challenge is catastrophic forgetting, primarily attributed to background shift in conventional detectors. While pseudo-labeling mitigates this in dense detectors, we identify a novel, distinct source of forgetting specific to DETR-like architectures: background foregrounding. This arises from the exhaustiveness constraint of the Hungarian matcher, which forcibly assigns every ground truth target to one prediction, even when predictions primarily cover background regions (i.e., low IoU). This erroneous supervision compels the model to misclassify background features as specific foreground classes, disrupting learned representations and accelerating forgetting. To address this, we propose a Quality-guided Min-Cost Max-Flow (Q-MCMF) matcher. To avoid forced assignments, Q-MCMF builds a flow graph and prunes implausible matches based on geometric quality. It then optimizes for the final matching that minimizes cost and maximizes valid assignments. This strategy eliminates harmful supervision from background foregrounding while maximizing foreground learning signals. Extensive experiments on the COCO dataset under various incremental settings demonstrate that our method consistently outperforms existing state-of-the-art approaches.