Strike a Balance in Continual Panoptic Segmentation
This work addresses the problem of continual learning in panoptic segmentation for computer vision applications, representing an incremental improvement with specific technical innovations.
The paper tackles continual panoptic segmentation by introducing a method called BalConpas that balances stability and adaptability through past-class backtrace distillation, class-proportional memory, and balanced anti-misguidance losses, achieving superior performance on the ADE20K dataset compared to state-of-the-art methods.
This study explores the emerging area of continual panoptic segmentation, highlighting three key balances. First, we introduce past-class backtrace distillation to balance the stability of existing knowledge with the adaptability to new information. This technique retraces the features associated with past classes based on the final label assignment results, performing knowledge distillation targeting these specific features from the previous model while allowing other features to flexibly adapt to new information. Additionally, we introduce a class-proportional memory strategy, which aligns the class distribution in the replay sample set with that of the historical training data. This strategy maintains a balanced class representation during replay, enhancing the utility of the limited-capacity replay sample set in recalling prior classes. Moreover, recognizing that replay samples are annotated only for the classes of their original step, we devise balanced anti-misguidance losses, which combat the impact of incomplete annotations without incurring classification bias. Building upon these innovations, we present a new method named Balanced Continual Panoptic Segmentation (BalConpas). Our evaluation on the challenging ADE20K dataset demonstrates its superior performance compared to existing state-of-the-art methods. The official code is available at https://github.com/jinpeng0528/BalConpas.