2nd Place Solution for PVUW Challenge 2024: Video Panoptic Segmentation
This work addresses video panoptic segmentation, a challenging task important for applications like video understanding and autonomous driving, but it is incremental as it builds on existing frameworks.
The paper tackled video panoptic segmentation by proposing an integrated solution that builds on the DVIS++ framework and adds an image semantic segmentation model, achieving state-of-the-art performance with VPQ scores of 56.36 and 57.12 in development and test phases, respectively, and securing 2nd place in the PVUW Challenge at CVPR2024.
Video Panoptic Segmentation (VPS) is a challenging task that is extends from image panoptic segmentation.VPS aims to simultaneously classify, track, segment all objects in a video, including both things and stuff. Due to its wide application in many downstream tasks such as video understanding, video editing, and autonomous driving. In order to deal with the task of video panoptic segmentation in the wild, we propose a robust integrated video panoptic segmentation solution. We use DVIS++ framework as our baseline to generate the initial masks. Then,we add an additional image semantic segmentation model to further improve the performance of semantic classes.Finally, our method achieves state-of-the-art performance with a VPQ score of 56.36 and 57.12 in the development and test phases, respectively, and ultimately ranked 2nd in the VPS track of the PVUW Challenge at CVPR2024.