Seg2Track++: Probabilistic Track Validation and Data Association for Multi-Object Tracking and Segmentation
For autonomous systems requiring robust multi-object tracking and segmentation, Seg2Track++ offers a zero-shot solution that mitigates track association failures and false-positive propagation.
Seg2Track++ integrates SAM2 segmentation with probabilistic track validation to achieve zero-shot MOTS, reducing false positives and improving identity preservation. On KITTI MOTS, it outperforms prior zero-shot methods.
Autonomous systems require robust Multi-Object Tracking and Segmentation (MOTS) to operate reliably in dynamic environments, ensuring consistent object identities and precise mask-level delineation. Foundation models such as SAM2 have shown strong zero-shot generalization for segmentation, but their direct application to MOTS is limited by unreliable track association and false-positive propagation. This work introduces Seg2Track++, a framework that integrates instance segmentation with SAM2 and a novel track management module to perform zero-shot MOTS with enhanced temporal consistency. Tracks are associated using Mask Centroid Distance (MCD) and Confidence-Aware Cost Modulation (CCM), while Probabilistic Track Validation (PTV) employs a Bernoulli filter to validate track existence and suppress ghost tracks. Experimental results on KITTI MOTS demonstrate improved identity preservation, reduced false-positive propagation, and robust track management without fine-tuning.