Unsupervised Spatio-temporal Latent Feature Clustering for Multiple-object Tracking and Segmentation
This addresses the challenge of multiple-object tracking and segmentation in video analysis, with potential applications in surveillance and autonomous systems, but it is incremental as it builds on existing methods like autoencoders and clustering.
The paper tackles the problem of assigning consistent temporal identifiers to multiple moving objects in video sequences by treating it as a spatio-temporal clustering problem, using an unsupervised deep heterogeneous autoencoder to learn features from segmentation masks and bounding boxes, and results show it outperforms several state-of-the-art methods on challenging datasets.
Assigning consistent temporal identifiers to multiple moving objects in a video sequence is a challenging problem. A solution to that problem would have immediate ramifications in multiple object tracking and segmentation problems. We propose a strategy that treats the temporal identification task as a spatio-temporal clustering problem. We propose an unsupervised learning approach using a convolutional and fully connected autoencoder, which we call deep heterogeneous autoencoder, to learn discriminative features from segmentation masks and detection bounding boxes. We extract masks and their corresponding bounding boxes from a pretrained instance segmentation network and train the autoencoders jointly using task-dependent uncertainty weights to generate common latent features. We then construct constraints graphs that encourage associations among objects that satisfy a set of known temporal conditions. The feature vectors and the constraints graphs are then provided to the kmeans clustering algorithm to separate the corresponding data points in the latent space. We evaluate the performance of our method using challenging synthetic and real-world multiple-object video datasets. Our results show that our technique outperforms several state-of-the-art methods.