Efficient Joint Detection and Multiple Object Tracking with Spatially Aware Transformer
This work addresses efficiency challenges in multi-object tracking for applications like surveillance and autonomous driving, though it is incremental as it builds on TransTrack.
The paper tackles the computational bottleneck in joint detection and multiple object tracking by proposing a lightweight transformer-based pipeline that achieves a state-of-the-art MOTA score of 73.20% while reducing model size by 58.73% and complexity by 78.72% compared to TransTrack.
We propose a light-weight and highly efficient Joint Detection and Tracking pipeline for the task of Multi-Object Tracking using a fully-transformer architecture. It is a modified version of TransTrack, which overcomes the computational bottleneck associated with its design, and at the same time, achieves state-of-the-art MOTA score of 73.20%. The model design is driven by a transformer based backbone instead of CNN, which is highly scalable with the input resolution. We also propose a drop-in replacement for Feed Forward Network of transformer encoder layer, by using Butterfly Transform Operation to perform channel fusion and depth-wise convolution to learn spatial context within the feature maps, otherwise missing within the attention maps of the transformer. As a result of our modifications, we reduce the overall model size of TransTrack by 58.73% and the complexity by 78.72%. Therefore, we expect our design to provide novel perspectives for architecture optimization in future research related to multi-object tracking.