Context-Aware Token Pruning and Discriminative Selective Attention for Transformer Tracking
This work addresses tracking accuracy issues in computer vision, particularly for applications requiring robust object tracking, and is incremental as it builds on existing token pruning methods by adding context preservation and selective attention mechanisms.
The paper tackles the problem of background and distractor interference in one-stream Transformer-based trackers, which weakens discriminative capability, by proposing CPDATrack, a framework that prunes less-informative tokens while preserving context and uses selective attention to suppress interference, achieving state-of-the-art performance with an average overlap of 75.1% on GOT-10k.
One-stream Transformer-based trackers have demonstrated remarkable performance by concatenating template and search region tokens, thereby enabling joint attention across all tokens. However, enabling an excessive proportion of background search tokens to attend to the target template tokens weakens the tracker's discriminative capability. Several token pruning methods have been proposed to mitigate background interference; however, they often remove tokens near the target, leading to the loss of essential contextual information and degraded tracking performance. Moreover, the presence of distractors within the search tokens further reduces the tracker's ability to accurately identify the target. To address these limitations, we propose CPDATrack, a novel tracking framework designed to suppress interference from background and distractor tokens while enhancing computational efficiency. First, a learnable module is integrated between two designated encoder layers to estimate the probability of each search token being associated with the target. Based on these estimates, less-informative background tokens are pruned from the search region while preserving the contextual cues surrounding the target. To further suppress background interference, a discriminative selective attention mechanism is employed that fully blocks search-to-template attention in the early layers. In the subsequent encoder layers, high-probability target tokens are selectively extracted from a localized region to attend to the template tokens, thereby reducing the influence of background and distractor tokens. The proposed CPDATrack achieves state-of-the-art performance across multiple benchmarks, particularly on GOT-10k, where it attains an average overlap of 75.1 percent.