Training Noise Token Pruning
This addresses computational efficiency in vision transformers for computer vision applications, but appears incremental as it builds on existing token pruning methods.
The paper tackles the problem of token pruning in vision transformers by introducing Training Noise Token (TNT) Pruning, which uses continuous additive noise during training for smooth optimization while maintaining discrete token dropping for computational efficiency in deployment. Results on ImageNet with ViT and DeiT architectures show advantages over previous pruning methods.
In the present work we present Training Noise Token (TNT) Pruning for vision transformers. Our method relaxes the discrete token dropping condition to continuous additive noise, providing smooth optimization in training, while retaining discrete dropping computational gains in deployment settings. We provide theoretical connections to Rate-Distortion literature, and empirical evaluations on the ImageNet dataset using ViT and DeiT architectures demonstrating TNT's advantages over previous pruning methods.