Multi-Exit Vision Transformer for Dynamic Inference
This work addresses latency requirements in edge computing and IoT systems with variable resources, though it is incremental as it adapts existing multi-exit concepts to Vision Transformers.
The authors tackled the problem of dynamic inference for Vision Transformers in time-critical IoT applications by proposing seven early exit branch architectures, demonstrating that each can effectively trade off accuracy and speed in classification and regression tasks.
Deep neural networks can be converted to multi-exit architectures by inserting early exit branches after some of their intermediate layers. This allows their inference process to become dynamic, which is useful for time critical IoT applications with stringent latency requirements, but with time-variant communication and computation resources. In particular, in edge computing systems and IoT networks where the exact computation time budget is variable and not known beforehand. Vision Transformer is a recently proposed architecture which has since found many applications across various domains of computer vision. In this work, we propose seven different architectures for early exit branches that can be used for dynamic inference in Vision Transformer backbones. Through extensive experiments involving both classification and regression problems, we show that each one of our proposed architectures could prove useful in the trade-off between accuracy and speed.