Vision Transformers for Mobile Applications: A Short Survey
It addresses the problem of deploying large-scale ViTs on mobile devices for researchers and practitioners, but is incremental as it reviews existing work without proposing novel solutions.
The paper surveys Vision Transformers (ViTs) designed for mobile applications, analyzing how to balance accuracy and latency for deployment on resource-constrained devices, but does not present new experimental results or concrete numbers.
Vision Transformers (ViTs) have demonstrated state-of-the-art performance on many Computer Vision Tasks. Unfortunately, deploying these large-scale ViTs is resource-consuming and impossible for many mobile devices. While most in the community are building for larger and larger ViTs, we ask a completely opposite question: How small can a ViT be within the tradeoffs of accuracy and inference latency that make it suitable for mobile deployment? We look into a few ViTs specifically designed for mobile applications and observe that they modify the transformer's architecture or are built around the combination of CNN and transformer. Recent work has also attempted to create sparse ViT networks and proposed alternatives to the attention module. In this paper, we study these architectures, identify the challenges and analyze what really makes a vision transformer suitable for mobile applications. We aim to serve as a baseline for future research direction and hopefully lay the foundation to choose the exemplary vision transformer architecture for your application running on mobile devices.