Hands-on Evaluation of Visual Transformers for Object Recognition and Detection
This work provides a hands-on evaluation for computer vision researchers and practitioners, showing ViTs' competitiveness and advantages in global context understanding, though it is incremental as it builds on existing models.
The paper compared Vision Transformers (ViTs) against CNNs for tasks like object recognition and detection, finding that hybrid and hierarchical ViTs, such as Swin and CvT, balanced accuracy and efficiency well, with data augmentation on medical images leading to significant performance improvements.
Convolutional Neural Networks (CNNs) for computer vision sometimes struggle with understanding images in a global context, as they mainly focus on local patterns. On the other hand, Vision Transformers (ViTs), inspired by models originally created for language processing, use self-attention mechanisms, which allow them to understand relationships across the entire image. In this paper, we compare different types of ViTs (pure, hierarchical, and hybrid) against traditional CNN models across various tasks, including object recognition, detection, and medical image classification. We conduct thorough tests on standard datasets like ImageNet for image classification and COCO for object detection. Additionally, we apply these models to medical imaging using the ChestX-ray14 dataset. We find that hybrid and hierarchical transformers, especially Swin and CvT, offer a strong balance between accuracy and computational resources. Furthermore, by experimenting with data augmentation techniques on medical images, we discover significant performance improvements, particularly with the Swin Transformer model. Overall, our results indicate that Vision Transformers are competitive and, in many cases, outperform traditional CNNs, especially in scenarios requiring the understanding of global visual contexts like medical imaging.