Scopeformer: n-CNN-ViT Hybrid Model for Intracranial Hemorrhage Classification
This work addresses a critical medical imaging problem for healthcare by improving classification accuracy, though it is incremental as it builds on existing CNN and ViT methods.
The authors tackled intracranial hemorrhage classification from CT scans by proposing a hybrid model that combines multiple CNNs with a Vision Transformer to generate feature-rich inputs, achieving a test accuracy of 98.04% and a weighted logarithmic loss of 0.0708.
We propose a feature generator backbone composed of an ensemble of convolutional neuralnetworks (CNNs) to improve the recently emerging Vision Transformer (ViT) models. We tackled the RSNA intracranial hemorrhage classification problem, i.e., identifying various hemorrhage types from computed tomography (CT) slices. We show that by gradually stacking several feature maps extracted using multiple Xception CNNs, we can develop a feature-rich input for the ViT model. Our approach allowed the ViT model to pay attention to relevant features at multiple levels. Moreover, pretraining the n CNNs using various paradigms leads to a diverse feature set and further improves the performance of the proposed n-CNN-ViT. We achieved a test accuracy of 98.04% with a weighted logarithmic loss value of 0.0708. The proposed architecture is modular and scalable in both the number of CNNs used for feature extraction and the size of the ViT.