Multiple Instance Neuroimage Transformer
This work addresses neuroimage classification for researchers, but it is incremental as it adapts existing transformer and multiple instance learning concepts to a specific domain.
The authors tackled the problem of classifying T1-weighted MRIs by proposing the Multiple Instance Neuroimage Transformer (MINiT), a convolution-free transformer model using multiple instance learning, and demonstrated its efficacy by training it to identify sex from two public datasets with learned attention maps highlighting relevant brain morphometry differences.
For the first time, we propose using a multiple instance learning based convolution-free transformer model, called Multiple Instance Neuroimage Transformer (MINiT), for the classification of T1weighted (T1w) MRIs. We first present several variants of transformer models adopted for neuroimages. These models extract non-overlapping 3D blocks from the input volume and perform multi-headed self-attention on a sequence of their linear projections. MINiT, on the other hand, treats each of the non-overlapping 3D blocks of the input MRI as its own instance, splitting it further into non-overlapping 3D patches, on which multi-headed self-attention is computed. As a proof-of-concept, we evaluate the efficacy of our model by training it to identify sex from T1w-MRIs of two public datasets: Adolescent Brain Cognitive Development (ABCD) and the National Consortium on Alcohol and Neurodevelopment in Adolescence (NCANDA). The learned attention maps highlight voxels contributing to identifying sex differences in brain morphometry. The code is available at https://github.com/singlaayush/MINIT.