Steerable Transformers for Volumetric Data
This work addresses the need for equivariant models in volumetric data processing, though it appears incremental as it builds upon existing steerable convolutional networks.
The authors tackled the problem of extending Vision Transformers to maintain equivariance to the special Euclidean group SE(d) for volumetric data, resulting in enhanced performance when adding steerable transformer layers to steerable convolutional networks in experiments across two and three dimensions.
We introduce Steerable Transformers, an extension of the Vision Transformer mechanism that maintains equivariance to the special Euclidean group $\mathrm{SE}(d)$. We propose an equivariant attention mechanism that operates on features extracted by steerable convolutions. Operating in Fourier space, our network utilizes Fourier space non-linearities. Our experiments in both two and three dimensions show that adding steerable transformer layers to steerable convolutional networks enhances performance.