Transformer Compressed Sensing via Global Image Tokens
This work addresses compressed sensing for medical imaging by introducing a more efficient transformer-based method, though it appears incremental as it builds on existing CS-MRI networks.
The paper tackled the limitations of convolutional neural networks in compressed sensing by proposing a novel image decomposition using Kaleidoscope tokens to enable global attention in transformers, achieving improved performance in CS-MRI with enhanced image quality and reduced model size.
Convolutional neural networks (CNN) have demonstrated outstanding Compressed Sensing (CS) performance compared to traditional, hand-crafted methods. However, they are broadly limited in terms of generalisability, inductive bias and difficulty to model long distance relationships. Transformer neural networks (TNN) overcome such issues by implementing an attention mechanism designed to capture dependencies between inputs. However, high-resolution tasks typically require vision Transformers (ViT) to decompose an image into patch-based tokens, limiting inputs to inherently local contexts. We propose a novel image decomposition that naturally embeds images into low-resolution inputs. These Kaleidoscope tokens (KD) provide a mechanism for global attention, at the same computational cost as a patch-based approach. To showcase this development, we replace CNN components in a well-known CS-MRI neural network with TNN blocks and demonstrate the improvements afforded by KD. We also propose an ensemble of image tokens, which enhance overall image quality and reduces model size. Supplementary material is available: https://github.com/uqmarlonbran/TCS.git