Anatomically Constrained Transformers for Echocardiogram Analysis
This work addresses a domain-specific problem for medical imaging researchers and clinicians by providing a more reliable and interpretable method for echocardiogram analysis, though it is incremental as it builds on existing transformer and masked autoencoding techniques.
The authors tackled the problem of video transformers learning spurious correlations from non-diagnostic regions in echocardiogram analysis by proposing ViACT, which integrates anatomical priors to focus representation learning on the myocardium, resulting in improved performance for tasks like left ventricular ejection fraction regression and cardiac amyloidosis detection with interpretable attention maps.
Video transformers have recently demonstrated strong potential for echocardiogram (echo) analysis, leveraging self-supervised pre-training and flexible adaptation across diverse tasks. However, like other models operating on videos, they are prone to learning spurious correlations from non-diagnostic regions such as image backgrounds. To overcome this limitation, we propose the Video Anatomically Constrained Transformer (ViACT), a novel framework that integrates anatomical priors directly into the transformer architecture. ViACT represents a deforming anatomical structure as a point set and encodes both its spatial geometry and corresponding image patches into transformer tokens. During pre-training, ViACT follows a masked autoencoding strategy that masks and reconstructs only anatomical patches, enforcing that representation learning is focused on the anatomical region. The pre-trained model can then be fine-tuned for tasks localized to this region. In this work we focus on the myocardium, demonstrating the framework on echo analysis tasks such as left ventricular ejection fraction (EF) regression and cardiac amyloidosis (CA) detection. The anatomical constraint focuses transformer attention within the myocardium, yielding interpretable attention maps aligned with regions of known CA pathology. Moreover, ViACT generalizes to myocardium point tracking without requiring task-specific components such as correlation volumes used in specialized tracking networks.