FocusMAE: Gallbladder Cancer Detection from Ultrasound Videos with Focused Masked Autoencoders
This work addresses the problem of limited generalization in automated Gallbladder Cancer detection for medical practitioners by introducing a novel video-based approach, though it is incremental as it builds on existing masked autoencoder techniques.
The study tackled automated Gallbladder Cancer detection from ultrasound videos by proposing FocusMAE, a method that biases masking tokens to high-information regions, achieving a state-of-the-art accuracy of 96.4% compared to 84% for image-based methods and 94.7% for video-based methods.
In recent years, automated Gallbladder Cancer (GBC) detection has gained the attention of researchers. Current state-of-the-art (SOTA) methodologies relying on ultrasound sonography (US) images exhibit limited generalization, emphasizing the need for transformative approaches. We observe that individual US frames may lack sufficient information to capture disease manifestation. This study advocates for a paradigm shift towards video-based GBC detection, leveraging the inherent advantages of spatiotemporal representations. Employing the Masked Autoencoder (MAE) for representation learning, we address shortcomings in conventional image-based methods. We propose a novel design called FocusMAE to systematically bias the selection of masking tokens from high-information regions, fostering a more refined representation of malignancy. Additionally, we contribute the most extensive US video dataset for GBC detection. We also note that, this is the first study on US video-based GBC detection. We validate the proposed methods on the curated dataset, and report a new state-of-the-art (SOTA) accuracy of 96.4% for the GBC detection problem, against an accuracy of 84% by current Image-based SOTA - GBCNet, and RadFormer, and 94.7% by Video-based SOTA - AdaMAE. We further demonstrate the generality of the proposed FocusMAE on a public CT-based Covid detection dataset, reporting an improvement in accuracy by 3.3% over current baselines. The source code and pretrained models are available at: https://gbc-iitd.github.io/focusmae