SegTransVAE: Hybrid CNN -- Transformer with Regularization for medical image segmentation
This work addresses medical image segmentation by combining CNN, Transformer, and VAE for the first time, offering incremental improvements in accuracy and efficiency.
The paper tackles the limitations of deep learning in medical image segmentation by proposing SegTransVAE, a hybrid CNN-Transformer-VAE network that improves Dice Score and 95%-Haudorff Distance on various datasets while maintaining comparable inference time to CNN-based methods.
Current research on deep learning for medical image segmentation exposes their limitations in learning either global semantic information or local contextual information. To tackle these issues, a novel network named SegTransVAE is proposed in this paper. SegTransVAE is built upon encoder-decoder architecture, exploiting transformer with the variational autoencoder (VAE) branch to the network to reconstruct the input images jointly with segmentation. To the best of our knowledge, this is the first method combining the success of CNN, transformer, and VAE. Evaluation on various recently introduced datasets shows that SegTransVAE outperforms previous methods in Dice Score and $95\%$-Haudorff Distance while having comparable inference time to a simple CNN-based architecture network. The source code is available at: https://github.com/itruonghai/SegTransVAE.