Hybrid Mamba-Transformer Decoder for Error-Correcting Codes
This work addresses decoding challenges in communication systems with a novel hybrid architecture, though it appears incremental as it builds on existing Mamba and Transformer components.
The paper tackles the problem of decoding error correction codes by introducing a hybrid Mamba-Transformer decoder with layer-wise masking and progressive loss, achieving significant performance improvements over Transformer-only and standard Mamba models across various linear codes.
We introduce a novel deep learning method for decoding error correction codes based on the Mamba architecture, enhanced with Transformer layers. Our approach proposes a hybrid decoder that leverages Mamba's efficient sequential modeling while maintaining the global context capabilities of Transformers. To further improve performance, we design a novel layer-wise masking strategy applied to each Mamba layer, allowing selective attention to relevant code features at different depths. Additionally, we introduce a progressive layer-wise loss, supervising the network at intermediate stages and promoting robust feature extraction throughout the decoding process. Comprehensive experiments across a range of linear codes demonstrate that our method significantly outperforms Transformer-only decoders and standard Mamba models.