LLM-Viterbi: Semantic-Aware Decoding for Convolutional Codes
For wireless communication systems transmitting text, this work introduces a novel semantic-aware decoding approach that significantly improves error correction by leveraging linguistic structure.
The paper proposes an LLM-Viterbi decoder that integrates a fine-tuned ByT5 language model into Viterbi decoding for text transmission over AWGN channels, achieving approximately 1.5 dB coding gain in BLER and over 50% improvement in semantic similarity over conventional Viterbi decoding.
Traditional wireless communications rely solely on bit-level channel coding for error correction, without exploiting the inherent linguistic structure of the data source. This paper proposes a large language model (LLM) Viterbi decoder that integrates LLM priors into the Viterbi decoding for text transmission over AWGN channels. The proposed decoder maintains multiple candidate paths during the Viterbi decoding and periodically evaluates path reliabilities using a fine-tuned Byte-level T5 (ByT5) language model. By combining channel reliability metrics with semantic probability from the LLM, it outputs the path that maximizes the joint likelihood of channel observations and linguistic coherence. Simulations show that our decoder achieves significant performance gains over conventional Viterbi decoding in terms of both block error rate (BLER) and semantic similarity. For convolutional codes with constraint length 3, it achieves approximately 1.5 dB more coding gain in BLER, with over 50% improvements in semantic similarity. The framework can extend to other structured data sources beyond text.