A note on the expected minimum error probability in equientropic channels
This work addresses a theoretical gap in information theory for practical coding applications, but it is incremental as it builds on existing concepts like channel capacity and random coding.
The paper tackles the problem of characterizing code quality for finite-bit encoding in equientropic channels by deriving an upper bound on the expected minimum error probability, showing it is minimized for codes with maximal marginal entropy, and demonstrates that random coding achieves this for the AWGN channel in the infinite message limit.
While the channel capacity reflects a theoretical upper bound on the achievable information transmission rate in the limit of infinitely many bits, it does not characterise the information transfer of a given encoding routine with finitely many bits. In this note, we characterise the quality of a code (i. e. a given encoding routine) by an upper bound on the expected minimum error probability that can be achieved when using this code. We show that for equientropic channels this upper bound is minimal for codes with maximal marginal entropy. As an instructive example we show for the additive white Gaussian noise (AWGN) channel that random coding---also a capacity achieving code---indeed maximises the marginal entropy in the limit of infinite messages.