Data-Driven Estimation of Capacity Upper Bounds
This work addresses a fundamental problem in information theory for researchers and engineers dealing with channel capacity estimation, but it is incremental as it builds on existing dual representations and neural estimators.
The paper tackles the problem of estimating capacity upper bounds for memoryless channels with unknown laws and continuous outputs, proposing a data-driven algorithm that uses a dual representation and a modified neural estimator, and shows that the estimated bounds closely converge to channel capacity or best-known lower bounds in numerical evaluations.
We consider the problem of estimating an upper bound on the capacity of a memoryless channel with unknown channel law and continuous output alphabet. A novel data-driven algorithm is proposed that exploits the dual representation of capacity where the maximization over the input distribution is replaced with a minimization over a reference distribution on the channel output. To efficiently compute the required divergence maximization between the conditional channel and the reference distribution, we use a modified mutual information neural estimator that takes the channel input as an additional parameter. We numerically evaluate our approach on different memoryless channels and show empirically that the estimated upper bounds closely converge either to the channel capacity or to best-known lower bounds.