Information-Theoretic Bounds and Approximations in Neural Population Coding
This work addresses a computational bottleneck for researchers in neuroscience and information theory by providing efficient methods to approximate mutual information in large neural populations, though it is incremental as it builds on existing information-theoretic frameworks.
The paper tackles the challenge of accurately calculating mutual information for high-dimensional variables in neural population coding by deriving asymptotic bounds and approximation formulas that remain valid in high-dimensional spaces, with numerical simulations confirming high accuracy and special cases achieving exact equality.
While Shannon's mutual information has widespread applications in many disciplines, for practical applications it is often difficult to calculate its value accurately for high-dimensional variables because of the curse of dimensionality. This paper is focused on effective approximation methods for evaluating mutual information in the context of neural population coding. For large but finite neural populations, we derive several information-theoretic asymptotic bounds and approximation formulas that remain valid in high-dimensional spaces. We prove that optimizing the population density distribution based on these approximation formulas is a convex optimization problem which allows efficient numerical solutions. Numerical simulation results confirmed that our asymptotic formulas were highly accurate for approximating mutual information for large neural populations. In special cases, the approximation formulas are exactly equal to the true mutual information. We also discuss techniques of variable transformation and dimensionality reduction to facilitate computation of the approximations.