The Limits of AI Explainability: An Algorithmic Information Theory Approach
It addresses the problem of balancing AI capabilities with interpretability for researchers and regulators, highlighting inherent trade-offs.
This paper tackles the fundamental limits of AI explainability by formalizing it as approximating complex models with simpler ones, using algorithmic information theory to prove that simpler explanations must differ from the original model and showing exponential growth in explanation complexity with input dimension.
This paper establishes a theoretical foundation for understanding the fundamental limits of AI explainability through algorithmic information theory. We formalize explainability as the approximation of complex models by simpler ones, quantifying both approximation error and explanation complexity using Kolmogorov complexity. Our key theoretical contributions include: (1) a complexity gap theorem proving that any explanation significantly simpler than the original model must differ from it on some inputs; (2) precise bounds showing that explanation complexity grows exponentially with input dimension but polynomially with error tolerance for Lipschitz functions; and (3) a characterization of the gap between local and global explainability, demonstrating that local explanations can be significantly simpler while maintaining accuracy in relevant regions. We further establish a regulatory impossibility theorem proving that no governance framework can simultaneously pursue unrestricted AI capabilities, human-interpretable explanations, and negligible error. These results highlight considerations likely to be relevant to the design, evaluation, and oversight of explainable AI systems.