Is explainable AI a race against model complexity?
This addresses the fundamental tension between model complexity and explainability in AI, highlighting potential limits for researchers and practitioners.
The paper argues that as AI models become larger and more complex, providing explanations for their predictions may become intractably challenging, and that models can remain useful even when explanations are incomplete or incorrect.
Explaining the behaviour of intelligent systems will get increasingly and perhaps intractably challenging as models grow in size and complexity. We may not be able to expect an explanation for every prediction made by a brain-scale model, nor can we expect explanations to remain objective or apolitical. Our functionalist understanding of these models is of less advantage than we might assume. Models precede explanations, and can be useful even when both model and explanation are incorrect. Explainability may never win the race against complexity, but this is less problematic than it seems.