The Challenge of Crafting Intelligible Intelligence
This tackles the challenge of AI interpretability for organizations deploying AI in critical settings, but it is incremental as it surveys existing work without presenting new results.
The paper addresses the problem of AI systems being difficult to understand due to complex techniques like deep learning, and argues that making AI intelligible is essential for trust in mission-critical applications, surveying recent work and highlighting research directions.
Since Artificial Intelligence (AI) software uses techniques like deep lookahead search and stochastic optimization of huge neural networks to fit mammoth datasets, it often results in complex behavior that is difficult for people to understand. Yet organizations are deploying AI algorithms in many mission-critical settings. To trust their behavior, we must make AI intelligible, either by using inherently interpretable models or by developing new methods for explaining and controlling otherwise overwhelmingly complex decisions using local approximation, vocabulary alignment, and interactive explanation. This paper argues that intelligibility is essential, surveys recent work on building such systems, and highlights key directions for research.