Deep Distilling: automated code generation using explainable deep learning
This addresses the challenge of making AI more interpretable and capable in domains requiring human understanding, such as science or common sense reasoning, though it appears incremental as it builds on existing explainable deep learning techniques.
The paper tackles the problem of AI systems lacking explainability and symbolic reasoning by introducing deep distilling, a method that learns patterns from data using explainable deep learning and condenses them into concise, executable code; it shows that this code generalizes out-of-distribution to solve problems orders-of-magnitude larger and more complex than training data, and for problems with known ground-truth rules, it discovers the rule set exactly.
Human reasoning can distill principles from observed patterns and generalize them to explain and solve novel problems. The most powerful artificial intelligence systems lack explainability and symbolic reasoning ability, and have therefore not achieved supremacy in domains requiring human understanding, such as science or common sense reasoning. Here we introduce deep distilling, a machine learning method that learns patterns from data using explainable deep learning and then condenses it into concise, executable computer code. The code, which can contain loops, nested logical statements, and useful intermediate variables, is equivalent to the neural network but is generally orders of magnitude more compact and human-comprehensible. On a diverse set of problems involving arithmetic, computer vision, and optimization, we show that deep distilling generates concise code that generalizes out-of-distribution to solve problems orders-of-magnitude larger and more complex than the training data. For problems with a known ground-truth rule set, deep distilling discovers the rule set exactly with scalable guarantees. For problems that are ambiguous or computationally intractable, the distilled rules are similar to existing human-derived algorithms and perform at par or better. Our approach demonstrates that unassisted machine intelligence can build generalizable and intuitive rules explaining patterns in large datasets that would otherwise overwhelm human reasoning.