Explainable Knowledge Graph Embedding: Inference Reconciliation for Knowledge Inferences Supporting Robot Actions
This addresses the need for explainable AI in robotics for non-experts, though it is incremental as it builds on existing knowledge graph embedding and interpretability methods.
The paper tackles the problem of explaining how knowledge graph embeddings affect robot decision-making by proposing an interpretable model that uses decision trees to approximate black-box predictions and provide natural language explanations. Results show the model enables non-experts to correct erratic robot behaviors due to nonsensical beliefs in the black-box.
Learned knowledge graph representations supporting robots contain a wealth of domain knowledge that drives robot behavior. However, there does not exist an inference reconciliation framework that expresses how a knowledge graph representation affects a robot's sequential decision making. We use a pedagogical approach to explain the inferences of a learned, black-box knowledge graph representation, a knowledge graph embedding. Our interpretable model, uses a decision tree classifier to locally approximate the predictions of the black-box model, and provides natural language explanations interpretable by non-experts. Results from our algorithmic evaluation affirm our model design choices, and the results of our user studies with non-experts support the need for the proposed inference reconciliation framework. Critically, results from our simulated robot evaluation indicate that our explanations enable non-experts to correct erratic robot behaviors due to nonsensical beliefs within the black-box.