RIO: Minimizing User Interaction in Debugging of Knowledge Bases
This work addresses the challenge of efficient debugging for users of knowledge bases where reliable prior fault estimates are hard to obtain, representing an incremental improvement over existing interactive systems.
The paper tackled the problem of minimizing user interaction in debugging knowledge bases by introducing a reinforcement learning strategy that adapts based on performance to reduce reliance on potentially misleading meta-information. The result showed that the method outperformed entropy-based and no-risk strategies in reducing required user interaction, as demonstrated on diverse real-world knowledge bases with scalability and decent reaction time.
The best currently known interactive debugging systems rely upon some meta-information in terms of fault probabilities in order to improve their efficiency. However, misleading meta information might result in a dramatic decrease of the performance and its assessment is only possible a-posteriori. Consequently, as long as the actual fault is unknown, there is always some risk of suboptimal interactions. In this work we present a reinforcement learning strategy that continuously adapts its behavior depending on the performance achieved and minimizes the risk of using low-quality meta information. Therefore, this method is suitable for application scenarios where reliable prior fault estimates are difficult to obtain. Using diverse real-world knowledge bases, we show that the proposed interactive query strategy is scalable, features decent reaction time, and outperforms both entropy-based and no-risk strategies on average w.r.t. required amount of user interaction.