Improving Competence for Reliable Autonomy
This addresses reliability issues in semi-autonomous systems for domains like robotics or AI, but it is incremental as it builds on existing competence-aware systems.
The paper tackles the problem of autonomous systems lacking features to handle all real-world scenarios by proposing a method for online competence improvement through feedback from human authority, resulting in better prediction of human involvement and improved performance.
Given the complexity of real-world, unstructured domains, it is often impossible or impractical to design models that include every feature needed to handle all possible scenarios that an autonomous system may encounter. For an autonomous system to be reliable in such domains, it should have the ability to improve its competence online. In this paper, we propose a method for improving the competence of a system over the course of its deployment. We specifically focus on a class of semi-autonomous systems known as competence-aware systems that model their own competence -- the optimal extent of autonomy to use in any given situation -- and learn this competence over time from feedback received through interactions with a human authority. Our method exploits such feedback to identify important state features missing from the system's initial model, and incorporates them into its state representation. The result is an agent that better predicts human involvement, leading to improvements in its competence and reliability, and as a result, its overall performance.