Measuring Competency of Machine Learning Systems and Enforcing Reliability
This work addresses reliability issues for ML systems in safety-critical applications like autonomous driving, though it appears incremental as it builds on existing competency assessment methods.
The paper tackled the problem of how environmental conditions affect machine learning agents' competency, showing that real-time competency assessments improve reliability in a simulated self-driving vehicle's obstacle avoidance task using a convolutional neural network.
We explore the impact of environmental conditions on the competency of machine learning agents and how real-time competency assessments improve the reliability of ML agents. We learn a representation of conditions which impact the strategies and performance of the ML agent enabling determination of actions the agent can make to maintain operator expectations in the case of a convolutional neural network that leverages visual imagery to aid in the obstacle avoidance task of a simulated self-driving vehicle.