Deep Binary Reinforcement Learning for Scalable Verification
This work addresses the problem of scalable verification for safety-critical applications in AI, representing an incremental improvement by adapting existing BNN methods to RL.
The paper tackled the challenge of verifying safety properties in deep reinforcement learning policies by training binarized neural networks (BNNs) tailored for RL, resulting in verifiable policies for Atari environments with demonstrated robustness properties.
The use of neural networks as function approximators has enabled many advances in reinforcement learning (RL). The generalization power of neural networks combined with advances in RL algorithms has reignited the field of artificial intelligence. Despite their power, neural networks are considered black boxes, and their use in safety-critical settings remains a challenge. Recently, neural network verification has emerged as a way to certify safety properties of networks. Verification is a hard problem, and it is difficult to scale to large networks such as the ones used in deep reinforcement learning. We provide an approach to train RL policies that are more easily verifiable. We use binarized neural networks (BNNs), a type of network with mostly binary parameters. We present an RL algorithm tailored specifically for BNNs. After training BNNs for the Atari environments, we verify robustness properties.