Graph Neural Networks and Reinforcement Learning for Behavior Generation in Semantic Environments
This addresses a specific issue in autonomous driving and simulation for researchers and engineers, but it is incremental as it adapts existing methods to a known bottleneck.
The paper tackled the problem of reinforcement learning in semantic environments where vectorial inputs require fixed sizes and lack invariance to vehicle order and count, by proposing graph neural networks with actor-critic reinforcement learning. They demonstrated in a highway lane-change scenario that graph neural networks can handle varying numbers and orders of vehicles during training and application.
Most reinforcement learning approaches used in behavior generation utilize vectorial information as input. However, this requires the network to have a pre-defined input-size -- in semantic environments this means assuming the maximum number of vehicles. Additionally, this vectorial representation is not invariant to the order and number of vehicles. To mitigate the above-stated disadvantages, we propose combining graph neural networks with actor-critic reinforcement learning. As graph neural networks apply the same network to every vehicle and aggregate incoming edge information, they are invariant to the number and order of vehicles. This makes them ideal candidates to be used as networks in semantic environments -- environments consisting of objects lists. Graph neural networks exhibit some other advantages that make them favorable to be used in semantic environments. The relational information is explicitly given and does not have to be inferred. Moreover, graph neural networks propagate information through the network and can gather higher-degree information. We demonstrate our approach using a highway lane-change scenario and compare the performance of graph neural networks to conventional ones. We show that graph neural networks are capable of handling scenarios with a varying number and order of vehicles during training and application.