Variational Autoencoders for exteroceptive perception in reinforcement learning-based collision avoidance
This work addresses computational efficiency for marine autonomous systems, but it is incremental as it applies an existing method (VAEs) to a specific domain problem.
The paper tackled the high computational cost of Deep Reinforcement Learning (DRL) for marine collision avoidance by using Variational Autoencoders (VAEs) to compress sensor data into a low-dimensional latent encoding, resulting in improved agent performance in path-following and collision avoidance tasks within a stochastic simulation environment.
Modern control systems are increasingly turning to machine learning algorithms to augment their performance and adaptability. Within this context, Deep Reinforcement Learning (DRL) has emerged as a promising control framework, particularly in the domain of marine transportation. Its potential for autonomous marine applications lies in its ability to seamlessly combine path-following and collision avoidance with an arbitrary number of obstacles. However, current DRL algorithms require disproportionally large computational resources to find near-optimal policies compared to the posed control problem when the searchable parameter space becomes large. To combat this, our work delves into the application of Variational AutoEncoders (VAEs) to acquire a generalized, low-dimensional latent encoding of a high-fidelity range-finding sensor, which serves as the exteroceptive input to a DRL agent. The agent's performance, encompassing path-following and collision avoidance, is systematically tested and evaluated within a stochastic simulation environment, presenting a comprehensive exploration of our proposed approach in maritime control systems.