Learning Partially Structured Environmental Dynamics for Marine Robotic Navigation
This addresses the challenge of controlling marine vehicles in non-static, partially structured ocean environments, which is incremental as it applies existing deep reinforcement learning to a specific domain with known bottlenecks.
The paper tackles the problem of marine robotic navigation in dynamic ocean environments with strong disturbances by proposing a deep reinforcement learning method to learn partially structured environmental dynamics, resulting in the robot successfully acting in complex spatiotemporal environments under both artificial and real ocean disturbances.
We investigate the scenario that a robot needs to reach a designated goal after taking a sequence of appropriate actions in a non-static environment that is partially structured. One application example is to control a marine vehicle to move in the ocean. The ocean environment is dynamic and oftentimes the ocean waves result in strong disturbances that can disturb the vehicle's motion. Modeling such dynamic environment is non-trivial, and integrating such model in the robotic motion control is particularly difficult. Fortunately, the ocean currents usually form some local patterns (e.g. vortex) and thus the environment is partially structured. The historically observed data can be used to train the robot to learn to interact with the ocean tidal disturbances. In this paper we propose a method that applies the deep reinforcement learning framework to learn such partially structured complex disturbances. Our results show that, by training the robot under artificial and real ocean disturbances, the robot is able to successfully act in complex and spatiotemporal environments.