Bifeng Song

Lei He, Aouf Nabil, Bifeng Song

Autonomous navigation in unknown complex environment is still a hard problem, especially for small Unmanned Aerial Vehicles (UAVs) with limited computation resources. In this paper, a neural network-based reactive controller is proposed for a quadrotor to fly autonomously in unknown outdoor environment. The navigation controller makes use of only current sensor data to generate the control signal without any optimization or configuration space searching, which reduces both memory and computation requirement. The navigation problem is modelled as a Markov Decision Process (MDP) and solved using deep reinforcement learning (DRL) method. Specifically, to get better understanding of the trained network, some model explanation methods are proposed. Based on the feature attribution, each decision making result during flight is explained using both visual and texture explanation. Moreover, some global analysis are also provided for experts to evaluate and improve the trained neural network. The simulation results illustrated the proposed method can make useful and reasonable explanation for the trained model, which is beneficial for both non-expert users and controller designer. Finally, the real world tests shown the proposed controller can navigate the quadrotor to goal position successfully and the reactive controller performs much faster than some conventional approach under the same computation resource.

Lei He, Nabil Aouf, James F. Whidborne et al.

In this paper, a deep reinforcement learning (DRL) method is proposed to address the problem of UAV navigation in an unknown environment. However, DRL algorithms are limited by the data efficiency problem as they typically require a huge amount of data before they reach a reasonable performance. To speed up the DRL training process, we developed a novel learning framework which combines imitation learning and reinforcement learning and building upon Twin Delayed DDPG (TD3) algorithm. We newly introduced both policy and Q-value network are learned using the expert demonstration during the imitation phase. To tackle the distribution mismatch problem transfer from imitation to reinforcement learning, both TD-error and decayed imitation loss are used to update the pre-trained network when start interacting with the environment. The performances of the proposed algorithm are demonstrated on the challenging 3D UAV navigation problem using depth cameras and sketched in a variety of simulation environments.