Yuanzhe Geng

Yuanzhe Geng, Erwu Liu, Rui Wang et al.

Cooperative communication is an effective approach to improve spectrum utilization. In order to reduce outage probability of communication system, most studies propose various schemes for relay selection and power allocation, which are based on the assumption of channel state information (CSI). However, it is difficult to get an accurate CSI in practice. In this paper, we study the outage probability minimizing problem subjected to a total transmission power constraint in a two-hop cooperative relay network. We use reinforcement learning (RL) methods to learn strategies for relay selection and power allocation, which do not need any prior knowledge of CSI but simply rely on the interaction with communication environment. It is noted that conventional RL methods, including most deep reinforcement learning (DRL) methods, cannot perform well when the search space is too large. Therefore, we first propose a DRL framework with an outage-based reward function, which is then used as a baseline. Then, we further propose a hierarchical reinforcement learning (HRL) framework and training algorithm. A key difference from other RL-based methods in existing literatures is that, our proposed HRL approach decomposes relay selection and power allocation into two hierarchical optimization objectives, which are trained in different levels. With the simplification of search space, the HRL approach can solve the problem of sparse reward, while the conventional RL method fails. Simulation results reveal that compared with traditional DRL method, the HRL training algorithm can reach convergence 30 training iterations earlier and reduce the outage probability by 5% in two-hop relay network with the same outage threshold.

Yuanzhe Geng, Erwu Liu, Rui Wang et al.

Route planning is important in transportation. Existing works focus on finding the shortest path solution or using metrics such as safety and energy consumption to determine the planning. It is noted that most of these studies rely on prior knowledge of road network, which may be not available in certain situations. In this paper, we design a route planning algorithm based on deep reinforcement learning (DRL) for pedestrians. We use travel time consumption as the metric, and plan the route by predicting pedestrian flow in the road network. We put an agent, which is an intelligent robot, on a virtual map. Different from previous studies, our approach assumes that the agent does not need any prior information about road network, but simply relies on the interaction with the environment. We propose a dynamically adjustable route planning (DARP) algorithm, where the agent learns strategies through a dueling deep Q network to avoid congested roads. Simulation results show that the DARP algorithm saves 52% of the time under congestion condition when compared with traditional shortest path planning algorithms.