Yiming Liu

Yuanzhe Geng, Erwu Liu, Rui Wang et al.

Cooperative communication is an effective approach to improve spectrum utilization. In order to reduce outage probability of communication system, most studies propose various schemes for relay selection and power allocation, which are based on the assumption of channel state information (CSI). However, it is difficult to get an accurate CSI in practice. In this paper, we study the outage probability minimizing problem subjected to a total transmission power constraint in a two-hop cooperative relay network. We use reinforcement learning (RL) methods to learn strategies for relay selection and power allocation, which do not need any prior knowledge of CSI but simply rely on the interaction with communication environment. It is noted that conventional RL methods, including most deep reinforcement learning (DRL) methods, cannot perform well when the search space is too large. Therefore, we first propose a DRL framework with an outage-based reward function, which is then used as a baseline. Then, we further propose a hierarchical reinforcement learning (HRL) framework and training algorithm. A key difference from other RL-based methods in existing literatures is that, our proposed HRL approach decomposes relay selection and power allocation into two hierarchical optimization objectives, which are trained in different levels. With the simplification of search space, the HRL approach can solve the problem of sparse reward, while the conventional RL method fails. Simulation results reveal that compared with traditional DRL method, the HRL training algorithm can reach convergence 30 training iterations earlier and reduce the outage probability by 5% in two-hop relay network with the same outage threshold.

Yuanzhe Geng, Erwu Liu, Rui Wang et al.

Route planning is important in transportation. Existing works focus on finding the shortest path solution or using metrics such as safety and energy consumption to determine the planning. It is noted that most of these studies rely on prior knowledge of road network, which may be not available in certain situations. In this paper, we design a route planning algorithm based on deep reinforcement learning (DRL) for pedestrians. We use travel time consumption as the metric, and plan the route by predicting pedestrian flow in the road network. We put an agent, which is an intelligent robot, on a virtual map. Different from previous studies, our approach assumes that the agent does not need any prior information about road network, but simply relies on the interaction with the environment. We propose a dynamically adjustable route planning (DARP) algorithm, where the agent learns strategies through a dueling deep Q network to avoid congested roads. Simulation results show that the DARP algorithm saves 52% of the time under congestion condition when compared with traditional shortest path planning algorithms.

Yanqi Zhao, Yiming Liu, Yong Yu et al.

A large number of IoT devices are connected via the Internet. However, most of these IoT devices are generally not perfect-by-design even have security weaknesses or vulnerabilities. Thus, it is essential to update these IoT devices securely, patching their vulnerabilities and protecting the safety of the involved users. Existing studies deliver secure and reliable updates based on blockchain network which serves as the transmission network. However, these approaches could compromise users privacy when updating the IoT devices. In this paper, we propose a new blockchain based privacy-preserving software updates protocol, which delivers secure and reliable updates with an incentive mechanism, as well protects the privacy of involved users. The vendor delivers the updates and it makes a commitment by using a smart contract to provide financial incentive to the transmission nodes who deliver the updates to the IoT devices. A transmission node gets financial incentive by providing a proof-of-delivery. The transmission node uses double authentication preventing signature (DAPS) to carry out the fair exchange to obtain the proof-of-delivery. Specifically, the transmission node exchanges an attribute-based signature from a IoT device by using DAPS. Then, it uses the attribute-based signature as a proof-of-delivery to receive financial incentives. Generally, the IoT device has to execute complex computation for an attribute-based signature (ABS). It is intolerable for resource limited devices. We propose a concrete outsourced attribute-based signature (OABS) scheme to resist the weakness. Then, we prove the security of the proposed OABS and the protocol as well. Finally, we implement smart contract in Solidity to demonstrate the validity of the proposed protocol.