Norman C. Beaulieu

Shaoyang Wang, Tiejun Lv, Wei Ni et al.

This paper presents a novel and effective deep reinforcement learning (DRL)-based approach to addressing joint resource management (JRM) in a practical multi-carrier non-orthogonal multiple access (MC-NOMA) system, where hardware sensitivity and imperfect successive interference cancellation (SIC) are considered. We first formulate the JRM problem to maximize the weighted-sum system throughput. Then, the JRM problem is decoupled into two iterative subtasks: subcarrier assignment (SA, including user grouping) and power allocation (PA). Each subtask is a sequential decision process. Invoking a deep deterministic policy gradient algorithm, our proposed DRL-based JRM (DRL-JRM) approach jointly performs the two subtasks, where the optimization objective and constraints of the subtasks are addressed by a new joint reward and internal reward mechanism. A multi-agent structure and a convolutional neural network are adopted to reduce the complexity of the PA subtask. We also tailor the neural network structure for the stability and convergence of DRL-JRM. Corroborated by extensive experiments, the proposed DRL-JRM scheme is superior to existing alternatives in terms of system throughput and resistance to interference, especially in the presence of many users and strong inter-cell interference. DRL-JRM can flexibly meet individual service requirements of users.

Xuewei Zhang, Tiejun Lv, Yuan Ren et al.

Aiming to minimize service delay, we propose a new random caching scheme in device-to-device (D2D)-assisted heterogeneous network. To support diversified viewing qualities of multimedia video services, each video file is encoded into a base layer (BL) and multiple enhancement layers (ELs) by scalable video coding (SVC). A super layer, including the BL and several ELs, is transmitted to every user. We define and quantify the service delay of multi-quality videos by deriving successful transmission probabilities when a user is served by a D2D helper, a small-cell base station (SBS) and a macro-cell base station (MBS). We formulate a delay minimization problem subject to the limited cache sizes of D2D helpers and SBSs. The structure of the optimal solutions to the problem is revealed, and then an improved standard gradient projection method is designed to effectively obtain the solutions. Both theoretical analysis and Monte-Carlo simulations validate the successful transmission probabilities. Compared with three benchmark caching policies, the proposed SVC-based random caching scheme is superior in terms of reducing the service delay.

Xuewei Zhang, Tiejun Lv, Yuan Ren et al.

We develop the optimal economical caching schemes in cache-enabled heterogeneous networks, while delivering multimedia video services with personalized viewing qualities to mobile users. By applying scalable video coding (SVC), each video file to be requested is divided into one base layer (BL) and several enhancement layers (ELs). In order to assign different transmission tasks, the serving small-cell base stations (SBSs) are grouped into K clusters. The SBSs are able to cache and cooperatively transmit BL and EL contents to the user. We analytically derive the expressions for successful transmission probability and ergodic service rate, and then the closed form of EConomical Efficiency (ECE) is obtained. In order to enhance the ECE performance, we formulate the ECE optimization problems for two cases. In the first case, with equal cache size equipped at each SBS, the layer caching indicator is determined. Since this problem is NP-hard, after the l0-norm approximation, the discrete optimization variables are relaxed to be continuous, and this relaxed problem is convex. Next, based on the optimal solution derived from the relaxed problem, we devise a greedystrategy based heuristic algorithm to achieve the near-optimal layer caching indicators. In the second case, the cache size for each SBS, the layer size and the layer caching indicator are jointly optimized. This problem is a mixed integer programming problem, which is more challenging. To effectively solve this problem, the original ECE maximization problem is divided into two subproblems. These two subproblems are iteratively solved until the original optimization problem is convergent. Numerical results verify the correctness of theoretical derivations. Additionally, compared to the most popular layer placement strategy, the performance superiority of the proposed SVC-based caching schemes is testified.