Shivendra S. Panwar

Shufan Wang, Guojun Xiong, Shichen Zhang et al.

We study the data packet transmission problem (mmDPT) in dense cell-free millimeter wave (mmWave) networks, i.e., users sending data packet requests to access points (APs) via uplinks and APs transmitting requested data packets to users via downlinks. Our objective is to minimize the average delay in the system due to APs' limited service capacity and unreliable wireless channels between APs and users. This problem can be formulated as a restless multi-armed bandits problem with fairness constraint (RMAB-F). Since finding the optimal policy for RMAB-F is intractable, existing learning algorithms are computationally expensive and not suitable for practical dynamic dense mmWave networks. In this paper, we propose a structured reinforcement learning (RL) solution for mmDPT by exploiting the inherent structure encoded in RMAB-F. To achieve this, we first design a low-complexity and provably asymptotically optimal index policy for RMAB-F. Then, we leverage this structure information to develop a structured RL algorithm called mmDPT-TS, which provably achieves an \tilde{O}(\sqrt{T}) Bayesian regret. More importantly, mmDPT-TS is computation-efficient and thus amenable to practical implementation, as it fully exploits the structure of index policy for making decisions. Extensive emulation based on data collected in realistic mmWave networks demonstrate significant gains of mmDPT-TS over existing approaches.

Shenghe Xu, Murali Kodialam, T. V. Lakshman et al.

Network traffic demand matrix is a critical input for capacity planning, anomaly detection and many other network management related tasks. The demand matrix is often computed from link load measurements. The traffic matrix (TM) estimation problem is the determination of the traffic demand matrix from link load measurements. The relationship between the link loads and the traffic matrix that generated the link load can be modeled as an under-determined linear system and has multiple feasible solutions. Therefore, prior knowledge of the traffic demand pattern has to be used in order to find a potentially feasible demand matrix. In this paper, we consider the TM estimation problem where we have information about the distribution of the demand sizes. This information can be obtained from the analysis of a few traffic matrices measured in the past or from operator experience. We develop an iterative projection based algorithm for the solution of this problem. If large number of past traffic matrices are accessible, we propose a Generative Adversarial Network (GAN) based approach for solving the problem. We compare the strengths of the two approaches and evaluate their performance for several networks using varying amounts of past data.

Shenghe Xu, Murali Kodialam, T. V. Lakshman et al.

Applications such as virtual reality and online gaming require low delays for acceptable user experience. A key task for over-the-top (OTT) service providers who provide these applications is sending traffic through the networks to minimize delays. OTT traffic is typically generated from multiple data centers which are multi-homed to several network ingresses. However, information about the path characteristics of the underlying network from the ingresses to destinations is not explicitly available to OTT services. These can only be inferred from external probing. In this paper, we combine network tomography with machine learning to minimize delays. We consider this problem in a general setting where traffic sources can choose a set of ingresses through which their traffic enter a black box network. The problem in this setting can be viewed as a reinforcement learning problem with constraints on a continuous action space, which to the best of our knowledge have not been investigated by the machine learning community. Key technical challenges to solving this problem include the high dimensionality of the problem and handling constraints that are intrinsic to networks. Evaluation results show that our methods achieve up to 60% delay reductions in comparison to standard heuristics. Moreover, the methods we develop can be used in a centralized manner or in a distributed manner by multiple independent agents.

Shenghe Xu, Pei Liu, Ran Wang et al.

With the increasing number of base stations (BSs) and network densification in 5G, interference management using link scheduling and power control are vital for better utilization of radio resources. However, the complexity of solving link scheduling and the power control problem grows exponentially with the number of BS. Due to high computation time, previous methods are useful for research purposes but impractical for real time usage. In this paper we propose to use deep neural networks (DNNs) to approximate optimal link scheduling and power control for the case with multiple small cells. A deep Q-network (DQN) estimates a suitable schedule, then a DNN allocates power for the corresponding schedule. Simulation results show that the proposed method achieves over five orders of magnitude speed-up with less than nine percent performance loss, making real time usage practical.

Rajeev Kumar, Robert S Margolies, Rittwik Jana et al.

With the evolution of HDTV and Ultra HDTV, the bandwidth requirement for IP-based TV content is rapidly increasing. Consumers demand uninterrupted service with a high Quality of Experience (QoE). Service providers are constantly trying to differentiate themselves by innovating new ways of distributing content more efficiently with lower cost and higher penetration. In this work, we propose a cost-efficient wireless framework (WiLiTV) for delivering live TV services, consisting of a mix of wireless access technologies (e.g. Satellite, WiFi and LTE overlay links). In the proposed architecture, live TV content is injected into the network at a few residential locations using satellite dishes. The content is then further distributed to other homes using a house-to-house WiFi network or via an overlay LTE network. Our problem is to construct an optimal TV distribution network with the minimum number of satellite injection points, while preserving the highest QoE, for different neighborhood densities. We evaluate the framework using realistic time-varying demand patterns and a diverse set of home location data. Our study demonstrates that the architecture requires 75 - 90% fewer satellite injection points, compared to traditional architectures. Furthermore, we show that most cost savings can be obtained using simple and practical relay routing solutions.

Zhili Guo, Yao Wang, Elza Erkip et al.

In this paper, a cooperative multicast scheme that uses Randomized Distributed Space Time Codes (R-DSTC), along with packet level Forward Error Correction (FEC), is studied. Instead of sending source packets and/or parity packets through two hops using R-DSTC as proposed in our prior work, the new scheme delivers both source packets and parity packets using only one hop. After the source station (access point, AP) first sends all the source packets, the AP as well as all nodes that have received all source packets together send the parity packets using R-DSTC. As more parity packets are transmitted, more nodes can recover all source packets and join the parity packet transmission. The process continues until all nodes acknowledge the receipt of enough packets for recovering the source packets. For each given node distribution, the optimum transmission rates for source and parity packets are determined such that the video rate that can be sustained at all nodes is maximized. This new scheme can support significantly higher video rates, and correspondingly higher PSNR of decoded video, than the prior approaches. Three suboptimal approaches, which do not require full information about user distribution or the feedback, and hence are more feasible in practice are also presented. The proposed suboptimal scheme with only the node count information and without feedback still outperforms our prior approach that assumes full channel information and no feedback.