Kun Chen

Kun Chen, Longbo Huang

Motivated by the increasing importance of providing delay-guaranteed services in general computing and communication systems, and the recent wide adoption of learning and prediction in network control, in this work, we consider a general stochastic single-server multi-user system and investigate the fundamental benefit of predictive scheduling in improving timely-throughput, being the rate of packets that are delivered to destinations before their deadlines. By adopting an error rate-based prediction model, we first derive a Markov decision process (MDP) solution to optimize the timely-throughput objective subject to an average resource consumption constraint. Based on a packet-level decomposition of the MDP, we explicitly characterize the optimal scheduling policy and rigorously quantify the timely-throughput improvement due to predictive-service, which scales as $Θ(p\left[C_{1}\frac{(a-a_{\max}q)}{p-q}ρ^τ+C_{2}(1-\frac{1}{p})\right](1-ρ^{D}))$, where $a, a_{\max}, ρ\in(0, 1), C_1>0, C_2\ge0$ are constants, $p$ is the true-positive rate in prediction, $q$ is the false-negative rate, $τ$ is the packet deadline and $D$ is the prediction window size. We also conduct extensive simulations to validate our theoretical findings. Our results provide novel insights into how prediction and system parameters impact performance and provide useful guidelines for designing predictive low-latency control algorithms.

Kechao Cai, Kun Chen, Longbo Huang et al.

Selecting the right web links for a website is important because appropriate links not only can provide high attractiveness but can also increase the website's revenue. In this work, we first show that web links have an intrinsic \emph{multi-level feedback structure}. For example, consider a $2$-level feedback web link: the $1$st level feedback provides the Click-Through Rate (CTR) and the $2$nd level feedback provides the potential revenue, which collectively produce the compound $2$-level revenue. We consider the context-free links selection problem of selecting links for a homepage so as to maximize the total compound $2$-level revenue while keeping the total $1$st level feedback above a preset threshold. We further generalize the problem to links with $n~(n\ge2)$-level feedback structure. The key challenge is that the links' multi-level feedback structures are unobservable unless the links are selected on the homepage. To our best knowledge, we are the first to model the links selection problem as a constrained multi-armed bandit problem and design an effective links selection algorithm by learning the links' multi-level structure with provable \emph{sub-linear} regret and violation bounds. We uncover the multi-level feedback structures of web links in two real-world datasets. We also conduct extensive experiments on the datasets to compare our proposed \textbf{LExp} algorithm with two state-of-the-art context-free bandit algorithms and show that \textbf{LExp} algorithm is the most effective in links selection while satisfying the constraint.