Eray Unsal Atay

Eray Unsal Atay, Venkat Chandrasekaran, Victoria Kostina

We study the rate-cost tradeoff in rate-limited control of general stochastic control systems, including nonlinear systems, over a finite horizon. At each time step, an encoder observes the state and transmits a description to a controller, which then selects the control action. For an average control-cost threshold $D$, we characterize the minimum achievable communication rate $R_n(D)$ via a nonasymptotic bound: $R_n(D)$ lies within an additive logarithmic gap of the optimal value of a directed-information minimization $F_n(D)$, namely, we show that $F_n(D) \le R_n(D) \le F_n(D)+\log \bigl(F_n(D)+3.4\bigr)+2+\frac{1}{n}$, in bits. This establishes directed information as the operationally relevant quantity governing rate-limited control, thereby broadening its utility beyond its previously established roles in causal source coding and linear quadratic Gaussian (LQG) control to general nonlinear control systems. We prove the upper bound constructively by building an encoding-and-control policy using the strong functional representation lemma at each time step. As special cases of our setting, our framework yields nonasymptotic bounds for sequential (causal) rate-distortion and LQG control.

Eray Unsal Atay, Igor Kadota, Eytan Modiano

We consider a single-hop wireless network with sources transmitting time-sensitive information to the destination over multiple unreliable channels. Packets from each source are generated according to a stochastic process with known statistics and the state of each wireless channel (ON/OFF) varies according to a stochastic process with unknown statistics. The reliability of the wireless channels is to be learned through observation. At every time slot, the learning algorithm selects a single pair (source, channel) and the selected source attempts to transmit its packet via the selected channel. The probability of a successful transmission to the destination depends on the reliability of the selected channel. The goal of the learning algorithm is to minimize the Age-of-Information (AoI) in the network over $T$ time slots. To analyze the performance of the learning algorithm, we introduce the notion of AoI regret, which is the difference between the expected cumulative AoI of the learning algorithm under consideration and the expected cumulative AoI of a genie algorithm that knows the reliability of the channels a priori. The AoI regret captures the penalty incurred by having to learn the statistics of the channels over the $T$ time slots. The results are two-fold: first, we consider learning algorithms that employ well-known solutions to the stochastic multi-armed bandit problem (such as $ε$-Greedy, Upper Confidence Bound, and Thompson Sampling) and show that their AoI regret scales as $Θ(\log T)$; second, we develop a novel learning algorithm and show that it has $O(1)$ regret. To the best of our knowledge, this is the first learning algorithm with bounded AoI regret.