Beyond Freshness and Semantics: A Coupon-Collector Framework for Effective Status Updates
For designers of real-time control systems over unreliable wireless channels, this work provides a principled framework and learning algorithm to ensure that transmitted packets are actually useful for control, moving beyond traditional freshness metrics.
This paper addresses the problem of scheduling status updates in energy-constrained wireless systems where each sample has a stochastic expiration time based on plant dynamics. The authors propose a coupon-collector framework, prove the optimal policy is doubly thresholded, and develop a Structure-Aware Q-learning algorithm that achieves up to 50% higher reward than age-based baselines.
For status update systems operating over unreliable energy-constrained wireless channels, we address Weaver's long-standing Level-C question: do my packets actually improve the plant's behavior? Each fresh sample carries a stochastic expiration time -- governed by the plant's instability dynamics -- after which the information becomes useless for control. Casting the problem as a coupon-collector variant with expiring coupons, we (i) formulate a two-dimensional average-reward MDP, (ii) prove that the optimal schedule is doubly thresholded in the receiver's freshness timer and the sender's stored lifetime, (iii) derive a closed-form policy for deterministic lifetimes, and (iv) design a Structure-Aware Q-learning algorithm (SAQ) that learns the optimal policy without knowing the channel success probability or lifetime distribution. Simulations validate our theoretical predictions: SAQ matches optimal Value Iteration performance while converging significantly faster than baseline Q-learning, and expiration-aware scheduling achieves up to 50% higher reward than age-based baselines by adapting transmissions to state-dependent urgency -- thereby delivering Level-C effectiveness under tight resource constraints.