Reinforcement Learning-based Admission Control in Delay-sensitive Service Systems
This addresses the challenge of providing delay guarantees in service systems like service function chains, which is incremental as it applies RL to a known bottleneck in admission control.
The paper tackles the problem of ensuring quality of service in delay-sensitive service systems by proposing a reinforcement learning-based admission controller that guarantees a probabilistic upper-bound on end-to-end delay while minimizing unnecessary rejections, using only queue length information without requiring network topology or system parameters.
Ensuring quality of service (QoS) guarantees in service systems is a challenging task, particularly when the system is composed of more fine-grained services, such as service function chains. An important QoS metric in service systems is the end-to-end delay, which becomes even more important in delay-sensitive applications, where the jobs must be completed within a time deadline. Admission control is one way of providing end-to-end delay guarantee, where the controller accepts a job only if it has a high probability of meeting the deadline. In this paper, we propose a reinforcement learning-based admission controller that guarantees a probabilistic upper-bound on the end-to-end delay of the service system, while minimizes the probability of unnecessary rejections. Our controller only uses the queue length information of the network and requires no knowledge about the network topology or system parameters. Since long-term performance metrics are of great importance in service systems, we take an average-reward reinforcement learning approach, which is well suited to infinite horizon problems. Our evaluations verify that the proposed RL-based admission controller is capable of providing probabilistic bounds on the end-to-end delay of the network, without using system model information.