Optimal Admission Control for Multiclass Queues with Time-Varying Arrival Rates via State Abstraction
This work addresses optimal admission control for queuing systems with time-varying arrivals, which is incremental as it builds on existing MDP methods but applies them to a novel multiclass setting with practical applications like financial fraud detection.
The authors tackled the problem of maximizing total price from processed tasks in a multiclass queue with time-varying arrivals by formulating it as a Markov Decision Process and solving it exactly, showing that their discrete-time solution approaches the continuous-time optimum as the time step is reduced, and they validated it on synthetic and real financial fraud data.
We consider a novel queuing problem where the decision-maker must choose to accept or reject randomly arriving tasks into a no buffer queue which are processed by $N$ identical servers. Each task has a price, which is a positive real number, and a class. Each class of task has a different price distribution and service rate, and arrives according to an inhomogenous Poisson process. The objective is to decide which tasks to accept so that the total price of tasks processed is maximised over a finite horizon. We formulate the problem as a discrete time Markov Decision Process (MDP) with a hybrid state space. We show that the optimal value function has a specific structure, which enables us to solve the hybrid MDP exactly. Moreover, we prove that as the time step is reduced, the discrete time solution approaches the optimal solution to the original continuous time problem. To improve the scalability of our approach to a greater number of task classes, we present an approximation based on state abstraction. We validate our approach on synthetic data, as well as a real financial fraud data set, which is the motivating application for this work.