Age of Information Aware VNF Scheduling in Industrial IoT Using Deep Reinforcement Learning
This work addresses delay-sensitive industrial IoT applications by optimizing VNF scheduling to improve information freshness and reduce costs, though it is incremental as it extends existing DRL methods to a multi-agent setting.
The paper tackles the NP-hard problem of VNF placement and scheduling in industrial IoT to minimize network cost and age of information under QoS constraints, using deep reinforcement learning; simulation results show that single-agent schemes outperform greedy algorithms, and multi-agent solutions reduce average cost but require more iterations for learning.
In delay-sensitive industrial internet of things (IIoT) applications, the age of information (AoI) is employed to characterize the freshness of information. Meanwhile, the emerging network function virtualization provides flexibility and agility for service providers to deliver a given network service using a sequence of virtual network functions (VNFs). However, suitable VNF placement and scheduling in these schemes is NP-hard and finding a globally optimal solution by traditional approaches is complex. Recently, deep reinforcement learning (DRL) has appeared as a viable way to solve such problems. In this paper, we first utilize single agent low-complex compound action actor-critic RL to cover both discrete and continuous actions and jointly minimize VNF cost and AoI in terms of network resources under end-to end Quality of Service constraints. To surmount the single-agent capacity limitation for learning, we then extend our solution to a multi-agent DRL scheme in which agents collaborate with each other. Simulation results demonstrate that single-agent schemes significantly outperform the greedy algorithm in terms of average network cost and AoI. Moreover, multi-agent solution decreases the average cost by dividing the tasks between the agents. However, it needs more iterations to be learned due to the requirement on the agents collaboration.