Visualizing the Loss Landscape of Actor Critic Methods with Applications in Inventory Optimization
This work provides insights into optimization for continuous control, particularly in supply chain operations, but is incremental as it focuses on visualization and application of existing methods.
The study visualized the loss landscapes of actor-critic methods in reinforcement learning, revealing characteristics of the actor loss function, and applied this to multi-store dynamic inventory control, achieving optimal policy shapes.
Continuous control is a widely applicable area of reinforcement learning. The main players of this area are actor-critic methods that utilize policy gradients of neural approximators as a common practice. The focus of our study is to show the characteristics of the actor loss function which is the essential part of the optimization. We exploit low dimensional visualizations of the loss function and provide comparisons for loss landscapes of various algorithms. Furthermore, we apply our approach to multi-store dynamic inventory control, a notoriously difficult problem in supply chain operations, and explore the shape of the loss function associated with the optimal policy. We modelled and solved the problem using reinforcement learning while having a loss landscape in favor of optimality.