Balancing Interpretability and Performance in Reinforcement Learning: An Adaptive Spectral Based Linear Approach
This work addresses the need for interpretable and high-performance RL in practical decision-making contexts like management, though it is incremental as it builds on existing ridge regression approaches.
The authors tackled the challenge of balancing interpretability and performance in reinforcement learning by proposing a spectral-based linear RL method with adaptive regularization, achieving near-optimal theoretical bounds and matching or outperforming baselines in decision quality on simulated and real-world datasets from Kuaishou and Taobao.
Reinforcement learning (RL) has been widely applied to sequential decision making, where interpretability and performance are both critical for practical adoption. Current approaches typically focus on performance and rely on post hoc explanations to account for interpretability. Different from these approaches, we focus on designing an interpretability-oriented yet performance-enhanced RL approach. Specifically, we propose a spectral based linear RL method that extends the ridge regression-based approach through a spectral filter function. The proposed method clarifies the role of regularization in controlling estimation error and further enables the design of an adaptive regularization parameter selection strategy guided by the bias-variance trade-off principle. Theoretical analysis establishes near-optimal bounds for both parameter estimation and generalization error. Extensive experiments on simulated environments and real-world datasets from Kuaishou and Taobao demonstrate that our method either outperforms or matches existing baselines in decision quality. We also conduct interpretability analyses to illustrate how the learned policies make decisions, thereby enhancing user trust. These results highlight the potential of our approach to bridge the gap between RL theory and practical decision making, providing interpretability, accuracy, and adaptability in management contexts.