Fuzzy Logic Theory-based Adaptive Reward Shaping for Robust Reinforcement Learning (FARS)
This work addresses the problem of sparse or fixed rewards in RL for high-dimensional, long-horizon tasks, offering a way to incorporate expert knowledge for more robust learning.
The paper introduces a fuzzy logic-based reward shaping method for reinforcement learning that adapts rewards based on agent state, improving exploration and stability. In autonomous drone racing benchmarks, the method achieves up to 5% higher success rates and faster convergence compared to non-fuzzy reward formulations.
Reinforcement learning (RL) often struggles in real-world tasks with high-dimensional state spaces and long horizons, where sparse or fixed rewards severely slow down exploration and cause agents to get trapped in local optima. This paper presents a fuzzy logic based reward shaping method that integrates human intuition into RL reward design. By encoding expert knowledge into adaptive and interpreable terms, fuzzy rules promote stable learning and reduce sensitivity to hyperparameters. The proposed method leverages these properties to adapt reward contributions based on the agent state, enabling smoother transitions between fast motion and precise control in challenging navigation tasks. Extensive simulation results on autonomous drone racing benchmarks show stable learning behavior and consistent task performance across scenarios of increasing difficulty. The proposed method achieves faster convergence and reduced performance variability across training seeds in more challenging environments, with success rates improving by up to approximately 5 percent compared to non fuzzy reward formulations.