Fragile, Robust, and Antifragile: A Perspective from Parameter Responses in Reinforcement Learning Under Stress
This work addresses robustness in reinforcement learning systems, which is crucial for real-world applications, though it appears incremental as it builds on existing stress analysis methods.
This paper tackles RL policy robustness by analyzing network parameters under internal and external stresses, classifying them as fragile, robust, or antifragile based on their influence on performance. The framework, validated on PPO agents in Mujoco environments, reveals antifragile parameters that enhance performance under stress.
This paper explores Reinforcement learning (RL) policy robustness by systematically analyzing network parameters under internal and external stresses. Inspired by synaptic plasticity in neuroscience, synaptic filtering introduces internal stress by selectively perturbing parameters, while adversarial attacks apply external stress through modified agent observations. This dual approach enables the classification of parameters as fragile, robust, or antifragile, based on their influence on policy performance in clean and adversarial settings. Parameter scores are defined to quantify these characteristics, and the framework is validated on PPO-trained agents in Mujoco continuous control environments. The results highlight the presence of antifragile parameters that enhance policy performance under stress, demonstrating the potential of targeted filtering techniques to improve RL policy adaptability. These insights provide a foundation for future advancements in the design of robust and antifragile RL systems.