The Pursuit of Diversity: Multi-Objective Testing of Deep Reinforcement Learning Agents
This addresses the need for more effective testing of DRL agents in safety-critical applications, representing an incremental improvement over single-objective tools.
The paper tackled the problem of discovering diverse failure scenarios in deep reinforcement learning agents for safety-critical domains by introducing INDAGO-Nexus, a multi-objective search approach that optimizes for failure likelihood and diversity, resulting in up to 83% more unique failures and 67% faster time-to-failure compared to existing methods.
Testing deep reinforcement learning (DRL) agents in safety-critical domains requires discovering diverse failure scenarios. Existing tools such as INDAGO rely on single-objective optimization focused solely on maximizing failure counts, but this does not ensure discovered scenarios are diverse or reveal distinct error types. We introduce INDAGO-Nexus, a multi-objective search approach that jointly optimizes for failure likelihood and test scenario diversity using multi-objective evolutionary algorithms with multiple diversity metrics and Pareto front selection strategies. We evaluated INDAGO-Nexus on three DRL agents: humanoid walker, self-driving car, and parking agent. On average, INDAGO-Nexus discovers up to 83% and 40% more unique failures (test effectiveness) than INDAGO in the SDC and Parking scenarios, respectively, while reducing time-to-failure by up to 67% across all agents.