Probabilistic Performance Guarantees for Multi-Task Reinforcement Learning
This addresses the problem of ensuring reliable policy deployment in safety-critical settings for researchers and practitioners in reinforcement learning, representing an incremental advance by adding formal guarantees to existing methods.
The paper tackles the lack of formal performance guarantees in multi-task reinforcement learning for safety-critical applications by introducing a new generalization bound that provides high-confidence guarantees on policy performance for unseen tasks, showing these guarantees are theoretically sound and informative at realistic sample sizes.
Multi-task reinforcement learning trains generalist policies that can execute multiple tasks. While recent years have seen significant progress, existing approaches rarely provide formal performance guarantees, which are indispensable when deploying policies in safety-critical settings. We present an approach for computing high-confidence guarantees on the performance of a multi-task policy on tasks not seen during training. Concretely, we introduce a new generalisation bound that composes (i) per-task lower confidence bounds from finitely many rollouts with (ii) task-level generalisation from finitely many sampled tasks, yielding a high-confidence guarantee for new tasks drawn from the same arbitrary and unknown distribution. Across state-of-the-art multi-task RL methods, we show that the guarantees are theoretically sound and informative at realistic sample sizes.