Balancing Multiple Objectives in Urban Traffic Control with Reinforcement Learning from AI Feedback
This addresses the problem of scalable user-aligned policy learning in domains with inherently conflicting objectives like urban traffic control, representing an incremental extension of existing RLAIF methods.
The paper tackles the challenge of balancing multiple conflicting objectives in urban traffic control by extending Reinforcement Learning from AI Feedback (RLAIF) to multi-objective systems, showing that it produces policies with balanced trade-offs reflecting user priorities without requiring laborious reward engineering.
Reward design has been one of the central challenges for real world reinforcement learning (RL) deployment, especially in settings with multiple objectives. Preference-based RL offers an appealing alternative by learning from human preferences over pairs of behavioural outcomes. More recently, RL from AI feedback (RLAIF) has demonstrated that large language models (LLMs) can generate preference labels at scale, mitigating the reliance on human annotators. However, existing RLAIF work typically focuses only on single-objective tasks, leaving the open question of how RLAIF handles systems that involve multiple objectives. In such systems trade-offs among conflicting objectives are difficult to specify, and policies risk collapsing into optimizing for a dominant goal. In this paper, we explore the extension of the RLAIF paradigm to multi-objective self-adaptive systems. We show that multi-objective RLAIF can produce policies that yield balanced trade-offs reflecting different user priorities without laborious reward engineering. We argue that integrating RLAIF into multi-objective RL offers a scalable path toward user-aligned policy learning in domains with inherently conflicting objectives.