LLM-Driven Composite Neural Architecture Search for Multi-Source RL State Encoding
This addresses the challenge of manual design for multi-source RL state encoders, which is incremental as it builds on existing NAS methods by incorporating intermediate-output signals.
The paper tackles the problem of designing state encoders for reinforcement learning with multiple information sources by formalizing it as composite neural architecture search, and the result is an LLM-driven pipeline that discovers higher-performing architectures with fewer candidate evaluations on a mixed-autonomy traffic control task.
Designing state encoders for reinforcement learning (RL) with multiple information sources -- such as sensor measurements, time-series signals, image observations, and textual instructions -- remains underexplored and often requires manual design. We formalize this challenge as a problem of composite neural architecture search (NAS), where multiple source-specific modules and a fusion module are jointly optimized. Existing NAS methods overlook useful side information from the intermediate outputs of these modules -- such as their representation quality -- limiting sample efficiency in multi-source RL settings. To address this, we propose an LLM-driven NAS pipeline that leverages language-model priors and intermediate-output signals to guide sample-efficient search for high-performing composite state encoders. On a mixed-autonomy traffic control task, our approach discovers higher-performing architectures with fewer candidate evaluations than traditional NAS baselines and the LLM-based GENIUS framework.