RLang: A Declarative Language for Describing Partial World Knowledge to Reinforcement Learning Agents
This provides a tool for researchers and practitioners to more flexibly encode partial world knowledge in reinforcement learning, though it is incremental as it builds on existing RL DSLs.
The authors tackled the problem of communicating domain knowledge to reinforcement learning agents by introducing RLang, a declarative language that can specify information about every element of a Markov decision process, resulting in a parser that grounds programs to an algorithm-agnostic partial world model and policy.
We introduce RLang, a domain-specific language (DSL) for communicating domain knowledge to an RL agent. Unlike existing RL DSLs that ground to \textit{single} elements of a decision-making formalism (e.g., the reward function or policy), RLang can specify information about every element of a Markov decision process. We define precise syntax and grounding semantics for RLang, and provide a parser that grounds RLang programs to an algorithm-agnostic \textit{partial} world model and policy that can be exploited by an RL agent. We provide a series of example RLang programs demonstrating how different RL methods can exploit the resulting knowledge, encompassing model-free and model-based tabular algorithms, policy gradient and value-based methods, hierarchical approaches, and deep methods.