Lights out: training RL agents robust to temporary blindness
This addresses robustness issues for RL agents in real-world applications where sensory inputs can be intermittent, though it is an incremental improvement over existing methods.
The paper tackles the problem of reinforcement learning agents failing when observations are temporarily missing, such as due to a broken light bulb, by introducing a neural network architecture and a novel n-step loss function that enables agents to withstand blindness periods longer than those seen in training.
Agents trained with DQN rely on an observation at each timestep to decide what action to take next. However, in real world applications observations can change or be missing entirely. Examples of this could be a light bulb breaking down, or the wallpaper in a certain room changing. While these situations change the actual observation, the underlying optimal policy does not change. Because of this we want our agent to continue taking actions until it receives a (recognized) observation again. To achieve this we introduce a combination of a neural network architecture that uses hidden representations of the observations and a novel n-step loss function. Our implementation is able to withstand location based blindness stretches longer than the ones it was trained on, and therefore shows robustness to temporary blindness. For access to our implementation, please email Nathan, Marije, or Pau.