Q-Networks for Binary Vector Actions
This addresses computational efficiency in RL for problems with large discrete action spaces, though it appears incremental as it builds on existing Q-learning methods.
The paper tackles reinforcement learning with binary vector actions by proposing a neural network architecture that approximates the action-value function as linear in actions but non-linear in states, enabling efficient greedy and softmax action selection. Empirical results in grid world and blocker tasks suggest the architecture is effective for RL problems with large discrete action sets.
In this paper reinforcement learning with binary vector actions was investigated. We suggest an effective architecture of the neural networks for approximating an action-value function with binary vector actions. The proposed architecture approximates the action-value function by a linear function with respect to the action vector, but is still non-linear with respect to the state input. We show that this approximation method enables the efficient calculation of greedy action selection and softmax action selection. Using this architecture, we suggest an online algorithm based on Q-learning. The empirical results in the grid world and the blocker task suggest that our approximation architecture would be effective for the RL problems with large discrete action sets.