Reinforcement learning in large, structured action spaces: A simulation study of decision support for spinal cord injury rehabilitation
This provides a proof of concept for using RL to assist physiotherapists in SCI rehabilitation, though it is incremental as it adapts existing methods to a specific domain.
The paper tackled the challenge of applying reinforcement learning to spinal cord injury rehabilitation, where large action spaces and limited data are problematic, by proposing two treatment grouping methods and finding that both improved treatment decisions, with domain knowledge-based grouping performing better.
Reinforcement learning (RL) has helped improve decision-making in several applications. However, applying traditional RL is challenging in some applications, such as rehabilitation of people with a spinal cord injury (SCI). Among other factors, using RL in this domain is difficult because there are many possible treatments (i.e., large action space) and few patients (i.e., limited training data). Treatments for SCIs have natural groupings, so we propose two approaches to grouping treatments so that an RL agent can learn effectively from limited data. One relies on domain knowledge of SCI rehabilitation and the other learns similarities among treatments using an embedding technique. We then use Fitted Q Iteration to train an agent that learns optimal treatments. Through a simulation study designed to reflect the properties of SCI rehabilitation, we find that both methods can help improve the treatment decisions of physiotherapists, but the approach based on domain knowledge offers better performance. Our findings provide a "proof of concept" that RL can be used to help improve the treatment of those with an SCI and indicates that continued efforts to gather data and apply RL to this domain are worthwhile.