Model-free reinforcement learning with noisy actions for automated experimental control in optics
This work addresses the tedious and complex task of controlling optical systems for researchers and engineers, offering an incremental improvement by applying existing RL algorithms to a specific domain without full noise modeling.
The researchers tackled the problem of automating laser light coupling into an optical fiber, which is challenging due to noise and non-linearities, by using model-free reinforcement learning agents that achieved 90% coupling efficiency, matching human performance but with faster operation.
Setting up and controlling optical systems is often a challenging and tedious task. The high number of degrees of freedom to control mirrors, lenses, or phases of light makes automatic control challenging, especially when the complexity of the system cannot be adequately modeled due to noise or non-linearities. Here, we show that reinforcement learning (RL) can overcome these challenges when coupling laser light into an optical fiber, using a model-free RL approach that trains directly on the experiment without pre-training on simulations. By utilizing the sample-efficient algorithms Soft Actor-Critic (SAC), Truncated Quantile Critics (TQC), or CrossQ, our agents learn to couple with 90% efficiency. A human expert reaches this efficiency, but the RL agents are quicker. In particular, the CrossQ agent outperforms the other agents in coupling speed while requiring only half the training time. We demonstrate that direct training on an experiment can replace extensive system modeling. Our result exemplifies RL's potential to tackle problems in optics, paving the way for more complex applications where full noise modeling is not feasible.