Position control of an acoustic cavitation bubble by reinforcement learning
This work addresses precise bubble manipulation for applications in fields like sonochemistry or medical ultrasound, representing an incremental improvement with a novel method for a known bottleneck.
The researchers tackled the problem of controlling the position of an acoustic cavitation bubble in a dual-frequency standing acoustic wave field using reinforcement learning, achieving a control speed up to 7 times faster than linear theory predictions.
A control technique is developed via Reinforcement Learning that allows arbitrary controlling of the position of an acoustic cavitation bubble in a dual-frequency standing acoustic wave field. The agent must choose the optimal pressure amplitude values to manipulate the bubble position in the range of $x/λ_0\in[0.05, 0.25]$. To train the agent an actor-critic off-policy algorithm (Deep Deterministic Policy Gradient) was used that supports continuous action space, which allows setting the pressure amplitude values continuously within $0$ and $1\, \mathrm{bar}$. A shaped reward function is formulated that minimizes the distance between the bubble and the target position and implicitly encourages the agent to perform the position control within the shortest amount of time. In some cases, the optimal control can be 7 times faster than the solution expected from the linear theory.