Domain Adaptation of Drag Reduction Policy to Partial Measurements
This addresses the challenge of implementing optimal control in real-world fluid systems with limited sensor access, though it is incremental as it builds on existing reinforcement learning methods.
The paper tackles the problem of adapting feedback control policies from full-state to partial measurements in fluid-based systems, demonstrating a method that reduces aerodynamic drag by reconstructing full measurements from partial data history in a simulated road vehicle environment.
Feedback control of fluid-based systems poses significant challenges due to their high-dimensional, nonlinear, and multiscale dynamics, which demand real-time, three-dimensional, multi-component measurements for sensing. While such measurements are feasible in digital simulations, they are often only partially accessible in the real world. In this paper, we propose a method to adapt feedback control policies obtained from full-state measurements to setups with only partial measurements. Our approach is demonstrated in a simulated environment by minimising the aerodynamic drag of a simplified road vehicle. Reinforcement learning algorithms can optimally solve this control task when trained on full-state measurements by placing sensors in the wake. However, in real-world applications, sensors are limited and typically only on the vehicle, providing only partial measurements. To address this, we propose to train a Domain Specific Feature Transfer (DSFT) map reconstructing the full measurements from the history of the partial measurements. By applying this map, we derive optimal policies based solely on partial data. Additionally, our method enables determination of the optimal history length and offers insights into the architecture of optimal control policies, facilitating their implementation in real-world environments with limited sensor information.