Optimisation of Structured Neural Controller Based on Continuous-Time Policy Gradient
This work addresses the challenge of tuning gains and uncertainty models in fixed-structure control synthesis for aerospace and similar domains, offering a hybrid approach that enhances performance while maintaining stability and interpretability.
The study tackled the problem of optimizing structured nonlinear controllers for continuous-time dynamic systems by combining analytically-derived controller structures with neural network tuning, resulting in improved performance for adaptive nonlinear controllers in online operation, as demonstrated in aerospace applications.
This study presents a policy optimisation framework for structured nonlinear control of continuous-time (deterministic) dynamic systems. The proposed approach prescribes a structure for the controller based on relevant scientific knowledge (such as Lyapunov stability theory or domain experiences) while considering the tunable elements inside the given structure as the point of parametrisation with neural networks. To optimise a cost represented as a function of the neural network weights, the proposed approach utilises the continuous-time policy gradient method based on adjoint sensitivity analysis as a means for correct and performant computation of cost gradient. This enables combining the stability, robustness, and physical interpretability of an analytically-derived structure for the feedback controller with the representational flexibility and optimised resulting performance provided by machine learning techniques. Such a hybrid paradigm for fixed-structure control synthesis is particularly useful for optimising adaptive nonlinear controllers to achieve improved performance in online operation, an area where the existing theory prevails the design of structure while lacking clear analytical understandings about tuning of the gains and the uncertainty model basis functions that govern the performance characteristics. Numerical experiments on aerospace applications illustrate the utility of the structured nonlinear controller optimisation framework.