A Passivity-Based Method for Accelerated Convex Optimisation
This work provides a general framework for developing optimization algorithms, potentially benefiting fields like machine learning and control systems, though it appears incremental as it builds on classical control concepts.
The authors tackled the problem of designing accelerated convex optimization algorithms by introducing a passivity-based methodology in continuous-time, which guarantees different convergence rate bounds for the objective function value.
This study presents a constructive methodology for designing accelerated convex optimisation algorithms in continuous-time domain. The two key enablers are the classical concept of passivity in control theory and the time-dependent change of variables that maps the output of the internal dynamic system to the optimisation variables. The Lyapunov function associated with the optimisation dynamics is obtained as a natural consequence of specifying the internal dynamics that drives the state evolution as a passive linear time-invariant system. The passivity-based methodology provides a general framework that has the flexibility to generate convex optimisation algorithms with the guarantee of different convergence rate bounds on the objective function value. The same principle applies to the design of online parameter update algorithms for adaptive control by re-defining the output of internal dynamics to allow for the feedback interconnection with tracking error dynamics.