Jorge I. Poveda

Daniel E. Ochoa, Jorge I. Poveda

We introduce a new closed-loop architecture for the online solution of approximate optimal control problems in the context of continuous-time systems. Specifically, we introduce the first algorithm that incorporates dynamic momentum in actor-critic structures to control continuous-time dynamic plants with an affine structure in the input. By incorporating dynamic momentum in our algorithm, we are able to accelerate the convergence properties of the closed-loop system, achieving superior transient performance compared to traditional gradient-descent based techniques. In addition, by leveraging the existence of past recorded data with sufficiently rich information properties, we dispense with the persistence of excitation condition traditionally imposed on the regressors of the critic and the actor. Given that our continuous-time momentum-based dynamics also incorporate periodic discrete-time resets that emulate restarting techniques used in the machine learning literature, we leverage tools from hybrid dynamical systems theory to establish asymptotic stability properties for the closed-loop system. We illustrate our results with a numerical example.

Muhammad U. Javed, Jorge I. Poveda, Xudong Chen

In this paper, we study the problem of robust global synchronization of resetting clocks in multi-agent networked systems, where by robust global synchronization we mean synchronization that is insensitive to arbitrarily small disturbances, and which is achieved from all initial conditions. In particular, we aim to address the following question: Given a set of homogeneous agents with periodic clocks sharing the same parameters, what kind of information flow topologies will guarantee that the resulting networked systems can achieve robust global synchronization? To address this question, we rely on the framework of robust hybrid dynamical systems and a class of distributed hybrid resetting algorithms. Using the hybrid-system approach, we provide a partial solution to the question: Specifically, we show that one can achieve robust global synchronization with no purely discrete-time solutions in any networked system whose underlying information flow topology is a rooted acyclic digraph. Such a result is complementary to the existing result [1] in which strongly connected digraphs are considered as the underlying information flow topologies of the networked systems. We have further computed in the paper the convergence time for a networked system to reach global synchronization. In particular, the computation reveals the relationship between convergence time and the structure of the underlying digraph. We illustrate our theoretical findings via numerical simulations towards the end of the paper.