Umesh Vaidya

Boumediene Hamzi, Umesh Vaidya

We present a kernel-based linear matrix inequality (LMI) approach for the approximate solution of Hamilton--Jacobi--Bellman (HJB) equations arising in nonlinear optimal control. The method represents the gradient of the value function in a reproducing kernel Hilbert space (RKHS) and uses a Schur-complement reformulation to convert the quadratic HJB inequality into an LMI that is linear in the kernel coefficients, yielding a convex semidefinite program. The novel ingredient is an explicit Riccati--Hessian \emph{equality} constraint at the equilibrium, which removes the trivial solution and forces the Hessian of the approximation to match the algebraic Riccati equation solution of the linearised system. We give a suboptimality bound $J(x_0;\hat u) - V^*(x_0)\le \varepsilon\,T(x_0)$ in which $T(x_0)$ depends only on the problem data and the working domain (not on the approximation), and an RKHS approximation rate. Numerical experiments on a corrected 1D polynomial benchmark and on the Van der Pol oscillator measure $\varepsilon$, the RKHS approximation error, and the closed-loop cost $J(x_0;\hat u)$ versus the optimal value $V^*(x_0)$. On the 1D problem with $V^*$ in the polynomial-kernel RKHS the method recovers $V^*$ to within $3\times10^{-7}$ and achieves $0.000\%$ suboptimality. On Van der Pol it achieves the smallest HJB residual ($\varepsilon\approx 2.62$) of any method tested, beats LQR on every initial condition, and is within $0.42\%$ of the best per-IC cost (Albrekht order 6). When $V^*$ is not in the chosen RKHS, the method degrades gracefully: residuals stop improving with more centres but suboptimality remains bounded ($\le 13\%$ on the 1D test).

Himanshu Sharma, Umesh Vaidya, Baskar Ganapathysubramanian

Sensors in buildings are used for a wide variety of applications such as monitoring air quality, contaminants, indoor temperature, and relative humidity. These are used for accessing and ensuring indoor air quality, and also for ensuring safety in the event of chemical and biological attacks. It follows that optimal placement of sensors become important to accurately monitor contaminant levels in the indoor environment. However, contaminant transport inside the indoor environment is governed by the indoor flow conditions which are affected by various uncertainties associated with the building systems including occupancy and boundary fluxes. Therefore, it is important to account for all associated uncertainties while designing the sensor layout. The transfer operator based framework provides an effective way to identify optimal placement of sensors. Previous work has been limited to sensor placements under deterministic scenarios. In this work we extend the transfer operator based approach for optimal sensor placement while accounting for building systems uncertainties. The methodology provides a probabilistic metric to gauge coverage under uncertain conditions. We illustrate the capabilities of the framework with examples exhibiting boundary flux uncertainty.

Arvind Raghunathan, Umesh Vaidya

Numerical solutions for the optimal feedback stabilization of discrete time dynamical systems is the focus of this paper. Set-theoretic notion of almost everywhere stability introduced by the Lyapunov measure, weaker than conventional Lyapunov function-based stabilization methods, is used for optimal stabilization. The linear Perron-Frobenius transfer operator is used to pose the optimal stabilization problem as an infinite dimensional linear program. Set-oriented numerical methods are used to obtain the finite dimensional approximation of the linear program. We provide conditions for the existence of stabilizing feedback controls and show the optimal stabilizing feedback control can be obtained as a solution of a finite dimensional linear program. The approach is demonstrated on stabilization of period two orbit in a controlled standard map.

Pranav Sharma, Bowen Huang, Umesh Vaidya et al.

In this paper, we propose linear operator theoretic framework involving Koopman operator for the data-driven identification of power system dynamics. We explicitly account for noise in the time series measurement data and propose robust approach for data-driven approximation of Koopman operator for the identification of nonlinear power system dynamics. The identified model is used for the prediction of state trajectories in the power system. The application of the framework is illustrated using an IEEE nine bus test system.

Amit Diwadkar, Umesh Vaidya

In this paper, we study the problem of state observation of nonlinear systems over an erasure channel. The notion of mean square exponential stability is used to analyze the stability property of observer error dynamics. The main results of this paper prove, fundamental limitation arises for mean square exponential stabilization of the observer error dynamics, expressed in terms of probability of erasure, and positive Lyapunov exponents of the system. Positive Lyapunov exponents are a measure of average expansion of nearby trajectories on an attractor set for nonlinear systems. Hence, the dependence of limitation results on the Lyapunov exponents highlights the important role played by non-equilibrium dynamics in observation over an erasure channel. The limitation on observation is also related to measure-theoretic entropy of the system, which is another measure of dynamical complexity. The limitation result for the observation of linear systems is obtained as a special case, where Lyapunov exponents are shown to emerge as the natural generalization of eigenvalues from linear systems to nonlinear systems.

Himanshu Sharma, Anthony D. Fontanini, Umesh Vaidya et al.

Dynamical system-based linear transfer Perron- Frobenius (P-F) operator framework is developed to address analysis and design problems in the building system. In particular, the problems of fast contaminant propagation and optimal placement of sensors in uncertain operating conditions of indoor building environment are addressed. The linear nature of transfer P-F operator is exploited to develop a computationally efficient numerical scheme based on the finite dimensional approximation of P-F operator for fast propagation of contaminants. The proposed scheme is an order of magnitude faster than existing methods that rely on simulation of an advection-diffusion partial differential equation for contami- nant transport. Furthermore, the system-theoretic notion of observability gramian is generalized to nonlinear flow fields using the transfer P-F operator. This developed notion of observability gramian for nonlinear flow field combined with the finite dimensional approximation of P-F operator is used to provide a systematic procedure for optimal placement of sensors under uncertain operating conditions. Simulation results are presented to demonstrate the applicability of the developed framework on the IEA-annex 2D benchmark problem.

Umesh Vaidya

In this paper, we study the stability problem of a stochastic, nonlinear, discrete-time system. We introduce a linear transfer operator-based Lyapunov measure as a new tool for stability verification of stochastic systems. Weaker set-theoretic notion of almost everywhere stochastic stability is introduced and verified, using Lyapunov measure-based stochastic stability theorems. Furthermore, connection between Lyapunov functions, a popular tool for stochastic stability verification, and Lyapunov measures is established. Using the duality property between the linear transfer Perron-Frobenius and Koopman operators, we show the Lyapunov measure and Lyapunov function used for the verification of stochastic stability are dual to each other. Set-oriented numerical methods are proposed for the finite dimensional approximation of the Perron-Frobenius operator; hence, Lyapunov measure is proposed. Stability results in finite dimensional approximation space are also presented. Finite dimensional approximation is shown to introduce further weaker notion of stability referred to as coarse stochastic stability. The results in this paper extend our earlier work on the use of Lyapunov measures for almost everywhere stability verification of deterministic dynamical systems ("Lyapunov Measure for Almost Everywhere Stability", {\it IEEE Trans. on Automatic Control}, Vol. 53, No. 1, Feb. 2008).

Amit Diwadkar, Sambarta Dasgupta, Umesh Vaidya

In this paper, we study the problem of control of discrete-time nonlinear systems in Lure form over erasure channels at the input and output. The input and output channel uncertainties are modeled as Bernoulli random variables. The main results of this paper provide sufficient condition for the mean square exponential stability of the closed loop system expressed in terms of statistics of channel uncertainty and plant characteristics. We also provide synthesis method for the design of observer-based controller that is robust to channel uncertainty. To prove the main results of this paper, we discover a stochastic variant of the well known Positive Real Lemma and principle of separation for stochastic nonlinear system. Application of the results for the stabilization of system in Lure form over packet-drop network is discussed. Finally a result for state feedback control of a Lure system with a general multiplicative uncertainty at actuation is discussed.

Apurba Kumar Das, Arvind Raghunathan, Umesh Vaidya

In this paper we develop linear transfer Perron Frobenius operator-based approach for optimal stabilization of stochastic nonlinear system. One of the main highlight of the proposed transfer operator based approach is that both the theory and computational framework developed for the optimal stabilization of deterministic dynamical system in [1] carries over to the stochastic case with little change. The optimal stabilization problem is formulated as an infinite dimensional linear program. Set oriented numerical methods are proposed for the finite dimensional approximation of the transfer operator and the controller. Simulation results are presented to verify the developed framework.

Apurba Kumar Das, Bowen Huang, Umesh Vaidya

In this paper, we propose a data-driven approach for control of nonlinear dynamical systems. The proposed data-driven approach relies on transfer Koopman and Perron-Frobenius (P-F) operators for linear representation and control of such systems. Systematic model-based frameworks involving linear transfer P-F operator were proposed for almost everywhere stability analysis and control design of a nonlinear dynamical system in previous works [1-3]. Lyapunov measure can be used as a tool to provide linear programming-based computational framework for stability analysis and almost everywhere stabilizing control design of a nonlinear system. In this paper, we show that those frameworks can be extended to a data-driven setting, where the finite dimensional approximation of linear transfer P-F operator and stabilizing feedback controller can be obtained from time-series data. We exploit the positivity and Markov property of these operators and their finite-dimensional approximation to provide {\it linear programming} based approach for designing an optimally stabilizing feedback controller.

Sai Pushpak, Umesh Vaidya

In this paper, we demonstrate the fragility of decentralized load-side frequency algorithms proposed in [1] against stochastic parametric uncertainty in power network model. The stochastic parametric uncertainty is motivated through the presence of renewable energy resources in power system model. We show that relatively small variance value of the parametric uncertainty affecting the system bus voltages cause the decentralized load-side frequency regulation algorithm to become stochastically unstable. The critical variance value of the stochastic bus voltages above which the decentralized control algorithm become mean square unstable is computed using an analytical framework developed in [2], [3]. Furthermore, the critical variance value is shown to decrease with the increase in the cost of the controllable loads and with the increase in penetration of renewable energy resources. Finally, simulation results on IEEE 68 bus system are presented to verify the main findings of the paper.

Sai Pushpak, Amit Diwadkar, Umesh Vaidya

In this technical note, we study the mean square stability-based analysis of stochastic continuous-time linear networked systems. The stochastic uncertainty is assumed to enter multiplicatively in system dynamics through input and output channels of the plant. Necessary and sufficient conditions for mean square exponential stability are expressed in terms of the input-output property of deterministic or nominal system dynamics captured by the {\it mean square} system norm and variance of channel uncertainty. The stability results can also be interpreted as a small gain theorem for continuous-time stochastic systems. Linear Matrix Inequalities (LMI)-based optimization formulation is provided for the computation of mean square system norm for stability analysis. For a special case of single input channel uncertainty, we also prove a fundamental limitation result that arises in the mean square exponential stabilization of the continuous-time linear system. Overall, the contributions in this work generalize the existing results on stability analysis from discrete-time linear systems to continuous-time linear systems with multiplicative uncertainty. Simulation results are presented for WSCC $9$ bus power system to demonstrate the application of the developed framework.

Sriram S. K. S. Narayanan, Umesh Vaidya

This paper presents Residual Koopman MPC (RK-MPC), a Koopman-based, data-driven model predictive control framework for quadruped locomotion that improves prediction fidelity while preserving real-time tractability. RK-MPC augments a nominal template model with a compact linear residual predictor learned from data in lifted coordinates, enabling systematic correction of model mismatch induced by contact variability and terrain disturbances with provable bounds on multi-step prediction error. The learned residual model is embedded within a convex quadratic-program MPC formulation, yielding a receding-horizon controller that runs onboard at 500 Hz and retains the structure and constraint-handling advantages of optimization-based control. We evaluate RK-MPC in both Gazebo simulation and Unitree Go1 hardware experiments, demonstrating reliable blind locomotion across contact disturbances, multiple gait schedules, and challenging off-road terrains including grass, gravel, snow, and ice. We further compare against Koopman/EDMD baselines using alternative observable dictionaries, including monomial and $SE(3)$-structured bases, and show that the residual correction improves multi-step prediction and closed-loop performance while reducing sensitivity to the choice of observables. Overall, RK-MPC provides a practical, hardware-validated pathway for data-driven predictive control of quadrupeds in unstructured environments. See https://sriram-2502.github.io/rk-mpc for implementation videos.

Yiqiang Han, Wenjian Hao, Umesh Vaidya

We develop a data-driven, model-free approach for the optimal control of the dynamical system. The proposed approach relies on the Deep Neural Network (DNN) based learning of Koopman operator for the purpose of control. In particular, DNN is employed for the data-driven identification of basis function used in the linear lifting of nonlinear control system dynamics. The controller synthesis is purely data-driven and does not rely on a priori domain knowledge. The OpenAI Gym environment, employed for Reinforcement Learning-based control design, is used for data generation and learning of Koopman operator in control setting. The method is applied to two classic dynamical systems on OpenAI Gym environment to demonstrate the capability.

Hyungjin Choi, Umesh Vaidya, Yongxin Chen

We consider a class of nonlinear control synthesis problems where the underlying mathematical models are not explicitly known. We propose a data-driven approach to stabilize the systems when only sample trajectories of the dynamics are accessible. Our method is founded on the density function based almost everywhere stability certificate that is dual to the Lyapunov function for dynamic systems. Unlike Lyapunov based methods, density functions lead to a convex formulation for a joint search of the control strategy and the stability certificate. This type of convex problem can be solved efficiently by invoking the machinery of the sum of squares (SOS). For the data-driven part, we exploit the fact that the duality results in the stability theory of the dynamical system can be understood using linear Perron-Frobenius and Koopman operators. This connection allows us to use data-driven methods developed to approximate these operators combined with the SOS techniques for the convex formulation of control synthesis. The efficacy of the proposed approach is demonstrated through several examples.

Yasaman Esfandiari, Aditya Balu, Keivan Ebrahimi et al.

Recent focus on robustness to adversarial attacks for deep neural networks produced a large variety of algorithms for training robust models. Most of the effective algorithms involve solving the min-max optimization problem for training robust models (min step) under worst-case attacks (max step). However, they often suffer from high computational cost from running several inner maximization iterations (to find an optimal attack) inside every outer minimization iteration. Therefore, it becomes difficult to readily apply such algorithms for moderate to large size real world data sets. To alleviate this, we explore the effectiveness of iterative descent-ascent algorithms where the maximization and minimization steps are executed in an alternate fashion to simultaneously obtain the worst-case attack and the corresponding robust model. Specifically, we propose a novel discrete-time dynamical system-based algorithm that aims to find the saddle point of a min-max optimization problem in the presence of uncertainties. Under the assumptions that the cost function is convex and uncertainties enter concavely in the robust learning problem, we analytically show that our algorithm converges asymptotically to the robust optimal solution under a general adversarial budget constraints as induced by $\ell_p$ norm, for $1\leq p\leq \infty$. Based on our proposed analysis, we devise a fast robust training algorithm for deep neural networks. Although such training involves highly non-convex robust optimization problems, empirical results show that the algorithm can achieve significant robustness compared to other state-of-the-art robust models on benchmark data sets.

Subhrajit Sinha, Pranav Sharma, Umesh Vaidya et al.

In this paper, we present a novel approach to identify the generators and states responsible for the small-signal stability of power networks. To this end, the newly developed notion of information transfer between the states of a dynamical system is used. In particular, using the concept of information transfer, which characterizes influence between the various states and a linear combination of states of a dynamical system, we identify the generators and states which are responsible for causing instability of the power network. While characterizing influence from state to state, information transfer can also describe influence from state to modes thereby generalizing the well-known notion of participation factor while at the same time overcoming some of the limitations of the participation factor. The developed framework is applied to study the three bus system identifying various cause of instabilities in the system. The simulation study is extended to IEEE 39 bus system.

Sambarta Dasgupta, Keivan Ebrahimi, Umesh Vaidya

Identification of the clusters from an unlabeled data set is one of the most important problems in Unsupervised Machine Learning. The state of the art clustering algorithms are based on either the statistical properties or the geometric properties of the data set. In this work, we propose a novel method to cluster the data points using dynamical systems theory. After constructing a gradient dynamical system using interaction potential, we prove that the asymptotic dynamics of this system will determine the cluster centers, when the dynamical system is initialized at the data points. Most of the existing heuristic-based clustering techniques suffer from a disadvantage, namely the stochastic nature of the solution. Whereas, the proposed algorithm is deterministic, and the outcome would not change over multiple runs of the proposed algorithm with the same input data. Another advantage of the proposed method is that the number of clusters, which is difficult to determine in practice, does not have to be specified in advance. Simulation results with are presented, and comparisons are made with the existing methods.

Amit Diwadkar, Umesh Vaidya

We study synchronization in scalar nonlinear systems connected over a linear network with stochastic uncertainty in their interactions. We provide a sufficient condition for the synchronization of such network systems expressed in terms of the parameters of the nonlinear scalar dynamics, the second and largest eigenvalues of the mean interconnection Laplacian, and the variance of the stochastic uncertainty. The sufficient condition is independent of network size thereby making it attractive for verification of synchronization in a large size network. The main contribution of this paper is to provide analytical characterization for the interplay of roles played by the internal dynamics of the nonlinear system, network topology, and uncertainty statistics in network synchronization. We show there exist important tradeoffs between these various network parameters necessary to achieve synchronization. We show for nearest neighbor networks with stochastic uncertainty in interactions there exists an optimal number of neighbors with maximum margin for synchronization. This proves in the presence of interaction uncertainty, too many connections among network components is just as harmful for synchronization as the lack of connection. We provide an analytical formula for the optimal gain required to achieve maximum synchronization margin thereby allowing us to compare various complex network topology for their synchronization property.