Suman Chakravorty

Dilshad Raihan Akkam Veettil, Suman Chakravorty

Recursive estimation of nonlinear dynamical systems is an important problem that arises in several engineering applications. Consistent and accurate propagation of uncertainties is important to ensuring good estimation performance. It is well known that the posterior state estimates in nonlinear problems may assume non-Gaussian multimodal densities. In the past, Gaussian mixture filters and particle filters were introduced to handle non-Gaussianity and nonlinearity. However, these methods have seen only limited success as most mixture filters attempt to fix the number of mixture modes during estimation process, and the particle filters suffer from the curse of dimensionality. In this paper, we propose a particle based Gaussian mixture filtering approach for the general nonlinear estimation problem that is free of the particle depletion problem inherent to most particle filters. We employ an ensemble of randomly sampled states for the propagation of state probability density. A Gaussian mixture model of the propagated uncertainty is then recovered by clustering the ensemble. The posterior density is obtained subsequently through a Kalman measurement update of the mixture modes. We prove the weak convergence of the PGM density to the true filter density assuming exponential forgetting of initial conditions by the true filter. The estimation performance of the proposed filtering approach is demonstrated through several test cases.

Karthikeya S Parunandi, Suman Chakravorty

Traditional stochastic optimal control methods that attempt to obtain an optimal feedback policy for nonlinear systems are computationally intractable. In this paper, we derive a decoupling principle between the open loop plan, and the closed loop feedback gains, that leads to a perturbation feedback control based solution to optimal control problems under action uncertainty, that is near-optimal to the third order. Extensive numerical simulations validate the theory, revealing a wide range of applicability, coping with medium levels of noise. The performance is compared with Nonlinear Model Predictive Control in several difficult robotic planning and control examples that show near identical performance to NMPC while requiring much lesser computational effort. It also leads us to raise the bigger question as to why NMPC should be used in robotic control as opposed to perturbation feedback approaches.

Dan Yu, Suman Chakravorty

In this paper, we consider the model reduction problem of large-scale systems, such as systems obtained through the discretization of partial differential equations. We propose a computationally optimal randomized proper orthogonal decomposition (RPOD*) technique to obtain the reduced order model by perturbing the primal and adjoint system using Gaussian white noise. We show that the computations required by the RPOD* algorithm is orders of magnitude cheaper when compared to the balanced proper orthogonal decomposition (BPOD) algorithm and BPOD output projection algorithm while the performance of the RPOD* algorithm is much better than BPOD output projection algorithm. It is optimal in the sense that a minimal number of snapshots is needed. We also relate the RPOD* algorithm to random projection algorithms. The method is tested on two advection-diffusion equations.

Dan Yu, Suman Chakravorty

This paper studies the state estimation problem of linear discrete-time systems with stochastic unknown inputs. The unknown input is a wide-sense stationary process while no other prior informaton needs to be known. We propose an autoregressive (AR) model based unknown input realization technique which allows us to recover the input statistics from the output data by solving an appropriate least squares problem, then fit an AR model to the recovered input statistics and construct an innovations model of the unknown inputs using the eigensystem realization algorithm (ERA). An augmented state system is constructed and the standard Kalman filter is applied for state estimation. A reduced order model (ROM) filter is also introduced to reduce the computational cost of the Kalman filter. Two numerical examples are given to illustrate the procedure.

Sriram Narayanan, Mohamed Naveed Gul Mohamed, Ishan Paranjape et al.

In recent years, computational power and data availability breakthroughs have revolutionized our ability to analyze complex physical systems through the inverse problem approach. Data-driven techniques like system identification and machine learning play an important role in this field, allowing us to gain insights into previously inaccessible phenomena. However, a major hurdle remains: How can meaningful information from partial measurements be extracted? In the aerospace domain, the challenge of state estimation is particularly pronounced due to the limited availability of observational data and the constraints imposed by sensor capabilities for tracking resident space objects (RSOs). To address these limitations, advanced compensation methodologies are required. Currently, range and bearing measurements obtained from radar and optical systems constitute the primary observational tools in the space situational awareness (SSA) community. In this work, we propose a novel framework that integrates a simplified reference dynamics model with a data-driven surrogate measurement model. This fusion process leverages the strengths of both models to estimate complex dynamical behaviors under conditions of partial observability. Extensive numerical experiments were conducted across multiple datasets to validate the proposed framework. The results demonstrate its efficacy in accurately reconstructing system dynamics from incomplete measurement data. Furthermore, to ensure the robustness of the framework, an initial consistency analysis of the surrogate modeling approach is presented. By addressing the current challenges and refining the integration of data-driven techniques with traditional physics-based modeling, this framework aims to advance state estimation methodologies in the aerospace sector.

Raman Goyal, Ran Wang, Mohamed Naveed Gul Mohamed et al.

This paper develops a data-based approach to the closed-loop output feedback control of nonlinear dynamical systems with a partial nonlinear observation model. We propose an information state based approach to rigorously transform the partially observed problem into a fully observed problem where the information state consists of the past several observations and control inputs. We further show the equivalence of the transformed and the initial partially observed optimal control problems and provide the conditions to solve for the deterministic optimal solution. We develop a data based generalization of the iterative Linear Quadratic Regulator (iLQR) to partially observed systems using a local linear time varying model of the information state dynamics approximated by an Autoregressive moving average (ARMA) model, that is generated using only the input-output data. This open-loop trajectory optimization solution is then used to design a local feedback control law, and the composite law then provides an optimum solution to the partially observed feedback design problem. The efficacy of the developed method is shown by controlling complex high dimensional nonlinear dynamical systems in the presence of model and sensing uncertainty.

Raman Goyal, Suman Chakravorty, Ran Wang et al.

We consider the problem of Reinforcement Learning for nonlinear stochastic dynamical systems. We show that in the RL setting, there is an inherent ``Curse of Variance" in addition to Bellman's infamous ``Curse of Dimensionality", in particular, we show that the variance in the solution grows factorial-exponentially in the order of the approximation. A fundamental consequence is that this precludes the search for anything other than ``local" feedback solutions in RL, in order to control the explosive variance growth, and thus, ensure accuracy. We further show that the deterministic optimal control has a perturbation structure, in that the higher order terms do not affect the calculation of lower order terms, which can be utilized in RL to get accurate local solutions.

Mohamed Naveed Gul Mohamed, Suman Chakravorty, Raman Goyal et al.

We consider the problem of nonlinear stochastic optimal control. This problem is thought to be fundamentally intractable owing to Bellman's "curse of dimensionality". We present a result that shows that repeatedly solving an open-loop deterministic problem from the current state with progressively shorter horizons, similar to Model Predictive Control (MPC), results in a feedback policy that is $O(ε^4)$ near to the true global stochastic optimal policy, where $ε$ is a perturbation parameter modulating the noise. We also show that the optimal deterministic feedback problem has a perturbation structure such that higher-order terms of the feedback law do not affect lower-order terms and that this structure is lost in the optimal stochastic feedback problem. Consequently, solving the Stochastic Dynamic Programming problem is highly susceptible to noise, even in low dimensional problems, and in practice, the MPC-type feedback law offers superior performance even for high noise levels.

Ran Wang, Karthikeya S. Parunandi, Aayushman Sharma et al.

The problem of Reinforcement Learning (RL) in an unknown nonlinear dynamical system is equivalent to the search for an optimal feedback law utilizing the simulations/ rollouts of the dynamical system. Most RL techniques search over a complex global nonlinear feedback parametrization making them suffer from high training times as well as variance. Instead, we advocate searching over a local feedback representation consisting of an open-loop sequence, and an associated optimal linear feedback law completely determined by the open-loop. We show that this alternate approach results in highly efficient training, the answers obtained are repeatable and hence reliable, and the resulting closed performance is superior to global state-of-the-art RL techniques. Finally, if we replan, whenever required, which is feasible due to the fast and reliable local solution, it allows us to recover global optimality of the resulting feedback law.

Mohamed Naveed Gul Mohamed, Suman Chakravorty, Dylan A. Shell

We consider the problem of robotic planning under uncertainty. This problem may be posed as a stochastic optimal control problem, complete solution to which is fundamentally intractable owing to the infamous curse of dimensionality. We report the results of an extensive simulation study in which we have compared two methods, both of which aim to salvage tractability by using alternative, albeit inexact, means for treating feedback. The first is a recently proposed method based on a near-optimal "decoupling principle" for tractable feedback design, wherein a nominal open-loop problem is solved, followed by a linear feedback design around the open-loop. The second is Model Predictive Control (MPC), a widely-employed method that uses repeated re-computation of the nominal open-loop problem during execution to correct for noise, though when interpreted as feedback, this can only said to be an implicit form. We examine a much wider range of noise levels than have been previously reported and empirical evidence suggests that the decoupling method allows for tractable planning over a wide range of uncertainty conditions without unduly sacrificing performance.

Karthikeya S Parunandi, Aayushman Sharma, Suman Chakravorty et al.

In this paper, we propose a structured linear parameterization of a feedback policy to solve the model-free stochastic optimal control problem. This parametrization is corroborated by a decoupling principle that is shown to be near-optimal under a small noise assumption, both in theory and by empirical analyses. Further, we incorporate a model-free version of the Iterative Linear Quadratic Regulator (ILQR) in a sample-efficient manner into our framework. Simulations on systems over a range of complexities reveal that the resulting algorithm is able to harness the superior second-order convergence properties of ILQR. As a result, it is fast and is scalable to a wide variety of higher dimensional systems. Comparisons are made with a state-of-the-art reinforcement learning algorithm, the Deep Deterministic Policy Gradient (DDPG) technique, in order to demonstrate the significant merits of our approach in terms of training-efficiency.

Ran Wang, Karthikeya Parunandi, Dan Yu et al.

This paper addresses the problem of learning the optimal control policy for a nonlinear stochastic dynamical system with continuous state space, continuous action space and unknown dynamics. This class of problems are typically addressed in stochastic adaptive control and reinforcement learning literature using model-based and model-free approaches respectively. Both methods rely on solving a dynamic programming problem, either directly or indirectly, for finding the optimal closed loop control policy. The inherent `curse of dimensionality' associated with dynamic programming method makes these approaches also computationally difficult. This paper proposes a novel decoupled data-based control (D2C) algorithm that addresses this problem using a decoupled, `open loop - closed loop', approach. First, an open-loop deterministic trajectory optimization problem is solved using a black-box simulation model of the dynamical system. Then, a closed loop control is developed around this open loop trajectory by linearization of the dynamics about this nominal trajectory. By virtue of linearization, a linear quadratic regulator based algorithm can be used for this closed loop control. We show that the performance of D2C algorithm is approximately optimal. Moreover, simulation performance suggests significant reduction in training time compared to other state of the art algorithms.

Dan Yu, Mohammandhussen Rafieisakhaei, Suman Chakravorty

This paper studies the stochastic optimal control problem for systems with unknown dynamics. A novel decoupled data based control (D2C) approach is proposed, which solves the problem in a decoupled "open loop-closed loop" fashion that is shown to be near-optimal. First, an open-loop deterministic trajectory optimization problem is solved using a black-box simulation model of the dynamical system using a standard nonlinear programming (NLP) solver. Then a Linear Quadratic Regulator (LQR) controller is designed for the nominal trajectory-dependent linearized system which is learned using input-output experimental data. Computational examples are used to illustrate the performance of the proposed approach with three benchmark problems.

Mohammadhussein Rafieisakhaei, Suman Chakravorty, P. R. Kumar

Optimizing measures of the observability Gramian as a surrogate for the estimation performance may provide irrelevant or misleading trajectories for planning under observation uncertainty.

Dan Yu, Mohammadhussein Rafieisakhaei, Suman Chakravorty

This paper studies the partially observed stochastic optimal control problem for systems with state dynamics governed by Partial Differential Equations (PDEs) that leads to an extremely large problem. First, an open-loop deterministic trajectory optimization problem is solved using a black box simulation model of the dynamical system. Next, a Linear Quadratic Gaussian (LQG) controller is designed for the nominal trajectory-dependent linearized system, which is identified using input-output experimental data consisting of the impulse responses of the optimized nominal system. A computational nonlinear heat example is used to illustrate the performance of the approach.

Dan Yu, Mohammadhussein Rafieisakhaei, Suman Chakravorty

This paper studies the stochastic optimal control problem for systems with unknown dynamics. First, an open-loop deterministic trajectory optimization problem is solved without knowing the explicit form of the dynamical system. Next, a Linear Quadratic Gaussian (LQG) controller is designed for the nominal trajectory-dependent linearized system, such that under a small noise assumption, the actual states remain close to the optimal trajectory. The trajectory-dependent linearized system is identified using input-output experimental data consisting of the impulse responses of the nominal system. A computational example is given to illustrate the performance of the proposed approach.

Mohammadhussein Rafieisakhaei, Suman Chakravorty, P. R. Kumar

We consider the problem of planning under observation and motion uncertainty for nonlinear robotics systems. Determining the optimal solution to this problem, generally formulated as a Partially Observed Markov Decision Process (POMDP), is computationally intractable. We propose a Trajectory-optimized Linear Quadratic Gaussian (T-LQG) approach that leads to quantifiably near-optimal solutions for the POMDP problem. We provide a novel "separation principle" for the design of an optimal nominal open-loop trajectory followed by an optimal feedback control law, which provides a near-optimal feedback control policy for belief space planning problems involving a polynomial order of calculations of minimum order.

Mohammadhussein Rafieisakhaei, Suman Chakravorty, P. R. Kumar

We consider nonlinear stochastic systems that arise in path planning and control of mobile robots. As is typical of almost all nonlinear stochastic systems, the optimally solving problem is intractable. We provide a design approach which yields a tractable design that is quantifiably near-optimal. We exhibit a "separation" principle under a small noise assumption consisting of the optimal open-loop design of nominal trajectory followed by an optimal feedback law to track this trajectory, which is different from the usual effort of separating estimation from control. As a corollary, we obtain a trajectory-optimized linear quadratic regulator design for stochastic nonlinear systems with Gaussian noise.

Saurav Agarwal, Vikram Shree, Suman Chakravorty

The SLAM problem is known to have a special property that when robot orientation is known, estimating the history of robot poses and feature locations can be posed as a standard linear least squares problem. In this work, we develop a SLAM framework that uses relative feature-to-feature measurements to exploit this structural property of SLAM. Relative feature measurements are used to pose a linear estimation problem for pose-to-pose orientation constraints. This is followed by solving an iterative non-linear on-manifold optimization problem to compute the maximum likelihood estimate for robot orientation given relative rotation constraints. Once the robot orientation is computed, we solve a linear problem for robot position and map estimation. Our approach reduces the computational burden of non-linear optimization by posing a smaller optimization problem as compared to standard graph-based methods for feature-based SLAM. Further, empirical results show our method avoids catastrophic failures that arise in existing methods due to using odometery as an initial guess for non-linear optimization, while its accuracy degrades gracefully as sensor noise is increased. We demonstrate our method through extensive simulations and comparisons with an existing state-of-the-art solver.

Mohammadhussein Rafieisakhaei, Suman Chakravorty, P. R. Kumar

Planning under motion and observation uncertainties requires solution of a stochastic control problem in the space of feedback policies. In this paper, we reduce the general (n^2+n)-dimensional belief space planning problem to an (n)-dimensional problem by obtaining a Linear Quadratic Gaussian (LQG) design with the best nominal performance. Then, by taking the underlying trajectory of the LQG controller as the decision variable, we pose a coupled design of trajectory, estimator, and controller design through a Non-Linear Program (NLP) that can be solved by a general NLP solver. We prove that under a first-order approximation and a careful usage of the separation principle, our approximations are valid. We give an analysis on the existing major belief space planning methods and show that our algorithm has the lowest computational burden. Finally, we extend our solution to contain general state and control constraints. Our simulation results support our design.

Mohammadhussein Rafieisakhaei, Suman Chakravorty, P. R. Kumar

Simultaneous Localization and Planning (SLAP) under process and measurement uncertainties is a challenge. It involves solving a stochastic control problem modeled as a Partially Observed Markov Decision Process (POMDP) in a general framework. For a convex environment, we propose an optimization-based open-loop optimal control problem coupled with receding horizon control strategy to plan for high quality trajectories along which the uncertainty of the state localization is reduced while the system reaches to a goal state with minimum control effort. In a static environment with non-convex state constraints, the optimization is modified by defining barrier functions to obtain collision-free paths while maintaining the previous goals. By initializing the optimization with trajectories in different homotopy classes and comparing the resultant costs, we improve the quality of the solution in the presence of action and measurement uncertainties. In dynamic environments with time-varying constraints such as moving obstacles or banned areas, the approach is extended to find collision-free trajectories. In this paper, the underlying spaces are continuous, and beliefs are non-Gaussian. Without obstacles, the optimization is a globally convex problem, while in the presence of obstacles it becomes locally convex. We demonstrate the performance of the method on different scenarios.

Amirhossein Tamjidi, Suman Chakravorty, Dylan Shell

This paper presents a new recursive information consensus filter for decentralized dynamic-state estimation. No structure is assumed about the topology of the network and local estimators are assumed to have access only to local information. The network need not be connected at all times. Consensus over priors which might become correlated is performed through Covariance Intersection (CI) and consensus over new information is handled using weights based on a Metropolis Hastings Markov Chains. We establish bounds for estimation performance and show that our method produces unbiased conservative estimates that are better than CI. The performance of the proposed method is evaluated and compared with competing algorithms on an atmospheric dispersion problem.

Mohammadhussein Rafieisakhaei, Amirhossein Tamjidi, Suman Chakravorty et al.

Planning under process and measurement uncertainties is a challenging problem. In its most general form it can be modeled as a Partially Observed Markov Decision Process (POMDP) problem. However POMDPs are generally difficult to solve when the underlying spaces are continuous, particularly when beliefs are non-Gaussian, and the difficulty is further exacerbated when there are also non-convex constraints on states. Existing algorithms to address such challenging POMDPs are expensive in terms of computation and memory. In this paper, we provide a feedback policy in non-Gaussian belief space via solving a convex program for common non-linear observation models. The solution involves a Receding Horizon Control strategy using particle filters for the non-Gaussian belief representation. We develop a way of capturing non-convex constraints in the state space and adapt the optimization to incorporate such constraints, as well. A key advantage of this method is that it does not introduce additional variables in the optimization problem and is therefore more scalable than existing constrained problems in belief space. We demonstrate the performance of the method on different scenarios.

Saurav Agarwal, Amirhossein Tamjidi, Suman Chakravorty

This paper presents a method for motion planning under uncertainty to deal with situations where ambiguous data associations result in a multimodal hypothesis on the robot state. In the global localization problem, sometimes referred to as the "lost or kidnapped robot problem", given little to no a priori pose information, the localization algorithm should recover the correct pose of a mobile robot with respect to a global reference frame. We present a Receding Horizon approach, to plan actions that sequentially disambiguate a multimodal belief to achieve tight localization on the correct pose in finite time, i.e., converge to a unimodal belief. Experimental results are presented using a physical ground robot operating in an artificial maze-like environment. We demonstrate two runs wherein the robot is given no a priori information about its initial pose and the planner is tasked to localize the robot.

Ali-akbar Agha-mohammadi, Saurav Agarwal, Sung-Kyun Kim et al.

Simultaneous localization and Planning (SLAP) is a crucial ability for an autonomous robot operating under uncertainty. In its most general form, SLAP induces a continuous POMDP (partially-observable Markov decision process), which needs to be repeatedly solved online. This paper addresses this problem and proposes a dynamic replanning scheme in belief space. The underlying POMDP, which is continuous in state, action, and observation space, is approximated offline via sampling-based methods, but operates in a replanning loop online to admit local improvements to the coarse offline policy. This construct enables the proposed method to combat changing environments and large localization errors, even when the change alters the homotopy class of the optimal trajectory. It further outperforms the state-of-the-art FIRM (Feedback-based Information RoadMap) method by eliminating unnecessary stabilization steps. Applying belief space planning to physical systems brings with it a plethora of challenges. A key focus of this paper is to implement the proposed planner on a physical robot and show the SLAP solution performance under uncertainty, in changing environments and in the presence of large disturbances, such as a kidnapped robot situation.

Saurav Agarwal, Amirhossein Tamjidi, Suman Chakravorty

Planning under uncertainty is a key requirement for physical systems due to the noisy nature of actuators and sensors. Using a belief space approach, planning solutions tend to generate actions that result in information seeking behavior which reduce state uncertainty. While recent work has dealt with planning for Gaussian beliefs, for many cases, a multi-modal belief is a more accurate representation of the underlying belief. This is particularly true in environments with information symmetry that cause uncertain data associations which naturally lead to a multi-modal hypothesis on the state. Thus, a planner cannot simply base actions on the most-likely state. We propose an algorithm that uses a Receding Horizon Planning approach to plan actions that sequentially disambiguate the multi-modal belief to a uni-modal Gaussian and achieve tight localization on the true state, called a Multi-Modal Motion Planner (M3P). By combining a Gaussian sampling-based belief space planner with M3P, and introducing a switching behavior in the planner and belief representation, we present a holistic end-to-end solution for the belief space planning problem. Simulation results for a 2D ground robot navigation problem are presented that demonstrate our method's performance.