Tyler Summers

Yi Guo, Kyri Baker, Emiliano Dall'Anese et al.

We propose a data-driven method to solve a stochastic optimal power flow (OPF) problem based on limited information about forecast error distributions. The objective is to determine power schedules for controllable devices in a power network to balance operation cost and conditional value-at-risk (CVaR) of device and network constraint violations. These decisions include scheduled power output adjustments and reserve policies, which specify planned reactions to forecast errors in order to accommodate fluctuating renewable energy sources. Instead of assuming the uncertainties across the networks follow prescribed probability distributions, we assume the distributions are only observable through a finite training dataset. By utilizing the Wasserstein metric to quantify differences between the empirical data-based distribution and the real data-generating distribution, we formulate a distributionally robust optimization OPF problem to search for power schedules and reserve policies that are robust to sampling errors inherent in the dataset. A simple numerical example illustrates inherent tradeoffs between operation cost and risk of constraint violation, and we show how our proposed method offers a data-driven framework to balance these objectives.

Nikolaos Kariotoglou, Maryam Kamgarpour, Tyler Summers et al.

One of the most fundamental problems in Markov decision processes is analysis and control synthesis for safety and reachability specifications. We consider the stochastic reach-avoid problem, in which the objective is to synthesize a control policy to maximize the probability of reaching a target set at a given time, while staying in a safe set at all prior times. We characterize the solution to this problem through an infinite dimensional linear program. We then develop a tractable approximation to the infinite dimensional linear program through finite dimensional approximations of the decision space and constraints. For a large class of Markov decision processes modeled by Gaussian mixtures kernels we show that through a proper selection of the finite dimensional space, one can further reduce the computational complexity of the resulting linear program. We validate the proposed method and analyze its potential with a series of numerical case studies.

Ahmad F. Taha, Nikolaos Gatsis, Tyler Summers et al.

We propose methods to solve time-varying, sensor and actuator (SaA) selection problems for uncertain cyber-physical systems. We show that many SaA selection problems for optimizing a variety of control and estimation metrics can be posed as semidefinite optimization problems with mixed-integer bilinear matrix inequalities (MIBMIs). Although this class of optimization problems are computationally challenging, we present tractable approaches that directly tackle MIBMIs, providing both upper and lower bounds, and that lead to effective heuristics for SaA selection. The upper and lower bounds are obtained via successive convex approximations and semidefinite programming relaxations, respectively, and selections are obtained with a novel slicing algorithm from the solutions of the bounding problems. Custom branch-and-bound and combinatorial greedy approaches are also developed for a broad class of systems for comparison. Finally, comprehensive numerical experiments are performed to compare the different methods and illustrate their effectiveness.

Tyler Summers, Maryam Kamgarpour

A key problem in emerging complex cyber-physical networks is the design of information and control topologies, including sensor and actuator selection and communication network design. These problems can be posed as combinatorial set function optimization problems to maximize a dynamic performance metric for the network. Some systems and control metrics feature a property called submodularity, which allows simple greedy algorithms to obtain provably near-optimal topology designs. However, many important metrics lack submodularity and therefore lack provable guarantees for using a greedy optimization approach. Here we show that performance guarantees can be obtained for greedy maximization of certain non-submodular functions of the controllability and observability Gramians. Our results are based on two key quantities: the submodularity ratio, which quantifies how far a set function is from being submodular, and the curvature, which quantifies how far a set function is from being supermodular.

Baiwei Guo, Orcun Karaca, Tyler Summers et al.

With the rising importance of large-scale network control, the problem of actuator placement has received increasing attention. Our goal in this paper is to find a set of actuators minimizing the metric that measures the average energy consumption of the control inputs while ensuring structural controllability of the network. As this problem is intractable, greedy algorithm can be used to obtain an approximate solution. To provide a performance guarantee for this approach, we first define the submodularity ratio for the metric under consideration and then reformulate the structural controllability constraint as a matroid constraint. This shows that the problem under study can be characterized by a matroid optimization involving a weakly submodular objective function. Then, we derive a novel performance guarantee for the greedy algorithm applied to this class of optimization problems. Finally, we show that the matroid feasibility check for the greedy algorithm can be cast as a maximum matching problem in a certain auxiliary bipartite graph related to the network graph.

Sebastian Nugroho, Ahmad F. Taha, Tyler Summers et al.

In most dynamic networks, it is impractical to measure all of the system states; instead, only a subset of the states are measured through sensors. Consequently, and unlike full state feedback controllers, output feedback control utilizes only the measured states to obtain a stable closed-loop performance. This paper explores the interplay between the selection of minimal number of sensors and actuators (SaA) that yield a stable closed-loop system performance. Through the formulation of the static output feedback control problem, we show that the simultaneous selection of minimal set of SaA is a combinatorial optimization problem with mixed-integer nonlinear matrix inequality constraints. To address the computational complexity, we develop two approaches: The first approach relies on integer/disjunctive programming principles, while the second approach is a simple algorithm that is akin to binary search routines. The optimality of the two approaches is also discussed. Numerical experiments are included showing the performance of the developed approaches.

Yitian Chen, Timothy L. Molloy, Tyler Summers et al.

In this paper, we propose and analyze a new method for online linear quadratic regulator (LQR) control with a priori unknown time-varying cost matrices. The cost matrices are revealed sequentially with the potential for future values to be previewed over a short window. Our novel method involves using the available cost matrices to predict the optimal trajectory, and a tracking controller to drive the system towards it. We adopted the notion of dynamic regret to measure the performance of this proposed online LQR control method, with our main result being that the (dynamic) regret of our method is upper bounded by a constant. Moreover, the regret upper bound decays exponentially with the preview window length, and is extendable to systems with disturbances. We show in simulations that our proposed method offers improved performance compared to other previously proposed online LQR methods.

Benjamin Gravell, Iman Shames, Tyler Summers

We propose a robust data-driven output feedback control algorithm that explicitly incorporates inherent finite-sample model estimate uncertainties into the control design. The algorithm has three components: (1) a subspace identification nominal model estimator; (2) a bootstrap resampling method that quantifies non-asymptotic variance of the nominal model estimate; and (3) a non-conventional robust control design method comprising a coupled optimal dynamic output feedback filter and controller with multiplicative noise. A key advantage of the proposed approach is that the system identification and robust control design procedures both use stochastic uncertainty representations, so that the actual inherent statistical estimation uncertainty directly aligns with the uncertainty the robust controller is being designed against. Moreover, the control design method accommodates a highly structured uncertainty representation that can capture uncertainty shape more effectively than existing approaches. We show through numerical experiments that the proposed robust data-driven output feedback controller can significantly outperform a certainty equivalent controller on various measures of sample complexity and stability robustness.

Shilin You, Gael Luna, Juned Shaikh et al.

We present an algorithm for planning trajectories that avoid obstacles and satisfy key-door precedence specifications expressed with a fragment of signal temporal logic. Our method includes a novel exact convex partitioning of the obstacle free space that encodes connectivity among convex free space sets, key sets, and door sets. We then construct an augmented graph of convex sets that exactly encodes the key-door precedence specifications. By solving a shortest path problem in this augmented graph of convex sets, our pipeline provides an exact solution up to a finite parameterization of the trajectory. To illustrate the effectiveness of our approach, we present a method to generate key-door mazes that provide challenging problem instances, and we perform numerical experiments to evaluate the proposed pipeline. Our pipeline is faster by several orders of magnitude than recent state-of-the art methods that use general purpose temporal logic tools.

Kaveh Fathian, Tyler Summers

We present CLIPPER+, an algorithm for finding maximal cliques in unweighted graphs for outlier-robust global registration. The registration problem can be formulated as a graph and solved by finding its maximum clique. This formulation leads to extreme robustness to outliers; however, finding the maximum clique is an NP-hard problem, and therefore approximation is required in practice for large-size problems. The performance of an approximation algorithm is evaluated by its computational complexity (the lower the runtime, the better) and solution accuracy (how close the solution is to the maximum clique). Accordingly, the main contribution of CLIPPER+ is outperforming the state-of-the-art in accuracy while maintaining a relatively low runtime. CLIPPER+ builds on prior work (CLIPPER [1] and PMC [2]) and prunes the graph by removing vertices that have a small core number and cannot be a part of the maximum clique. This will result in a smaller graph, on which the maximum clique can be estimated considerably faster. We evaluate the performance of CLIPPER+ on standard graph benchmarks, as well as synthetic and real-world point cloud registration problems. These evaluations demonstrate that CLIPPER+ has the highest accuracy and can register point clouds in scenarios where over $99\%$ of associations are outliers. Our code and evaluation benchmarks are released at https://github.com/ariarobotics/clipperp.

Amir Ali Farzin, Yuen-Man Pun, Philipp Braun et al.

We consider max-min and min-max problems with objective functions that are possibly non-smooth, submodular with respect to the minimiser and concave with respect to the maximiser. We investigate the performance of a zeroth-order method applied to this problem. The method is based on the subgradient of the Lovász extension of the objective function with respect to the minimiser and based on Gaussian smoothing to estimate the smoothed function gradient with respect to the maximiser. In expectation sense, we prove the convergence of the algorithm to an $ε$-saddle point in the offline case. Moreover, we show that, in the expectation sense, in the online setting, the algorithm achieves $O(\sqrt{N\bar{P}_N})$ online duality gap, where $N$ is the number of iterations and $\bar{P}_N$ is the path length of the sequence of optimal decisions. The complexity analysis and hyperparameter selection are presented for all the cases. The theoretical results are illustrated via numerical examples.

Olalekan Ogunmolu, Nicholas Gans, Tyler Summers

Multistage decision policies provide useful control strategies in high-dimensional state spaces, particularly in complex control tasks. However, they exhibit weak performance guarantees in the presence of disturbance, model mismatch, or model uncertainties. This brittleness limits their use in high-risk scenarios. We present how to quantify the sensitivity of such policies in order to inform of their robustness capacity. We also propose a minimax iterative dynamic game framework for designing robust policies in the presence of disturbance/uncertainties. We test the quantification hypothesis on a carefully designed deep neural network policy; we then pose a minimax iterative dynamic game (iDG) framework for improving policy robustness in the presence of adversarial disturbances. We evaluate our iDG framework on a mecanum-wheeled robot, whose goal is to find a ocally robust optimal multistage policy that achieve a given goal-reaching task. The algorithm is simple and adaptable for designing meta-learning/deep policies that are robust against disturbances, model mismatch, or model uncertainties, up to a disturbance bound. Videos of the results are on the author's website, http://ecs.utdallas.edu/~opo140030/iros18/iros2018.html, while the codes for reproducing our experiments are on github, https://github.com/lakehanne/youbot/tree/rilqg. A self-contained environment for reproducing our results is on docker, https://hub.docker.com/r/lakehanne/youbotbuntu14/

Sai Haneesh Allu, Itay Kadosh, Tyler Summers et al.

We present a modular robotic system for autonomous exploration and semantic updating of large-scale unknown environments. Our approach enables a mobile robot to build, revisit, and update a hybrid semantic map that integrates a 2D occupancy grid for geometry with a topological graph for object semantics. Unlike prior methods that rely on manual teleoperation or precollected datasets, our two-phase approach achieves end-to-end autonomy: first, a modified frontier-based exploration algorithm with dynamic search windows constructs a geometric map; second, using a greedy trajectory planner, environments are revisited, and object semantics are updated using open-vocabulary object detection and segmentation. This modular system, compatible with any metric SLAM framework, supports continuous operation by efficiently updating the semantic graph to reflect short-term and long-term changes such as object relocation, removal, or addition. We validate the approach on a Fetch robot in real-world indoor environments of approximately $8,500$m$^2$ and $117$m$^2$, demonstrating robust and scalable semantic mapping and continuous adaptation, marking a fully autonomous integration of exploration, mapping, and semantic updating on a physical robot.

Gael Luna, Tyler Summers

We present a trajectory optimization algorithm for the traveling salesman problem (TSP) in graphs of convex sets (GCS). Our framework uses an augmented graph of convex sets to encode the TSP specification and solve it exactly as a shortest path problem in GCS. We establish a precise relationship between the landmark Bellman-Held-Karp algorithm and the augmented graph of convex sets with a TSP specification. Additionally, we present a branch and bound heuristic that uses minimum 1-trees to obtain certifiably optimal or near optimal solutions and scales to problems far larger than the exact framework can handle. To assess and certify performance, we explore several alternative lower bounds.

Yu Xiang, Sai Haneesh Allu, Rohith Peddi et al.

We introduce a new trajectory optimization method for robotic grasping based on a point-cloud representation of robots and task spaces. In our method, robots are represented by 3D points on their link surfaces. The task space of a robot is represented by a point cloud that can be obtained from depth sensors. Using the point-cloud representation, goal reaching in grasping can be formulated as point matching, while collision avoidance can be efficiently achieved by querying the signed distance values of the robot points in the signed distance field of the scene points. Consequently, a constrained nonlinear optimization problem is formulated to solve the joint motion and grasp planning problem. The advantage of our method is that the point-cloud representation is general to be used with any robot in any environment. We demonstrate the effectiveness of our method by performing experiments on a tabletop scene and a shelf scene for grasping with a Fetch mobile manipulator and a Franka Panda arm. The project page is available at \url{https://irvlutd.github.io/GraspTrajOpt}

Amir Ali Farzin, Yuen-Man Pun, Philipp Braun et al.

We consider the minimisation problem of submodular functions and investigate the application of a zeroth-order method to this problem. The method is based on exploiting a Gaussian smoothing random oracle to estimate the smoothed function gradient. We prove the convergence of the algorithm to a global $ε$-approximate solution in the offline case and show that the algorithm is Hannan-consistent in the online case with respect to static regret. Moreover, we show that the algorithm achieves $O(\sqrt{NP_N^\ast})$ dynamic regret, where $N$ is the number of iterations and $P_N^\ast$ is the path length. The complexity analysis and hyperparameter selection are presented for all the cases. The theoretical results are illustrated via numerical examples.

Elad Michael, Tyler Summers, Tony A. Wood et al.

With advances in image processing and machine learning, it is now feasible to incorporate semantic information into the problem of simultaneous localisation and mapping (SLAM). Previously, SLAM was carried out using lower level geometric features (points, lines, and planes) which are often view-point dependent and error prone in visually repetitive environments. Semantic information can improve the ability to recognise previously visited locations, as well as maintain sparser maps for long term SLAM applications. However, SLAM in repetitive environments has the critical problem of assigning measurements to the landmarks which generated them. In this paper, we use k-best assignment enumeration to compute marginal assignment probabilities for each measurement landmark pair, in real time. We present numerical studies on the KITTI dataset to demonstrate the effectiveness and speed of the proposed framework.

Matilde Gargiani, Andrea Zanelli, Andrea Martinelli et al.

Despite their success, policy gradient methods suffer from high variance of the gradient estimate, which can result in unsatisfactory sample complexity. Recently, numerous variance-reduced extensions of policy gradient methods with provably better sample complexity and competitive numerical performance have been proposed. After a compact survey on some of the main variance-reduced REINFORCE-type methods, we propose ProbAbilistic Gradient Estimation for Policy Gradient (PAGE-PG), a novel loopless variance-reduced policy gradient method based on a probabilistic switch between two types of updates. Our method is inspired by the PAGE estimator for supervised learning and leverages importance sampling to obtain an unbiased gradient estimator. We show that PAGE-PG enjoys a $\mathcal{O}\left( ε^{-3} \right)$ average sample complexity to reach an $ε$-stationary solution, which matches the sample complexity of its most competitive counterparts under the same setting. A numerical evaluation confirms the competitive performance of our method on classical control tasks.

Benjamin Gravell, Tyler Summers

Computationally tractable methods are developed for centralized goal assignment and planning of collision-free polynomial-in-time trajectories for systems of multiple aerial robots. The method first assigns robots to goals to minimize total time-in-motion based on initial trajectories. By coupling the assignment and trajectory generation, the initial motion plans tend to require only limited collision resolution. The plans are then refined by checking for potential collisions and resolving them using either start time delays or altitude assignment. Numerical experiments using both methods show significant reductions in the total time required for agents to arrive at goals with only modest additional computational effort in comparison to state-of-the-art prior work, enabling planning for thousands of agents.

Benjamin Gravell, Iman Shames, Tyler Summers

We present a midpoint policy iteration algorithm to solve linear quadratic optimal control problems in both model-based and model-free settings. The algorithm is a variation of Newton's method, and we show that in the model-based setting it achieves cubic convergence, which is superior to standard policy iteration and policy gradient algorithms that achieve quadratic and linear convergence, respectively. We also demonstrate that the algorithm can be approximately implemented without knowledge of the dynamics model by using least-squares estimates of the state-action value function from trajectory data, from which policy improvements can be obtained. With sufficient trajectory data, the policy iterates converge cubically to approximately optimal policies, and this occurs with the same available sample budget as the approximate standard policy iteration. Numerical experiments demonstrate effectiveness of the proposed algorithms.

Benjamin Gravell, Tyler Summers

Despite decades of research and recent progress in adaptive control and reinforcement learning, there remains a fundamental lack of understanding in designing controllers that provide robustness to inherent non-asymptotic uncertainties arising from models estimated with finite, noisy data. We propose a robust adaptive control algorithm that explicitly incorporates such non-asymptotic uncertainties into the control design. The algorithm has three components: (1) a least-squares nominal model estimator; (2) a bootstrap resampling method that quantifies non-asymptotic variance of the nominal model estimate; and (3) a non-conventional robust control design method using an optimal linear quadratic regulator (LQR) with multiplicative noise. A key advantage of the proposed approach is that the system identification and robust control design procedures both use stochastic uncertainty representations, so that the actual inherent statistical estimation uncertainty directly aligns with the uncertainty the robust controller is being designed against. We show through numerical experiments that the proposed robust adaptive controller can significantly outperform the certainty equivalent controller on both expected regret and measures of regret risk.

Benjamin Gravell, Yi Guo, Tyler Summers

We give algorithms for designing near-optimal sparse controllers using policy gradient with applications to control of systems corrupted by multiplicative noise, which is increasingly important in emerging complex dynamical networks. Various regularization schemes are examined and incorporated into the optimization by the use of gradient, subgradient, and proximal gradient methods. Numerical experiments on a large networked system show that the algorithms converge to performant sparse mean-square stabilizing controllers.

Benjamin Gravell, Peyman Mohajerin Esfahani, Tyler Summers

The linear quadratic regulator (LQR) problem has reemerged as an important theoretical benchmark for reinforcement learning-based control of complex dynamical systems with continuous state and action spaces. In contrast with nearly all recent work in this area, we consider multiplicative noise models, which are increasingly relevant because they explicitly incorporate inherent uncertainty and variation in the system dynamics and thereby improve robustness properties of the controller. Robustness is a critical and poorly understood issue in reinforcement learning; existing methods which do not account for uncertainty can converge to fragile policies or fail to converge at all. Additionally, intentional injection of multiplicative noise into learning algorithms can enhance robustness of policies, as observed in ad hoc work on domain randomization. Although policy gradient algorithms require optimization of a non-convex cost function, we show that the multiplicative noise LQR cost has a special property called gradient domination, which is exploited to prove global convergence of policy gradient algorithms to the globally optimum control policy with polynomial dependence on problem parameters. Results are provided both in the model-known and model-unknown settings where samples of system trajectories are used to estimate policy gradients.

Edgar Bolívar, Siavash Rezazadeh, Tyler Summers et al.

Design of robotic systems that safely and efficiently operate in uncertain operational conditions, such as rehabilitation and physical assistance robots, remains an important challenge in the field. Current methods for the design of energy efficient series elastic actuators use an optimization formulation that typically assumes known operational conditions. This approach could lead to actuators that cannot perform in uncertain environments because elongation, speed, or torque requirements may be beyond actuator specifications when the operation deviates from its nominal conditions. Addressing this gap, we propose a convex optimization formulation to design the stiffness of series elastic actuators to minimize energy consumption and satisfy actuator constraints despite uncertainty due to manufacturing of the spring, unmodeled dynamics, efficiency of the transmission, and the kinematics and kinetics of the load. In our formulation, we express energy consumption as a scalar convex-quadratic function of compliance. In the unconstrained case, this quadratic equation provides an analytical solution to the optimal value of stiffness that minimizes energy consumption for arbitrary periodic reference trajectories. As actuator constraints, we consider peak motor torque, peak motor velocity, limitations due to the speed-torque relationship of DC motors, and peak elongation of the spring. As a simulation case study, we apply our formulation to the robust design of a series elastic actuator for a powered prosthetic ankle. Our simulation results indicate that a small trade-off between energy efficiency and robustness is justified to design actuators that can operate with uncertainty.

Tyler Summers, Justin Ruths

Many important complex networks, including critical infrastructure and emerging industrial automation systems, are becoming increasingly intricate webs of interacting feedback control loops. A fundamental concern is to quantify the control properties and performance limitations of the network as a function of its dynamical structure and control architecture. We study performance bounds for networks in terms of optimal feedback control costs. We provide a set of complementary bounds as a function of the system dynamics and actuator structure. For unstable network dynamics, we characterize a tradeoff between feedback control performance and the number of control inputs, in particular showing that optimal cost can increase exponentially with the size of the network. We also derive a bound on the performance of the worst-case actuator subset for stable networks, providing insight into dynamics properties that affect the potential efficacy of actuator selection. We illustrate our results with numerical experiments that analyze performance in regular and random networks.

Tyler Summers, Changyuan Li, Maryam Kamgarpour

We consider a problem of information structure design in team decision problems and team games. We propose simple, scalable greedy algorithms for adding a set of extra information links to optimize team performance and resilience to non-cooperative and adversarial agents. We show via a simple counterexample that the set function mapping additional information links to team performance is in general not supermodular. Although this implies that the greedy algorithm is not accompanied by worst-case performance guarantees, we illustrate through numerical experiments that it can produce effective and often optimal or near optimal information structure modifications.

Tyler Summers, Iman Shames, John Lygeros et al.

We consider a network topology design problem in which an initial undirected graph underlying the network is given and the objective is to select a set of edges to add to the graph to optimize the coherence of the resulting network. We show that network coherence is a submodular function of the network topology. As a consequence, a simple greedy algorithm is guaranteed to produce near optimal edge set selections. We also show that fast rank one updates of the Laplacian pseudoinverse using generalizations of the Sherman-Morrison formula and an accelerated variant of the greedy algorithm can speed up the algorithm by several orders of magnitude in practice. These allow our algorithms to scale to network sizes far beyond those that can be handled by convex relaxation heuristics.