Ankur Mehta

Zida Wu, Mathieu Lauriere, Samuel Jia Cong Chua et al.

Mean Field Games (MFGs) have the ability to handle large-scale multi-agent systems, but learning Nash equilibria in MFGs remains a challenging task. In this paper, we propose a deep reinforcement learning (DRL) algorithm that achieves population-dependent Nash equilibrium without the need for averaging or sampling from history, inspired by Munchausen RL and Online Mirror Descent. Through the design of an additional inner-loop replay buffer, the agents can effectively learn to achieve Nash equilibrium from any distribution, mitigating catastrophic forgetting. The resulting policy can be applied to various initial distributions. Numerical experiments on four canonical examples demonstrate our algorithm has better convergence properties than SOTA algorithms, in particular a DRL version of Fictitious Play for population-dependent policies.

Zida Wu, Mathieu Lauriere, Matthieu Geist et al.

Mean Field Games (MFGs) offer a powerful framework for studying large-scale multi-agent systems. Yet, learning Nash equilibria in MFGs remains a challenging problem, particularly when the initial distribution is unknown or when the population is subject to common noise. In this paper, we introduce an efficient deep reinforcement learning (DRL) algorithm designed to achieve population-dependent Nash equilibria without relying on averaging or historical sampling, inspired by Munchausen RL and Online Mirror Descent. The resulting policy is adaptable to various initial distributions and sources of common noise. Through numerical experiments on seven canonical examples, we demonstrate that our algorithm exhibits superior convergence properties compared to state-of-the-art algorithms, particularly a DRL version of Fictitious Play for population-dependent policies. The performance in the presence of common noise underscores the robustness and adaptability of our approach.

Wenzhong Yan, Ankur Mehta

Locomotive robots that do not rely on electronics and/or electromagnetic components will open up new perspectives and applications for robotics. However, these robots usually involve complicated and tedious fabrication processes, limiting their applications. Here, we develop an easy-to-fabricate crawling robot by embedding simple control and actuation into origami-inspired mechanisms through folding, eliminating the need for discrete electronics and transducers. Our crawling robot locomotes through directional friction propelled by an onboard origami self-sustained oscillator, which generates periodic actuation from a single source of constant power. The crawling robot is lightweight (~ 3.8 gram), ultra low-cost (~ US $1), nonmagnetic, and electronic-free; it may enable practical applications in extreme environments, e.g., large radiation or magnetic fields. The robot can be fabricated through a monolithic origami-inspired folding-based method with universal materials, i.e., sheet materials and conductive threads. This rapid design and fabrication approach enables the programmable assembly of various mechanisms within this manufacturing paradigm, laying the foundation for autonomous, untethered robots without requiring electronics.

Zida Wu, Zhaoliang Zheng, Ankur Mehta

Modern autonomous systems are purposed for many challenging scenarios, where agents will face unexpected events and complicated tasks. The presence of disturbance noise with control command and unknown inputs can negatively impact robot performance. Previous research of joint input and state estimation separately studied the continuous and discrete cases without any prior information. This paper combines the continuous and discrete input cases into a unified theory based on the Expectation-Maximum (EM) algorithm. By introducing prior knowledge of events as the constraint, inequality optimization problems are formulated to determine a gain matrix or dynamic weights to realize an optimal input estimation with lower variance and more accurate decision-making. Finally, statistical results from experiments show that our algorithm owns 81\% improvement of the variance than KF and 47\% improvement than RKF in continuous space; a remarkable improvement of right decision-making probability of our input estimator in discrete space, identification ability is also analyzed by experiments.

Kenny Chen, Brett T. Lopez, Ali-akbar Agha-mohammadi et al.

Field robotics in perceptually-challenging environments require fast and accurate state estimation, but modern LiDAR sensors quickly overwhelm current odometry algorithms. To this end, this paper presents a lightweight frontend LiDAR odometry solution with consistent and accurate localization for computationally-limited robotic platforms. Our Direct LiDAR Odometry (DLO) method includes several key algorithmic innovations which prioritize computational efficiency and enables the use of dense, minimally-preprocessed point clouds to provide accurate pose estimates in real-time. This is achieved through a novel keyframing system which efficiently manages historical map information, in addition to a custom iterative closest point solver for fast point cloud registration with data structure recycling. Our method is more accurate with lower computational overhead than the current state-of-the-art and has been extensively evaluated in multiple perceptually-challenging environments on aerial and legged robots as part of NASA JPL Team CoSTAR's research and development efforts for the DARPA Subterranean Challenge.

Wenzhong Yan, Ankur Mehta

Origami-inspired robots are of particular interest given their potential for rapid and accessible design and fabrication of elegant designs and complex functionalities through cutting and folding of flexible 2D sheets or even strings, i.e.printable manufacturing. Yet, origami robots still require bulky, rigid components or electronics for actuation and control to accomplish tasks with reliability, programmability, ability to output substantial force, and durability, restricting their full potential. Here, we present a printable self-sustained compliant oscillator that generates periodic actuation using only constant electrical power, without discrete components or electronic control hardware. This oscillator is robust (9 out of 10 prototypes worked successfully on the first try), configurable (with tunable periods from 3 s to 12 s), powerful (can overcome hydrodynamic resistance to consistently propel a swimmer at ~1.6 body lengths/min), and long-lasting (~10^3 cycles); it enables driving macroscale devices with prescribed autonomous behaviors, e.g. locomotion and sequencing. This oscillator is also fully functional underwater and in high magnetic fields. Our analytical model characterizes essential parameters of the oscillation period, enabling programmable design of the oscillator. The printable oscillator can be integrated into origami-inspired systems seamlessly and monolithically, allowing rapid design and prototyping; the resulting integrated devices are lightweight, low-cost, compliant, electronic-free, and nonmagnetic, enabling practical applications in extreme areas. We demonstrate the functionalities of the oscillator with: (i) autonomous gliding of a printable swimmer, (ii) LED flashing, and (iii) fluid stirring. This work paves the way for realizing fully printable autonomous robots with a high integration of actuation and control.

Wenzhong Yan, Dawei Zhao, Ankur Mehta

We propose a computational design tool to enable casual end-users to easily design, fabricate, and assemble flat-pack furniture with guaranteed manufacturability. Using our system, users select parameterized components from a library and constrain their dimensions. Then they abstractly specify connections among components to define the furniture. Once fabrication specifications (e.g. materials) designated, the mechanical implementation of the furniture is automatically handled by leveraging encoded domain expertise. Afterwards, the system outputs 3D models for visualization and mechanical drawings for fabrication. We demonstrate the validity of our approach by designing, fabricating, and assembling a variety of flat-pack (scaled) furniture on demand.

Yuki Shirai, Xuan Lin, Ankur Mehta et al.

Although Trajectory Optimization (TO) is one of the most powerful motion planning tools, it suffers from expensive computational complexity as a time horizon increases in cluttered environments. It can also fail to converge to a globally optimal solution. In this paper, we present Lazy Trajectory Optimization (LTO) that unifies local short-horizon TO and global Graph-Search Planning (GSP) to generate a long-horizon global optimal trajectory. LTO solves TO with the same constraints as the original long-horizon TO with improved time complexity. We also propose a TO-aware cost function that can balance both solution cost and planning time. Since LTO solves many nearly identical TO in a roadmap, it can provide an informed warm-start for TO to accelerate the planning process. We also present proofs of the computational complexity and optimality of LTO. Finally, we demonstrate LTO's performance on motion planning problems for a 2 DOF free-flying robot and a 21 DOF legged robot, showing that LTO outperforms existing algorithms in terms of its runtime and reliability.

Chang Liu, Wenzhong Yan, Ankur Mehta

Orthogonally assembled double-layered corrugated (OADLC) mechanisms are a class of foldable structures that harness origami-inspired methods to enhance the structural stiffness of resulting devices; these mechanisms have extensive applications due to their lightweight, compact nature as well as their high strength-to-weight ratio. However, the design of these mechanisms remains challenging. Here, we propose an efficient method to rapidly design OADLC mechanisms from desired behavioral specifications, i.e. in-plane stiffness and out-of-plane stiffness. Based on an equivalent plate model, we develop and validate analytical formulas for the behavioral specifications of OADLC mechanisms; the analytical formulas can be described as expressions of design parameters. On the basis of the analytical expressions, we formulate the design of OADLC mechanisms from behavioral specifications into an optimization problem that minimizes the weight with given design constraints. The 2D folding patterns of the optimized OADLC mechanisms can be generated automatically and directly delivered for fabrication. Our rapid design method is demonstrated by developing stiffness-enhanced mechanisms with a desired out-of-plane stiffness for a foldable gripper that enables a blimp to perch steadily under air disturbance and weight limit.

Kenny Chen, Alexandra Pogue, Brett T. Lopez et al.

Monocular depth inference has gained tremendous attention from researchers in recent years and remains as a promising replacement for expensive time-of-flight sensors, but issues with scale acquisition and implementation overhead still plague these systems. To this end, this work presents an unsupervised learning framework that is able to predict at-scale depth maps and egomotion, in addition to camera intrinsics, from a sequence of monocular images via a single network. Our method incorporates both spatial and temporal geometric constraints to resolve depth and pose scale factors, which are enforced within the supervisory reconstruction loss functions at training time. Only unlabeled stereo sequences are required for training the weights of our single-network architecture, which reduces overall implementation overhead as compared to previous methods. Our results demonstrate strong performance when compared to the current state-of-the-art on multiple sequences of the KITTI driving dataset and can provide faster training times with its reduced network complexity.

Sahba Aghajani Pedram, Peter Walker Ferguson, Changyeob Shin et al.

In this paper, we present a synergic learning algorithm to address the task of indirect manipulation of an unknown deformable tissue. Tissue manipulation is a common yet challenging task in various surgical interventions, which makes it a good candidate for robotic automation. We propose using a linear approximate Q-learning method in which human knowledge contributes to selecting useful yet simple features of tissue manipulation while the algorithm learns to take optimal actions and accomplish the task. The algorithm is implemented and evaluated on a simulation using the OpenCV and CHAI3D libraries. Successful simulation results for four different configurations which are based on realistic tissue manipulation scenarios are presented. Results indicate that with a careful selection of relatively simple and intuitive features, the developed Q-learning algorithm can successfully learn an optimal policy without any prior knowledge of tissue dynamics or camera intrinsic/extrinsic calibration parameters.

Tsang-Kai Chang, Shengkang Chen, Ankur Mehta

Extended Kalman filter (EKF) does not guarantee consistent mean and covariance under linearization, even though it is the main framework for robotic localization. While Lie group improves the modeling of the state space in localization, the EKF on Lie group still relies on the arbitrary Gaussian assumption in face of nonlinear models. We instead use von Mises filter for orientation estimation together with the conventional Kalman filter for position estimation, and thus we are able to characterize the first two moments of the state estimates. Since the proposed algorithm holds a solid probabilistic basis, it is fundamentally relieved from the inconsistency problem. Furthermore, we extend the localization algorithm to fully circular representation even for position, which is similar to grid patterns found in mammalian brains and in recurrent neural networks. The applicability of the proposed algorithms is substantiated not only by strong mathematical foundation but also by the comparison against other common localization methods.

Amr Alanwar, Hazem Said, Ankur Mehta et al.

Distributed state estimation strongly depends on collaborative signal processing, which often requires excessive communication and computation to be executed on resource-constrained sensor nodes. To address this problem, we propose an event-triggered diffusion Kalman filter, which collects measurements and exchanges messages between nodes based on a local signal indicating the estimation error. On this basis, we develop an energy-aware state estimation algorithm that regulates the resource consumption in wireless networks and ensures the effectiveness of every consumed resource. The proposed algorithm does not require the nodes to share its local covariance matrices, and thereby allows considerably reducing the number of transmission messages. To confirm its efficiency, we apply the proposed algorithm to the distributed simultaneous localization and time synchronization problem and evaluate it on a physical testbed of a mobile quadrotor node and stationary custom ultra-wideband wireless devices. The obtained experimental results indicate that the proposed algorithm allows saving 86% of the communication overhead associated with the original diffusion Kalman filter while causing deterioration of performance by 16% only. We make the Matlab code and the real testing data available online.