Mark W. Mueller

Dingqi Zhang, Antonio Loquercio, Xiangyu Wu et al. · berkeley

This paper proposes an adaptive near-hover position controller for quadcopters, which can be deployed to quadcopters of very different mass, size and motor constants, and also shows rapid adaptation to unknown disturbances during runtime. The core algorithmic idea is to learn a single policy that can adapt online at test time not only to the disturbances applied to the drone, but also to the robot dynamics and hardware in the same framework. We achieve this by training a neural network to estimate a latent representation of the robot and environment parameters, which is used to condition the behaviour of the controller, also represented as a neural network. We train both networks exclusively in simulation with the goal of flying the quadcopters to goal positions and avoiding crashes to the ground. We directly deploy the same controller trained in the simulation without any modifications on two quadcopters in the real world with differences in mass, size, motors, and propellers with mass differing by 4.5 times. In addition, we show rapid adaptation to sudden and large disturbances up to one-third of the mass of the quadcopters. We perform an extensive evaluation in both simulation and the physical world, where we outperform a state-of-the-art learning-based adaptive controller and a traditional PID controller specifically tuned to each platform individually. Video results can be found at https://youtu.be/U-c-LbTfvoA.

Shuxiao Chen, Bike Zhang, Mark W. Mueller et al.

Reinforcement learning (RL) has become a promising approach to developing controllers for quadrupedal robots. Conventionally, an RL design for locomotion follows a position-based paradigm, wherein an RL policy outputs target joint positions at a low frequency that are then tracked by a high-frequency proportional-derivative (PD) controller to produce joint torques. In contrast, for the model-based control of quadrupedal locomotion, there has been a paradigm shift from position-based control to torque-based control. In light of the recent advances in model-based control, we explore an alternative to the position-based RL paradigm, by introducing a torque-based RL framework, where an RL policy directly predicts joint torques at a high frequency, thus circumventing the use of a PD controller. The proposed learning torque control framework is validated with extensive experiments, in which a quadruped is capable of traversing various terrain and resisting external disturbances while following user-specified commands. Furthermore, compared to learning position control, learning torque control demonstrates the potential to achieve a higher reward and is more robust to significant external disturbances. To our knowledge, this is the first sim-to-real attempt for end-to-end learning torque control of quadrupedal locomotion.

Carlo Bosio, Matteo Guarrera, Alberto Sangiovanni-Vincentelli et al.

Large Language models (LLMs) have shown promise as generators of symbolic control policies, producing interpretable program-like representations through iterative search. However, these models are not capable of separating the functional structure of a policy from the numerical values it is parametrized by, thus making the search process slow and inefficient. We propose a hybrid approach that decouples structural synthesis from parameter optimization by introducing an additional optimization layer for local parameter search. In our method, the numerical parameters of LLM-generated programs are extracted and optimized numerically to maximize task performance. With this integration, an LLM iterates over the functional structure of programs, while a separate optimization loop is used to find a locally optimal set of parameters accompanying candidate programs. We evaluate our method on a set of control tasks, showing that it achieves higher returns and improved sample efficiency compared to purely LLM-guided search. We show that combining symbolic program synthesis with numerical optimization yields interpretable yet high-performing policies, bridging the gap between language-model-guided design and classical control tuning. Our code is available at https://sites.google.com/berkeley.edu/colmo.

Eric Foss, Andrew Tai, Carlo Bosio et al.

Collaborative aerial transportation of tethered payloads is fundamentally limited by space, power, and weight constraints. Conventional approaches rely on static equilibrium conditions, where each vehicle tilts to generate the forces that ensure they maintain a formation geometry that avoids aerodynamic interactions and collision. This horizontal thrust component represents a significant energy penalty compared to the ideal case in which each vehicle produces purely vertical thrust to lift the payload. Operating in tighter tether configurations can minimize this effect, but at the cost of either having to fly the vehicles in closer proximity, which risks collision, or significantly increasing the length of the tether, which increases complexity and reduces potential use-cases. We propose operating the tether-suspended flying system at a rotating equilibrium. By maintaining steady circular motion, centrifugal forces provide the necessary horizontal tether tension, allowing each quadrotor to generate purely vertical thrust and thus reducing the total force (and power) required compared to an equilibrium where the thrusts are not vertical. It also allows for a wider range of tether configurations to be used without sacrificing efficiency. Results demonstrate that rotating equilibria can reduce power consumption relative to static lifting by up to 20%, making collaborative aerial solutions more practically relevant.

Xiangyu Wu, Jun Zeng, Andrea Tagliabue et al.

Limited flight distance and time is a common problem for multicopters. We propose a method for finding the optimal speed and sideslip angle of a multicopter flying a given path to achieve either the longest flight distance or time. Since flight speed and sideslip are often free variables in multicopter path planning, they can be changed without changing the mission. The proposed method is based on a novel multivariable extremum seeking controller with adaptive step size, which is inspired by recent work from the machine learning community on stochastic optimization. Our method (a) does not require a power consumption model of the vehicle, (b) is computationally efficient and runs on low-cost embedded computers in real-time, and (c) converges faster than the standard extremum seeking controller with constant step size. We prove the stability of this approach and validate it through outdoor experiments. The method is shown to converge with different payloads and in the presence of wind. Compared to flying at the maximum achievable speed in the experiments with a uniformly selected random sideslip angle, flying at the optimal range speed and sideslip on average increases the flight range by 14.3% without payload and 19.4% with a box payload. In addition, compared to hovering, flying at the optimal endurance speed and sideslip increases the flight time by 7.5% without payload and 14.4% with a box payload. A video can be found at https://youtu.be/aLds8LVfogk.

Shuxiao Chen, Xiangyu Wu, Mark W. Mueller et al.

The capabilities of autonomous flight with unmanned aerial vehicles (UAVs) have significantly increased in recent times. However, basic problems such as fast and robust geo-localization in GPS-denied environments still remain unsolved. Existing research has primarily concentrated on improving the accuracy of localization at the cost of long and varying computation time in various situations, which often necessitates the use of powerful ground station machines. In order to make image-based geo-localization online and pragmatic for lightweight embedded systems on UAVs, we propose a framework that is reliable in changing scenes, flexible about computing resource allocation and adaptable to common camera placements. The framework is comprised of two stages: offline database preparation and online inference. At the first stage, color images and depth maps are rendered as seen from potential vehicle poses quantized over the satellite and topography maps of anticipated flying areas. A database is then populated with the global and local descriptors of the rendered images. At the second stage, for each captured real-world query image, top global matches are retrieved from the database and the vehicle pose is further refined via local descriptor matching. We present field experiments of image-based localization on two different UAV platforms to validate our results.

Nathan Bucki, Jerry Tang, Mark W. Mueller

A novel quadcopter capable of changing shape mid-flight is presented, allowing for operation in four configurations with the capability of sustained hover in three. This is accomplished without requiring actuators beyond the four motors typical of a quadcopter. Morphing is achieved through freely-rotating hinges that allow the vehicle arms to fold downwards by either reducing or reversing thrust forces. Constraints placed on the control inputs of the vehicle prevent the arms from folding or unfolding unexpectedly. This allows for the use of existing quadcopter controllers and trajectory generation algorithms with only minimal added complexity. For our experimental vehicle at hover, we find that these constraints result in a 36% reduction of the maximum yaw torque the vehicle can produce, but do not result in a reduction of the maximum thrust or roll and pitch torques. Experimental results show that, for a typical maneuver, the added limits have a negligible effect on trajectory tracking performance. Finally, the ability to change configurations is shown to enable the vehicle to traverse small passages, perch on hanging wires, and perform limited grasping tasks.

Junseok Lee, Xiangyu Wu, Seung Jae Lee et al.

This paper introduces a collision avoidance system for navigating a multicopter in cluttered outdoor environments based on the recent memory-less motion planner, rectangular pyramid partitioning using integrated depth sensors (RAPPIDS). The RAPPIDS motion planner generates collision-free flight trajectories at high speed with low computational cost using only the latest depth image. In this work we extend it to improve the performance of the planner by taking the following issues into account. (a) Changes in the dynamic characteristics of the multicopter that occur during flight, such as changes in motor input/output characteristics due to battery voltage drop. (b) The noise of the flight sensor, which can cause unwanted control input components. (c) Planner utility function which may not be suitable for the cluttered environment. Therefore, in this paper we introduce solutions to each of the above problems and propose a system for the successful operation of the RAPPIDS planner in an outdoor cluttered flight environment. At the end of the paper, we validate the proposed method's effectiveness by presenting the flight experiment results in a forest environment. A video can be found at www.youtube.com/channel/UCK-gErmvZlBODN5gQpNcpsg

Karan P. Jain, Minos Park, Mark W. Mueller

Modular robots have been rising in popularity for a variety of applications, and autonomous midair docking is a necessary task for real world deployment of these robots. We present a state estimator based on the extended Kalman filter for relative localization of one multirotor with respect to another using only onboard sensors, specifically an inertial measurement unit and a camera-marker pair. Acceleration and angular velocity measurements along with relative pose measurements from a camera on the first multirotor looking at a marker on the second multirotor are used to estimate the relative position and velocity of the first multirotor with respect to the second, and the absolute attitude of the first multirotor. We also present a control architecture to use these onboard state estimates to control the first multirotor at a desired setpoint with respect to the second. The performance of the estimator and control architecture are experimentally validated by successfully and repeatably performing midair docking -- a task that requires relative position precision on the order of a centimeter.

Jiaming Zha, Mark W. Mueller

Multicopters with collision-resilient designs can operate with trajectories involving collisions. This paper presents a sampling-based method that can exploit collisions for better motion planning. The method is built upon the basis of the RRT* algorithm and takes advantages of fast motion primitive generation and collision checking for multicopters. It generates collision states by detecting potential intersections between motion primitives and obstacles, and connects these states with other sampled states to form collision-inclusive trajectories. We show that allowing collision helps improve the performance of the sampling-based planner in narrow spaces like tunnels. Finally, an experiment of tracking the trajectory generated by the collision-inclusive planner is presented.

Karan P. Jain, Jerry Tang, Koushil Sreenath et al.

Energy sources such as batteries do not decrease in mass after consumption, unlike combustion-based fuels. We present the concept of staging energy sources, i.e. consuming energy in stages and ejecting used stages, to progressively reduce the mass of aerial vehicles in-flight which reduces power consumption, and consequently increases flight time. A flight time vs. energy storage mass analysis is presented to show the endurance benefit of staging to multirotors. We consider two specific problems in discrete staging -- optimal order of staging given a certain number of energy sources, and optimal partitioning of a given energy storage mass budget into a given number of stages. We then derive results for two continuously staged cases -- an internal combustion engine driving propellers and a rocket engine. Notably, we show that a multicopter powered by internal combustion has an upper limit on achievable flight time independent of the available fuel mass, but no such limit exists for rocket propulsion. Lastly, we validate the analysis with flight experiments on a custom two-stage battery-powered quadcopter. This quadcopter can eject a battery stage after consumption in-flight using a custom-designed mechanism, and continue hovering using the next stage. The experimental flight times match well with those predicted from the analysis for our vehicle. We achieve a 19% increase in flight time using the batteries in two stages as compared to a single stage.

Jiaming Zha, Xiangyu Wu, Joseph Kroeger et al.

Aerial vehicles with collision resilience can operate with more confidence in environments with obstacles that are hard to detect and avoid. This paper presents the methodology used to design a collision resilient aerial vehicle with icosahedron tensegrity structure. A simplified stress analysis of the tensegrity frame under impact forces is performed to guide the selection of its components. In addition, an autonomous controller is presented to reorient the vehicle from an arbitrary orientation on the ground to help it take off. Experiments show that the vehicle can successfully reorient itself after landing upside-down and can survive collisions with speed up to 6.5m/s.

Jaeseung Byun, Simo A. Mäkiharju, Mark W. Mueller

This paper presents a round-trip strategy of multirotors subject to unknown flow disturbances. During the outbound flight, the vehicle immediately utilizes the wind disturbance estimations in feedback control, as an attempt to reduce the tracking error. During this phase, the disturbance estimations with respect to the position are also recorded for future use. For the return flight, the disturbances previously collected are then routed through a feedforward controller. The major assumption here is that the disturbances may vary over space, but not over time during the same mission. We demonstrate the effectiveness of this feedforward strategy via experiments with two different types of wind flows; a simple jet flow and a more complex flow. To use as a baseline case, a cascaded PD controller with an additional feedback loop for disturbance estimation was employed for outbound flights. To display our contributions regarding the additional feedforward approach, an additional feedforward correction term obtained via prerecorded data was integrated for the return flight. Compared to the baseline controller, the feedforward controller was observed to produce 43% less RMSE position error at a vehicle ground velocity of 1 m/s with 6 m/s of environmental wind velocity. This feedforward approach also produced 14% less RMSE position error for the complex flows as well.

Xiangyu Wu, Mark W. Mueller

Limited flight range is a common problem for multicopters. To alleviate this problem, we propose a method for finding the optimal speed and heading of a multicopter when flying a given path to achieve the longest flight range. Based on a novel multivariable extremum seeking controller with adaptive step size, the method (a) does not require any power consumption model of the vehicle, (b) can adapt to unknown disturbances, (c) can be executed online, and (d) converges faster than the standard extremum seeking controller with constant step size. We conducted indoor experiments to validate the effectiveness of this method under different payloads and initial conditions, and showed that it is able to converge more than 30% faster than the standard extremum seeking controller. This method is especially useful for applications such as package delivery, where the size and weight of the payload differ for different deliveries and the power consumption of the vehicle is hard to model.

Nathan Bucki, Junseok Lee, Mark W. Mueller

We present RAPPIDS: a novel collision checking and planning algorithm for multicopters that is capable of quickly finding local collision-free trajectories given a single depth image from an onboard camera. The primary contribution of this work is a new pyramid-based spatial partitioning method that enables rapid collision detection between candidate trajectories and the environment. By leveraging the efficiency of our collision checking method, we shown how a local planning algorithm can be run at high rates on computationally constrained hardware, evaluating thousands of candidate trajectories in milliseconds. The performance of the algorithm is compared to existing collision checking methods in simulation, showing our method to be capable of evaluating orders of magnitude more trajectories per second. Experimental results are presented showing a quadcopter quickly navigating a previously unseen cluttered environment by running the algorithm on an ODROID-XU4 at 30 Hz.

Karan P. Jain, Mark W. Mueller

We present a novel approach to increase the flight time of a multirotor via mid-air docking and in-flight battery switching. A main quadcopter flying using a primary battery has a docking platform attached to it. A 'flying battery' - a small quadcopter carrying a secondary battery - is equipped with docking legs that can mate with the main quadcopter's platform. Connectors between the legs and the platform establish electrical contact on docking, and enable power transfer from the secondary battery to the main quadcopter. A custom-designed circuit allows arbitrary switching between the primary battery and secondary battery. We demonstrate the concept in a flight experiment involving repeated docking, battery switching, and undocking. The experiment increases the flight time of the main quadcopter by a factor of 4.7x compared to solo flight, and 2.2x a theoretical limit for that given multirotor. Importantly, this increase in flight time is not associated with a large increase in overall vehicle mass or size, leaving the main quadcopter in fundamentally the same safety class.

Nathan Bucki, Mark W. Mueller

We present a continuous-time collision detection algorithm for quickly detecting whether certain polynomial trajectories in time intersect with convex obstacles. The algorithm is used in conjunction with an existing multicopter trajectory generation method to achieve rapid, obstacle-aware motion planning in environments with both static convex obstacles and dynamic convex obstacles whose boundaries do not rotate. In general, this problem is difficult because the presence of convex obstacles makes the feasible space of trajectories nonconvex. The performance of the algorithm is benchmarked using Monte Carlo simulations, and experimental results are presented that demonstrate the use of the method to plan collision-free multicopter trajectories in milliseconds in environments with both static and dynamic obstacles.

Saman Fahandezh-Saadi, Mark W. Mueller

This paper presents a method to improve the localization accuracy of robots operating in a range-based localization network. The method is favorable especially when the robots operate in harsh environments where the access to a robust and reliable localization system is limited. A state estimator is used for a six degree of freedom object using inertial sensors as well as an Ultra-wideband (UWB) range measurement sensor. The estimator is incorporated into an adaptive algorithm, improving the localization quality of an agent by using a mobile UWB ranging sensor, where the mobile anchor moves to improve localization quality. The algorithm reconstructs localization network in real-time to minimize the determinant of the covariance matrix in the sense of least square error. Finally, the proposed algorithm is experimentally validated in a network consisting of one mobile and four fixed anchors.

Saman Fahandezh-Saadi, Mark W. Mueller

A state estimator is derived for an agent with the ability to measure single ranges to fixed points in its environment, and equipped with an accelerometer and a rate gyroscope. The state estimator makes no agent-specific assumptions, and can be immediately applied to any rigid body with these sensors. Also, the state estimator doesn't use any trilateration-based method to calculate position from range measurements. As the considered system can only make a single range measurement at a time, we present a greedy optimization algorithm for selecting the best measurement. Experiments in an indoor testbed using an externally controlled multicopter demonstrate the efficacy of the algorithm, specifically showing an improvement over a naïve strategy of a fixed sequence of measurements. In separate experiments, the algorithm is also used in feedback control, to control the position of the multicopter.