Coen de Visser

Prashant Solanki, Isabelle El-Hajj, Jasper van Beers et al.

We unify Hamilton-Jacobi (HJ) reachability and Reinforcement Learning (RL) through a proposed running cost formulation. We prove that the resultant travel-cost value function is the unique bounded viscosity solution of a time-dependent Hamilton-Jacobi Bellman (HJB) Partial Differential Equation (PDE) with zero terminal data, whose negative sublevel set equals the strict backward-reachable tube. Using a forward reparameterization and a contraction inducing Bellman update, we show that fixed points of small-step RL value iteration converge to the viscosity solution of the forward discounted HJB. Experiments on a classical benchmark validate this connection by demonstrating convergence of learned value functions toward semi-Lagrangian HJB solutions and by quantifying approximation error across the state space. These results empirically support the theoretical analysis, showing that the proposed framework preserves reachability-based safety semantics while remaining compatible with deep RL implementations.

Sihao Sun, Giovanni Cioffi, Coen de Visser et al.

Fault-tolerant control is crucial for safety-critical systems, such as quadrotors. State-of-art flight controllers can stabilize and control a quadrotor even when subjected to the complete loss of a rotor. However, these methods rely on external sensors, such as GPS or motion capture systems, for state estimation. To the best of our knowledge, this has not yet been achieved with only onboard sensors. In this paper, we propose the first algorithm that combines fault-tolerant control and onboard vision-based state estimation to achieve position control of a quadrotor subjected to complete failure of one rotor. Experimental validations show that our approach is able to accurately control the position of a quadrotor during a motor failure scenario, without the aid of any external sensors. The primary challenge to vision-based state estimation stems from the inevitable high-speed yaw rotation (over 20 rad/s) of the damaged quadrotor, causing motion blur to cameras, which is detrimental to visual inertial odometry (VIO). We compare two types of visual inputs to the vision-based state estimation algorithm: standard frames and events. Experimental results show the advantage of using an event camera especially in low light environments due to its inherent high dynamic range and high temporal resolution. We believe that our approach will render autonomous quadrotors safer in both GPS denied or degraded environments. We release both our controller and VIO algorithm open source.

Jasper van Beers, Coen de Visser

Ensuring the reliability and validity of data-driven quadrotor model predictions is essential for their accepted and practical use. This is especially true for grey- and black-box models wherein the mapping of inputs to predictions is not transparent and subsequent reliability notoriously difficult to ascertain. Nonetheless, such techniques are frequently and successfully used to identify quadrotor models. Prediction intervals (PIs) may be employed to provide insight into the consistency and accuracy of model predictions. This paper estimates such PIs for polynomial and Artificial Neural Network (ANN) quadrotor aerodynamic models. Two existing ANN PI estimation techniques - the bootstrap method and the quality driven method - are validated numerically for quadrotor aerodynamic models using an existing high-fidelity quadrotor simulation. Quadrotor aerodynamic models are then identified on real quadrotor flight data to demonstrate their utility and explore their sensitivity to model interpolation and extrapolation. It is found that the ANN-based PIs widen considerably when extrapolating and remain constant, or shrink, when interpolating. While this behaviour also occurs for the polynomial PIs, it is of lower magnitude. The estimated PIs establish probabilistic bounds within which the quadrotor model outputs will likely lie, subject to modelling and measurement uncertainties that are reflected through the PI widths.

Prashant Solanki, Jasper Van Beers, Coen De Visser

This paper studies obstacle avoidance under translation invariant dynamics using an avoid-side travel cost Hamilton Jacobi formulation. For running costs that are zero outside an obstacle and strictly negative inside it, we prove that the value function is non-positive everywhere, equals zero exactly outside the avoid set, and is strictly negative exactly on it. Under translation invariance, this yields a reuse principle: the value of any translated obstacle is obtained by translating a single template value function. We show that the pointwise minimum of translated template values exactly characterizes the union of the translated single-obstacle avoid sets and provides a conservative inner certificate of unavoidable collision in clutter. To reduce conservatism, we introduce a blockwise composition framework in which subsets of obstacles are merged and solved jointly. This yields a hierarchy of conservative certificates from singleton reuse to the exact clutter value, together with monotonicity under block merging and an exactness criterion based on the existence of a common clutter avoiding control. The framework is illustrated on a Dubins car example in a repeated clutter field.

Prashant Solanki, Nikolaus Vertovec, Yannik Schnitzer et al.

Recent approaches to leveraging deep learning for computing reachable sets of continuous-time dynamical systems have gained popularity over traditional level-set methods, as they overcome the curse of dimensionality. However, as with level-set methods, considerable care needs to be taken in limiting approximation errors, particularly since no guarantees are provided during training on the accuracy of the learned reachable set. To address this limitation, we introduce an epsilon-approximate Hamilton-Jacobi Partial Differential Equation (HJ-PDE), which establishes a relationship between training loss and accuracy of the true reachable set. To formally certify this approximation, we leverage Satisfiability Modulo Theories (SMT) solvers to bound the residual error of the HJ-based loss function across the domain of interest. Leveraging Counter Example Guided Inductive Synthesis (CEGIS), we close the loop around learning and verification, by fine-tuning the neural network on counterexamples found by the SMT solver, thus improving the accuracy of the learned reachable set. To the best of our knowledge, Certified Approximate Reachability (CARe) is the first approach to provide soundness guarantees on learned reachable sets of continuous dynamical systems.

Bram Strack van Schijndel, Sihao Sun, Coen de Visser

This paper presents a novel method for fast and robust detection of actuator failures on quadrotors. The proposed algorithm has very little model dependency. A Kalman filter estimator estimates a stochastic effectiveness factor for every actuator, using only onboard RPM, gyro and accelerometer measurements. Then, a hypothesis test identifies the failed actuator. This algorithm is validated online in real-time, also as part of an active fault tolerant control system. Loss of actuator effectiveness is induced by ejecting the propellers from the motors. The robustness of this algorithm is further investigated offline over a range of parameter settings by replaying real flight data containing 26 propeller ejections. The detection delays are found to be in the 30 to 130 ms range, without missed detections or false alarms occurring.

Sihao Sun, Matthias Baert, Bram Adriaan Strack van Schijndel et al.

This study has developed a fault-tolerant controller that is able to recover a quadrotor from arbitrary initial orientations and angular velocities, despite the complete failure of a rotor. This cascaded control method includes a position/altitude controller, an almost-global convergence attitude controller, and a control allocation method based on quadratic programming. As a major novelty, a constraint of undesirable angular velocity is derived and fused into the control allocator, which significantly improves the recovery performance. For validation, we have conducted a set of Monte-Carlo simulation to test the reliability of the proposed method of recovering the quadrotor from arbitrary initial attitude/rate conditions. In addition, real-life flight tests have been performed. The results demonstrate that the post-failure quadrotor can recover after being casually tossed into the air.

Sihao Sun, Xuerui Wang, Qiping Chu et al.

In order to further expand the flight envelope of quadrotors under actuator failures, we design a nonlinear sensor-based fault-tolerant controller to stabilize a quadrotor with failure of two opposing rotors in the high-speed flight condition (> 8m/s). The incremental nonlinear dynamic inversion (INDI) approach which excels in handling model uncertainties is adopted to compensate for the significant unknown aerodynamic effects. The internal dynamics of such an underactuated system have been analyzed, and subsequently stabilized by re-defining the control output. The proposed method can be generalized to control a quadrotor under single-rotor-failure and nominal conditions. For validation, flight tests have been carried out in a large-scale open jet wind tunnel. The position of a damaged quadrotor can be controlled in the presence of significant wind disturbances. A linear quadratic regulator (LQR) approach from the literature has been compared to demonstrate the advantages of the proposed nonlinear method in the windy and high-speed flight condition.

Willem Eerland, Coen de Visser, Erik-Jan van Kampen

We define a SDP framework based on the RLSTD algorithm and multivariate simplex B-splines. We introduce a local forget factor capable of preserving the continuity of the simplex splines. This local forget factor is integrated with the RLSTD algorithm, resulting in a modified RLSTD algorithm that is capable of tracking time-varying systems. We present the results of two numerical experiments, one validating SDP and comparing it with NDP and another to show the advantages of the modified RLSTD algorithm over the original. While SDP requires more computations per time-step, the experiment shows that for the same amount of function approximator parameters, there is an increase in performance in terms of stability and learning rate compared to NDP. The second experiment shows that SDP in combination with the modified RLSTD algorithm allows for faster recovery compared to the original RLSTD algorithm when system parameters are altered, paving the way for an adaptive high-performance non-linear control method.