W.P.M.H. Heemels

M. C. F. Donkers, P. Tabuada, W. P. M. H. Heemels

In this paper, we present two control laws that are tailored for control applications in which computational and/or communication resources are scarce. Namely, we consider minimum attention control, where the `attention' that a control task requires is minimised given certain performance requirements, and anytime attention control, where the performance under the `attention' given by a scheduler is maximised. Here, we interpret `attention' as the inverse of the time elapsed between two consecutive executions of a control task. By focussing on linear plants, by allowing for only a finite number of possible intervals between two subsequent executions of the control task, by making a novel extension to the notion of control Lyapunov functions and taking these novel extended control Lyapunov function to be infinity-norm-based, we can formulate the aforementioned control problems as online linear programs, which can be solved efficiently. Furthermore, we provide techniques to construct suitable infinity-norm-based extended control Lyapunov functions for our purposes. Finally, we illustrate the resulting control laws using numerical examples. In particular, we show that minimum attention control outperforms an alternative implementation-aware control law available in the literature.

Timo de Groot, Maurice Heemels, Tom Oomen et al.

Reset systems can overcome fundamental limitations of linear time-invariant control. The recently introduced notion of scaled (relative) graphs provides a promising framework for developing graphical analysis and design tools for reset systems, in line with widely adopted loopshaping methods for linear systems. The aim of this paper is to derive techniques for over-bounding the scaled graph of reset systems, and obtain insights in their accuracy. We exploit connections between quadratic dissipativity and scaled graphs to recast the over-bounding problem as the search for piecewise quadratic storage functions. Using specific sampling techniques, we reveal a fundamental limitation of general scaled graph approximation methods that are based on quadratic dissipativity.

J. S. van Hulst, W. P. M. H. Heemels, D. J. Antunes

Reinforcement learning (RL) is a powerful framework for decision-making in uncertain environments, but it often requires large amounts of data to learn an optimal policy. We address this challenge by incorporating prior model knowledge to guide exploration and accelerate the learning process. Specifically, we assume access to a model set that contains the true transition kernel and reward function. We optimize over this model set to obtain upper and lower bounds on the Q-function, which are then used to guide the exploration of the agent. We provide theoretical guarantees on the convergence of the Q-function to the optimal Q-function under the proposed class of exploring policies. Furthermore, we also introduce a data-driven regularized version of the model set optimization problem that ensures the convergence of the class of exploring policies to the optimal policy. Lastly, we show that when the model set has a specific structure, namely the bounded-parameter MDP (BMDP) framework, the regularized model set optimization problem becomes convex and simple to implement. In this setting, we also prove finite-time convergence to the optimal policy under mild assumptions. We demonstrate the effectiveness of the proposed exploration strategy, which we call BUMEX (Bounded Uncertainty Model-based Exploration), in a simulation study. The results indicate that the proposed method can significantly accelerate learning in benchmark examples. A toolbox is available at https://github.com/JvHulst/BUMEX.

Yuting Fu, Jochen Seemann, Caspar Hanselaar et al.

Automated Driving (AD) systems have the potential to increase safety, comfort and energy efficiency. Recently, major automotive companies have started testing and validating AD systems (ADS) on public roads. Nevertheless, the commercial deployment and wide adoption of ADS have been moderate, partially due to system functional insufficiencies (FI) that undermine passenger safety and lead to hazardous situations on the road. FIs are defined in ISO 21448 Safety Of The Intended Functionality (SOTIF). FIs are insufficiencies in sensors, actuators and algorithm implementations, including neural networks and probabilistic calculations. Examples of FIs in ADS include inaccurate ego-vehicle localization on the road, incorrect prediction of a cyclist maneuver, unreliable detection of a pedestrian, etc. The main goal of our study is to formulate a generic architectural design pattern, which is compatible with existing methods and ADS, to improve FI mitigation and enable faster commercial deployment of ADS. First, we studied the 2021 autonomous vehicles disengagement reports published by the California Department of Motor Vehicles (DMV). The data clearly show that disengagements are five times more often caused by FIs rather than by system faults. We then made a comprehensive list of insufficiencies and their characteristics by analyzing over 10 hours of publicly available road test videos. In particular, we identified insufficiency types in four major categories: world model, motion plan, traffic rule, and operational design domain. The insufficiency characterization helps making the SOTIF analyses of triggering conditions more systematic and comprehensive. Based on our FI characterization, simulation experiments and literature survey, we define a novel generic architectural design pattern Daruma to dynamically select the channel that is least likely to have a FI at the moment.