Christophe Prieur

Humberto Stein Shiromoto, Vincent Andrieu, Christophe Prieur

We consider nonlinear control systems for which there exist some structural obstacles to the design of classical continuous stabilizing feedback laws. More precisely, it is studied systems for which the backstepping tool for the design of stabilizers can not be applied. On the contrary, it leads to feedback laws such that the origin of the closed-loop system is not globally asymptotically stable, but a suitable attractor (strictly containing the origin) is practically asymptotically stable. Then, a design method is suggested to build a hybrid feedback law combining a backstepping controller with a locally stabilizing controller. The results are illustrated for a nonlinear system which, due to the structure of the system, does not have a priori any globally stabilizing backstepping controller.

Humberto Stein Shiromoto, Vincent Andrieu, Christophe Prieur

For an ISS system, by analyzing local and non-local properties, it is obtained different input-to-state gains. The interconnection of a system having two input-to-state gains with a system having a single ISS gain is analyzed. By employing the Small Gain Theorem for the local (resp. non-local) gains composition, it is concluded about the local (resp. global) stability of the origin (resp. of a compact set). Additionally, if the region of local stability of the origin strictly includes the region attraction of the compact set, then it is shown that the origin is globally asymptotically stable. An example illustrates the approach.

Humberto Stein Shiromoto, Vincent Andrieu, Christophe Prieur

Systems for which the backstepping technique cannot be applied are considered. A criterion for the design of a hybrid feedback law is proposed by blending a local stabilizer with a backstepping controller. This hybrid feedback law renders the origin globally asymptotically stable for the closed-loop system. The selection criterion is based on choice of the size of a compact set included in the basin of attraction of the local controller. The results are illustrated by simulations.

Christophe Prieur, Mircea Lazar, Bogdan Robu

In this paper we consider the limiting case of neural networks (NNs) architectures when the number of neurons in each hidden layer and the number of hidden layers tend to infinity thus forming a continuum, and we derive approximation errors as a function of the number of neurons and/or hidden layers. Firstly, we consider the case of neural networks with a single hidden layer and we derive an integral infinite width neural representation that generalizes existing continuous neural networks (CNNs) representations. Then we extend this to deep residual CNNs that have a finite number of integral hidden layers and residual connections. Secondly, we revisit the relation between neural ODEs and deep residual NNs and we formalize approximation errors via discretization techniques. Then, we merge these two approaches into a unified homogeneous representation of NNs as a Distributed Parameter neural Network (DiPaNet) and we show that most of the existing finite and infinite-dimensional NNs architectures are related via homogenization/discretization with the DiPaNet representation. Our approach is purely deterministic and applies to general, uniformly continuous matrix weight functions. Relations with neural fields and other neural integro-differential equations are discussed along with further possible generalizations and applications of the DiPaNet framework.

Paul Daoudi, Bogdan Robu, Christophe Prieur et al.

This paper addresses the problem of integrating local guide policies into a Reinforcement Learning agent. For this, we show how to adapt existing algorithms to this setting before introducing a novel algorithm based on a noisy policy-switching procedure. This approach builds on a proper Approximate Policy Evaluation (APE) scheme to provide a perturbation that carefully leads the local guides towards better actions. We evaluated our method on a set of classical Reinforcement Learning problems, including safety-critical systems where the agent cannot enter some areas at the risk of triggering catastrophic consequences. In all the proposed environments, our agent proved to be efficient at leveraging those policies to improve the performance of any APE-based Reinforcement Learning algorithm, especially in its first learning stages.

Paul Daoudi, Bojan Mavkov, Bogdan Robu et al.

This paper presents a learning-based control strategy for non-linear throttle valves with an asymmetric hysteresis, leading to a near-optimal controller without requiring any prior knowledge about the environment. We start with a carefully tuned Proportional Integrator (PI) controller and exploit the recent advances in Reinforcement Learning (RL) with Guides to improve the closed-loop behavior by learning from the additional interactions with the valve. We test the proposed control method in various scenarios on three different valves, all highlighting the benefits of combining both PI and RL frameworks to improve control performance in non-linear stochastic systems. In all the experimental test cases, the resulting agent has a better sample efficiency than traditional RL agents and outperforms the PI controller.

Paul Daoudi, Christophe Prieur, Bogdan Robu et al.

Off-dynamics Reinforcement Learning (ODRL) seeks to transfer a policy from a source environment to a target environment characterized by distinct yet similar dynamics. In this context, traditional RL agents depend excessively on the dynamics of the source environment, resulting in the discovery of policies that excel in this environment but fail to provide reasonable performance in the target one. In the few-shot framework, a limited number of transitions from the target environment are introduced to facilitate a more effective transfer. Addressing this challenge, we propose an innovative approach inspired by recent advancements in Imitation Learning and conservative RL algorithms. The proposed method introduces a penalty to regulate the trajectories generated by the source-trained policy. We evaluate our method across various environments representing diverse off-dynamics conditions, where access to the target environment is extremely limited. These experiments include high-dimensional systems relevant to real-world applications. Across most tested scenarios, our proposed method demonstrates performance improvements compared to existing baselines.

Aneel Tanwani, Christophe Prieur, Sophie Tarbouriech

We consider a system of linear hyperbolic PDEs where the state at one of the boundary points is controlled using the measurements of another boundary point. Because of the disturbances in the measurement, the problem of designing dynamic controllers is considered so that the closed-loop system is robust with respect to measurement errors. Assuming that the disturbance is a locally essentially bounded measurable function of time, we derive a disturbance-to-state estimate which provides an upper bound on the maximum norm of the state (with respect to the spatial variable) at each time in terms of $\mathcal{L}^\infty$-norm of the disturbance up to that time. The analysis is based on constructing a Lyapunov function for the closed-loop system, which leads to controller synthesis and the conditions on system dynamics required for stability. As an application of this stability notion, the problem of quantized control for hyperbolic PDEs is considered where the measurements sent to the controller are communicated using a quantizer of finite length. The presence of quantizer yields practical stability only, and the ultimate bounds on the norm of the state trajectory are also derived.