Tobias Damm, Peter Benner, Jan Hauth
The stochastic $H^\infty$-norm is defined as the $L^2$-induced norm of the input-output operator of a stochastic linear system. Like the deterministic $H^\infty$-norm it is characterised by a version of the bounded real lemma, but without a frequency domain description or a Hamiltonian condition. Therefore, we base its computation on a parametrised algebraic Riccati-type matrix equation.
Tobias Damm, Kazuhiro Sato, Axel Vierling
We suggest and compare different methods for the numerical solution of Lyapunov like equations with application to control of Markovian jump linear systems. First, we consider fixed point iterations and associated Krylov subspace formulations. Second, we reformulate the equation as an optimization problem and consider steepest descent, conjugate gradient, and a trust-region method. Numerical experiments illustrate that for large-scale problems the trust-region method is more effective than the steepest descent and the conjugate gradient methods. The fixed-point approach, however, is superior to the optimization methods. As an application we consider a networked control system, where the Markov jumps are induced by the wireless communication protocol.
Tatjana Krau, Jorge Mandlmaier, Tobias Damm et al.
Reinforcement Learning (RL) has demonstrated strong potential for industrial process control, yet policies trained in simulation often suffer from a significant sim-to-real gap when deployed on physical hardware. This work systematically analyzes how core Markov Decision Process (MDP) design choices -- state composition, target inclusion, reward formulation, termination criteria, and environment dynamics models -- affect this transfer. Using a color mixing task, we evaluate different MDP configurations and mixing dynamics across simulation and real-world experiments. We validate our findings on physical hardware, demonstrating that physics-based dynamics models achieve up to 50% real-world success under strict precision constraints where simplified models fail entirely. Our results provide practical MDP design guidelines for deploying RL in industrial process control.
Peter Benner, Tobias Damm, Yolanda Rocio Rodriguez Cruz
We consider two approaches to balanced truncation of stochastic linear systems, which follow from different generalizations of the reachability Gramian of deterministic systems. Both preserve mean-square asymptotic stability, but only the second leads to a stochastic H-infinity-type bound for the approximation error of the truncated system.