Moritz Schneider

Moritz Schneider, Jens Lang, Rüdiger Weiner

Dynamical systems with sub-processes evolving on many different time scales are ubiquitous in applications. Their efficient solution is greatly enhanced by automatic time step variation. This paper is concerned with the theory, construction and application of IMEX-Peer methods that are super-convergent for variable step sizes and A-stable in the implicit part. IMEX schemes combine the necessary stability of implicit and low computational costs of explicit methods to efficiently solve systems of ordinary differential equations with both stiff and non-stiff parts included in the source term. To construct super-convergent IMEX-Peer methods which keep their higher order for variable step sizes and exhibit favourable linear stability properties, we derive necessary and sufficient conditions on the nodes and coefficient matrices and apply an extrapolation approach based on already computed stage values. New super-convergent IMEX-Peer methods of order $s+1$ for $s=2,3,4$ stages are given as result of additional order conditions which maintain the super-convergence property independent of step size changes. Numerical experiments and a comparison to other super-convergent IMEX-Peer methods show the potential of the new methods when applied with local error control.

Sascha Marton, Moritz Schneider

Neural architectures such as Recurrent Neural Networks (RNNs), Transformers, and State-Space Models have shown great success in handling sequential data by learning temporal dependencies. Decision Trees (DTs), on the other hand, remain a widely used class of models for structured tabular data but are typically not designed to capture sequential patterns directly. Instead, DT-based approaches for time-series data often rely on feature engineering, such as manually incorporating lag features, which can be suboptimal for capturing complex temporal dependencies. To address this limitation, we introduce ReMeDe Trees, a novel recurrent DT architecture that integrates an internal memory mechanism, similar to RNNs, to learn long-term dependencies in sequential data. Our model learns hard, axis-aligned decision rules for both output generation and state updates, optimizing them efficiently via gradient descent. We provide a proof-of-concept study on synthetic benchmarks to demonstrate the effectiveness of our approach.

Richard Meyes, Moritz Schneider, Tobias Meisen

The demand for more transparency of decision-making processes of deep reinforcement learning agents is greater than ever, due to their increased use in safety critical and ethically challenging domains such as autonomous driving. In this empirical study, we address this lack of transparency following an idea that is inspired by research in the field of neuroscience. We characterize the learned representations of an agent's policy network through its activation space and perform partial network ablations to compare the representations of the healthy and the intentionally damaged networks. We show that the healthy agent's behavior is characterized by a distinct correlation pattern between the network's layer activation and the performed actions during an episode and that network ablations, which cause a strong change of this pattern, lead to the agent failing its trained control task. Furthermore, the learned representation of the healthy agent is characterized by a distinct pattern in its activation space reflecting its different behavioral stages during an episode, which again, when distorted by network ablations, leads to the agent failing its trained control task. Concludingly, we argue in favor of a new perspective on artificial neural networks as objects of empirical investigations, just as biological neural systems in neuroscientific studies, paving the way towards a new standard of scientific falsifiability with respect to research on transparency and interpretability of artificial neural networks.