Adam Andersson

Adam Andersson, Kristoffer Andersson, Per Ljung

In this article we investigate the theoretical convergence properties of the fictitious-play approximation procedure applied to coupled FBSDE systems for finite-player non-zero-sum stochastic differential games. Under one set of assumptions, the convergence is shown to be geometric. Under an additional structural assumption, the geometric convergence rate further improves to a super-exponential rate in a special class of games. To the best of our knowledge, this provides the first convergence analysis of fictitious play for fully coupled FBSDEs. A numerical experiment with a linear-quadratic interbank borrowing and lending problem confirms the geometric convergence.

Kasper Bågmark, Adam Andersson, Stig Larsson et al.

A numerical scheme for approximating the nonlinear filtering density is introduced and its convergence rate is established, theoretically under a parabolic Hörmander condition, and empirically in numerical examples. In a prediction step, between the noisy and partial measurements at discrete times, the scheme approximates the Fokker--Planck equation with a deep splitting scheme, followed by an exact update through Bayes' formula. This results in a classical prediction-update filtering algorithm that operates online for new observation sequences post-training. The algorithm employs a sampling-based Feynman--Kac approach, designed to mitigate the curse of dimensionality. As a corollary we obtain the convergence rate for the approximation of the Fokker--Planck equation alone, disconnected from the filtering problem. The convergence analysis is complemented by a nonlinear $10$-dimensional numerical example demonstrating the robustness of the method.

Mika Persson, Jonas Lidman, Jacob Ljungberg et al.

This work presents a conceptual study on the application of Multi-Agent Reinforcement Learning (MARL) for decentralized control of unmanned aerial vehicles to relay a critical data package to a known position. For this purpose, a family of deterministic games is introduced, designed for scaling studies for MARL. A robust baseline policy is proposed, which is based on restricting agent motion envelopes and applying Dijkstra's algorithm. Experimental results show that two off-the-shelf MARL algorithms perform competitively with the baseline for a small number of agents, but scalability issues arise as the number of agents increase.

Kristoffer Andersson, Adam Andersson, Cornelis W. Oosterlee

In this paper, we propose a deep learning based numerical scheme for strongly coupled FBSDEs, stemming from stochastic control. It is a modification of the deep BSDE method in which the initial value to the backward equation is not a free parameter, and with a new loss function being the weighted sum of the cost of the control problem, and a variance term which coincides with the mean squared error in the terminal condition. We show by a numerical example that a direct extension of the classical deep BSDE method to FBSDEs, fails for a simple linear-quadratic control problem, and motivate why the new method works. Under regularity and boundedness assumptions on the exact controls of time continuous and time discrete control problems, we provide an error analysis for our method. We show empirically that the method converges for three different problems, one being the one that failed for a direct extension of the deep BSDE method.