Leonardo Stella

Leonardo Stella, Dario Bauso

Given a large number of homogeneous players that are distributed across three possible states, we consider the problem in which these players have to control their transition rates, while minimizing a cost. The optimal transition rates are based on the players' knowledge of their current state and of the distribution of all the other players, and this introduces mean-field terms in the running and the terminal cost. The first contribution involves a mean-field game model that brings together macroscopic and microscopic dynamics. We obtain the mean-field equilibrium associated with this model, by solving the corresponding initial-terminal value problem. We perform an asymptotic analysis to obtain a stationary equilibrium for the system. The second contribution involves the study of the microscopic dynamics of the system for a finite number of players that interact in a structured environment modeled by an interaction topology. The third contribution is the specialization of the model to describe honeybee swarms, virus propagation, and cascading failures in interconnected smart-grids. A numerical analysis is conducted which involves two types of cyber-attacks. We simulate in which ways failures propagate across the interconnected smart grids and the impact on the grids frequencies. We reframe our analysis within the context of Lyapunov's linearisation method and stability theory of nonlinear systems and Kuramoto coupled oscillators model.

Tuo Zhang, Leonardo Stella, Julian Barreiro-Gomez

Despite its groundbreaking success, multi-agent reinforcement learning (MARL) still suffers from instability and nonstationarity. Replicator dynamics, the most well-known model from evolutionary game theory (EGT), provide a theoretical framework for the convergence of the trajectories to Nash equilibria and, as a result, have been used to ensure formal guarantees for MARL algorithms in stable game settings. However, they exhibit the opposite behavior in other settings, which poses the problem of finding alternatives to ensure convergence. In contrast, innovative dynamics, such as the Brown-von Neumann-Nash (BNN) or Smith, result in periodic trajectories with the potential to approximate Nash equilibria. Yet, no MARL algorithms based on these dynamics have been proposed. In response to this challenge, we develop a novel experience replay-based MARL algorithm that incorporates revision protocols as tunable hyperparameters. We demonstrate, by appropriately adjusting the revision protocols, that the behavior of our algorithm mirrors the trajectories resulting from these dynamics. Importantly, our contribution provides a framework capable of extending the theoretical guarantees of MARL algorithms beyond replicator dynamics. Finally, we corroborate our theoretical findings with empirical results.

Claire Palmer, Ben Roullier, Muhammad Aamir et al.

The COVID19 pandemic has demonstrated a need for remote learning and virtual learning applications such as virtual reality (VR) and tablet-based solutions. Creating complex learning scenarios by developers is highly time-consuming and can take over a year. It is also costly to employ teams of system analysts, developers and 3D artists. There is a requirement to provide a simple method to enable lecturers to create their own content for their laboratory tutorials. Research has been undertaken into developing generic models to enable the semi-automatic creation of a virtual learning tools for subjects that require practical interactions with the lab resources. In addition to the system for creating digital twins, a case study describing the creation of a virtual learning application for an electrical laboratory tutorial has been presented.

Claire Palmer, Ben Roullier, Muhammad Aamir et al.

There is a need for remote learning and virtual learning applications such as virtual reality (VR) and tablet-based solutions which the current pandemic has demonstrated. Creating complex learning scenarios by developers is highly time-consuming and can take over a year. There is a need to provide a simple method to enable lecturers to create their own content for their laboratory tutorials. Research is currently being undertaken into developing generic models to enable the semi-automatic creation of a virtual learning application. A case study describing the creation of a virtual learning application for an electrical laboratory tutorial is presented.

Leonardo Stella, Dario Bauso

Given a large population of players, each player has three possible choices between option 1 or 2 or no option. The two options are equally favorable and the population has to reach consensus on one of the two options quickly and in a distributed way. The more popular an option is, the more likely it is to be chosen by uncommitted players. Uncommitted players can be attracted by those committed to any of the other two options through a cross-inhibitory signal. This model originates in the context of honeybees swarms, and we generalize it to duopolistic competition and opinion dynamics. The contributions of this work include (1) the formulation of an evolutionary game model to explain the behavioral traits of the honeybees, (2) the study of the individuals and collective behavior including equilibrium points and stability, (3) the extension of the results to the case of structured environment via complex network theory, (4) the analysis of the impact of the connectivity on consensus, and (5) the study of absolute stability for the collective system under time-varying and uncertain cross-inhibitory parameter.