Mario di Bernardo

Daniel Alberto Burbano Lombana, Mario di Bernardo

In this paper, we propose a multiplex proportional-integral approach, for solving consensus problems in networks of heterogeneous nodes dynamics affected by constant disturbances. The proportional and integral actions are deployed on two different layers across the network, each with its own topology. Sufficient conditions for convergence are derived that depend upon the structure of the network, the parameters characterizing the control layers and the node dynamics. The effectiveness of the theoretical results is illustrated using a power network model as a representative example.

Marco Coraggio, Pietro DeLellis, Mario di Bernardo

Complex networks are a successful framework to describe collective behaviour in many applications, but a notable gap remains in the current literature, that of proving asymptotic convergence in networks of piecewise-smooth systems. Indeed, a wide variety of physical systems display discontinuous dynamics that change abruptly, including dry friction mechanical oscillators, electrical power converters, and biological neurons. In this paper, we study how to enforce global asymptotic state-synchronization in these networks. Specifically, we propose the addition of a distributed discontinuous coupling action to the commonly used diffusive coupling protocol. Moreover, we provide analytical estimates of the thresholds on the coupling gains required for convergence, and highlight the importance of a new connectivity measure, which we named minimum density. The theoretical results are illustrated by a set of representative examples.

Maria Lombardi, Davide Liuzza, Mario di Bernardo

Designing artificial cyber-agents able to interact with human safely, smartly and in a natural way is a current open problem in control. Solving such an issue will allow the design of cyber-agents capable of co-operatively interacting with people in order to fulfil common joint tasks in a multitude of different applications. This is particularly relevant in the context of healthcare applications. Indeed, the use has been proposed of artificial agents interacting and coordinating their movements with those of a patient suffering from social or motor disorders. Specifically, it has been shown that an artificial agent exhibiting certain kinematic properties could provide innovative and efficient rehabilitation strategies for these patients. Moreover, it has also been shown that the level of motor coordination is enhanced if these kinematic properties are similar to those of the individual it is interacting with. In this paper we discuss, first, a new method based on Markov Chains to confer "human motor characteristics" on a virtual agent, so as that it can coordinate its motion with that of a target individual while exhibiting specific kinematic properties. Then, we embed such synthetic model in a control architecture based on reinforcement learning to synthesize a cyber-agent able to mimic the behaviour of a specific human performing a joint motor task with one or more individuals.

Francesco De Lellis, Marco Coraggio, Giovanni Russo et al.

One of the major challenges in Deep Reinforcement Learning for control is the need for extensive training to learn the policy. Motivated by this, we present the design of the Control-Tutored Deep Q-Networks (CT-DQN) algorithm, a Deep Reinforcement Learning algorithm that leverages a control tutor, i.e., an exogenous control law, to reduce learning time. The tutor can be designed using an approximate model of the system, without any assumption about the knowledge of the system's dynamics. There is no expectation that it will be able to achieve the control objective if used stand-alone. During learning, the tutor occasionally suggests an action, thus partially guiding exploration. We validate our approach on three scenarios from OpenAI Gym: the inverted pendulum, lunar lander, and car racing. We demonstrate that CT-DQN is able to achieve better or equivalent data efficiency with respect to the classic function approximation solutions.

Sara Maria Brancato, Francesco De Lellis, Davide Salzano et al.

We investigate the problem of using a learning-based strategy to stabilize a synthetic toggle switch via an external control approach. To overcome the data efficiency problem that would render the algorithm unfeasible for practical use in synthetic biology, we adopt a sim-to-real paradigm where the policy is learnt via training on a simplified model of the toggle switch and it is then subsequently exploited to control a more realistic model of the switch parameterized from in-vivo experiments. Our in-silico experiments confirm the viability of the approach suggesting its potential use for in-vivo control implementations.

Luigi Catello, Italo Napolitano, Davide Salzano et al.

We propose a Reinforcement Learning framework for sparse indirect control of large-scale multi-agent systems, where few controlled agents shape the collective behavior of many uncontrolled agents. The approach addresses this multi-scale challenge by coupling ODEs (modeling controlled agents) with a PDE (describing the uncontrolled population density), capturing how microscopic control achieves macroscopic objectives. Our method combines model-free Reinforcement Learning with adaptive interaction strength compensation to overcome sparse actuation limitations. Numerical validation demonstrates effective density control, with the system achieving target distributions while maintaining robustness to disturbances and measurement noise, confirming that learning-based sparse control can replace computationally expensive online optimization.

Fabrizia Auletta, Rachel W. Kallen, Mario di Bernardo et al.

This study uses supervised machine learning (SML) and explainable artificial intelligence (AI) to model, predict and understand human decision-making during skillful joint-action. Long short-term memory networks were trained to predict the target selection decisions of expert and novice actors completing a dyadic herding task. Results revealed that the trained models were expertise specific and could not only accurately predict the target selection decisions of expert and novice herders but could do so at timescales that preceded an actor's conscious intent. To understand what differentiated the target selection decisions of expert and novice actors, we then employed the explainable-AI technique, SHapley Additive exPlanation, to identify the importance of informational features (variables) on model predictions. This analysis revealed that experts were more influenced by information about the state of their co-herders compared to novices. The utility of employing SML and explainable-AI techniques for investigating human decision-making is discussed.

Francesco De Lellis, Marco Coraggio, Giovanni Russo et al.

In addressing control problems such as regulation and tracking through reinforcement learning, it is often required to guarantee that the acquired policy meets essential performance and stability criteria such as a desired settling time and steady-state error prior to deployment. Motivated by this necessity, we present a set of results and a systematic reward shaping procedure that (i) ensures the optimal policy generates trajectories that align with specified control requirements and (ii) allows to assess whether any given policy satisfies them. We validate our approach through comprehensive numerical experiments conducted in two representative environments from OpenAI Gym: the Inverted Pendulum swing-up problem and the Lunar Lander. Utilizing both tabular and deep reinforcement learning methods, our experiments consistently affirm the efficacy of our proposed framework, highlighting its effectiveness in ensuring policy adherence to the prescribed control requirements.

Lorenzo Giordano, Josep M. Olm, Mario di Bernardo

Synchronization in networks of coupled oscillators is classically studied via the Kuramoto model, whose intrinsic nonlinearity limits analytical tractability and complicates control design. Complex-valued extensions circumvent this by embedding phase dynamics into a higher-dimensional linear state space, where regulating complex-state moduli to a common value recovers Kuramoto phase behavior. Existing approaches to address this problem correspond, within a unified control framework, to state-feedback and hybrid reset-based strategies, each with performance constraints. We propose two switched control designs that overcome these limitations: a switched feedforward law ensuring exact phase correspondence at all times, and a feedforward plus sliding-mode law achieving finite-time convergence without spectral gain tuning. Additionally, we present a non-autonomous complex-valued MIMO sliding-mode controller that enforces phase locking at a prescribed frequency in finite time, independent of natural frequencies and coupling strengths. Simulations confirm improved transient response, steady-state accuracy, and robustness, including synchronization of heterogeneous networks where the classical real-valued Kuramoto model fails.

Dimitrios G. Patsatzis, Mario di Bernardo, Lucia Russo et al.

We present GoRINNs: numerical analysis-informed (shallow) neural networks for the solution of inverse problems of non-linear systems of conservation laws. GoRINNs is a hybrid/blended machine learning scheme based on high-resolution Godunov schemes for the solution of the Riemann problem in hyperbolic Partial Differential Equations (PDEs). In contrast to other existing machine learning methods that learn the numerical fluxes or just parameters of conservative Finite Volume methods, relying on deep neural networks (that may lead to poor approximations due to the computational complexity involved in their training), GoRINNs learn the closures of the conservation laws per se based on "intelligently" numerical-assisted shallow neural networks. Due to their structure, in particular, GoRINNs provide explainable, conservative schemes, that solve the inverse problem for hyperbolic PDEs, on the basis of approximate Riemann solvers that satisfy the Rankine-Hugoniot condition. The performance of GoRINNs is assessed via four benchmark problems, namely the Burgers', the Shallow Water, the Lighthill-Whitham-Richards and the Payne-Whitham traffic flow models. The solution profiles of these PDEs exhibit shock waves, rarefactions and/or contact discontinuities at finite times. We demonstrate that GoRINNs provide a very high accuracy both in the smooth and discontinuous regions.

Francesco De Lellis, Marco Coraggio, Nathan C. Foster et al.

We present a data-driven control architecture for modifying the kinematics of robots and artificial avatars to encode specific information such as the presence or not of an emotion in the movements of an avatar or robot driven by a human operator. We validate our approach on an experimental dataset obtained during the reach-to-grasp phase of a pick-and-place task.

Gian Carlo Maffettone, Alain Boldini, Mario di Bernardo et al.

The design of control systems for the spatial self-organization of mobile agents is an open challenge across several engineering domains, including swarm robotics and synthetic biology. Here, we propose a bio-inspired leader-follower solution, which is aware of energy constraints of mobile agents and is apt to deal with large swarms. Akin to many natural systems, control objectives are formulated for the entire collective, and leaders and followers are allowed to plastically switch their role in time. We frame a density control problem, modeling the agents' population via a system of nonlinear partial differential equations. This approach allows for a compact description that inherently avoids the curse of dimensionality and improves analytical tractability. We derive analytical guarantees for the existence of desired steady-state solutions and their local stability for one-dimensional and higher-dimensional problems. We numerically validate our control methodology, offering support to the effectiveness, robustness, and versatility of our proposed bio-inspired control strategy.

Stefano Covone, Italo Napolitano, Francesco De Lellis et al.

We propose a decentralized reinforcement learning solution for multi-agent shepherding of non-cohesive targets using policy-gradient methods. Our architecture integrates target-selection with target-driving through Proximal Policy Optimization, overcoming discrete-action constraints of previous Deep Q-Network approaches and enabling smoother agent trajectories. This model-free framework effectively solves the shepherding problem without prior dynamics knowledge. Experiments demonstrate our method's effectiveness and scalability with increased target numbers and limited sensing capabilities.

Sara Maria Brancato, Davide Salzano, Francesco De Lellis et al.

A key problem toward the use of microorganisms as bio-factories is reaching and maintaining cellular communities at a desired density and composition so that they can efficiently convert their biomass into useful compounds. Promising technological platforms for the real time, scalable control of cellular density are bioreactors. In this work, we developed a learning-based strategy to expand the toolbox of available control algorithms capable of regulating the density of a \textit{single} bacterial population in bioreactors. Specifically, we used a sim-to-real paradigm, where a simple mathematical model, calibrated using a few data, was adopted to generate synthetic data for the training of the controller. The resulting policy was then exhaustively tested in vivo using a low-cost bioreactor known as Chi.Bio, assessing performance and robustness. In addition, we compared the performance with more traditional controllers (namely, a PI and an MPC), confirming that the learning-based controller exhibits similar performance in vivo. Our work showcases the viability of learning-based strategies for the control of cellular density in bioreactors, making a step forward toward their use for the control of the composition of microbial consortia.

F. De Lellis, M. Coraggio, G. Russo et al.

We present an architecture where a feedback controller derived on an approximate model of the environment assists the learning process to enhance its data efficiency. This architecture, which we term as Control-Tutored Q-learning (CTQL), is presented in two alternative flavours. The former is based on defining the reward function so that a Boolean condition can be used to determine when the control tutor policy is adopted, while the latter, termed as probabilistic CTQL (pCTQL), is instead based on executing calls to the tutor with a certain probability during learning. Both approaches are validated, and thoroughly benchmarked against Q-Learning, by considering the stabilization of an inverted pendulum as defined in OpenAI Gym as a representative problem.

Ricardo Cardona-Rivera, Francesco Lo Iudice, Antonio Grotta et al.

The problem of partitioning a power grid into a set of microgrids, or islands, is of interest for both the design of future smart grids, and as a last resort to restore power dispatchment in sections of a grid affected by an extreme failure. In the literature this problem is usually solved by turning it into a combinatorial optimization problem, often solved through generic heruristic methods such as Genetic Algorithms or Tabu Search. In this paper, we take a different route and obtain the grid partition by exploiting the synchronization dynamics of a cyberlayer of Kuramoto oscillators, each parameterized as a rough approximation of the dynamics of the grid's node it corresponds to. We present first a centralised algorithm and then a decentralised strategy. In the former, nodes are aggregated based on their internode synchronization times while in the latter they exploit synchronization of the oscillators in the cyber layer to selforganise into islands. Our preliminary results show that the heuristic synchronization based algorithms do converge towards partitions that are comparable to those obtained via other more cumbersome and computationally expensive optimization-based methods.

Francesco De Lellis, Giovanni Russo, Mario di Bernardo

We introduce a control-tutored reinforcement learning (CTRL) algorithm. The idea is to enhance tabular learning algorithms by means of a control strategy with limited knowledge of the system model. By tutoring the learning process, the learning rate can be substantially reduced. We use the classical problem of stabilizing an inverted pendulum as a benchmark to numerically illustrate the advantages and disadvantages of the approach.

Francesco De Lellis, Fabrizia Auletta, Giovanni Russo et al.

We introduce a control-tutored reinforcement learning (CTRL) algorithm. The idea is to enhance tabular learning algorithms so as to improve the exploration of the state-space, and substantially reduce learning times by leveraging some limited knowledge of the plant encoded into a tutoring model-based control strategy. We illustrate the benefits of our novel approach and its effectiveness by using the problem of controlling one or more agents to herd and contain within a goal region a set of target free-roving agents in the plane.

Francesco De Lellis, Fabrizia Auletta, Giovanni Russo et al.

In this extended abstract we introduce a novel control-tutored Q-learning approach (CTQL) as part of the ongoing effort in developing model-based and safe RL for continuous state spaces. We validate our approach by applying it to a challenging multi-agent herding control problem.

Davide Marchese, Marco Coraggio, S. John Hogan et al.

The Painlevé paradox is a phenomenon that causes instability in mechanical systems subjects to unilateral constraints. While earlier studies were mostly focused on abstract theoretical settings, recent work confirmed the occurrence of the paradox in realistic set-ups. In this paper, we investigate the dynamics and presence of the Painlevé phenomenon in a twolinks robot in contact with a moving belt, through a bifurcation study. Then, we use the results of this analysis to inform the design of control strategies able to keep the robot sliding on the belt and avoid the onset of undesired lift-off. To this aim, through numerical simulations, we synthesise and compare a PID strategy and a hybrid force/motion control scheme, finding that the latter is able to guarantee better performance and avoid the onset of bouncing motion due to the Painlevé phenomenon.

Maria Lombardi, Davide Liuzza, Mario di Bernardo

In many joint-action scenarios, humans and robots have to coordinate their movements to accomplish a given shared task. Lifting an object together, sawing a wood log, transferring objects from a point to another are all examples where motor coordination between humans and machines is a crucial requirement. While the dyadic coordination between a human and a robot has been studied in previous investigations, the multi-agent scenario in which a robot has to be integrated into a human group still remains a less explored field of research. In this paper we discuss how to synthesise an artificial agent able to coordinate its motion in human ensembles. Driven by a control architecture based on deep reinforcement learning, such an artificial agent will be able to autonomously move itself in order to synchronise its motion with that of the group while exhibiting human-like kinematic features. As a paradigmatic coordination task we take a group version of the so-called mirror-game which is highlighted as a good benchmark in the human movement literature.

Francesco Alderisio, Maria Lombardi, Gianfranco Fiore et al.

Movement coordination in human ensembles has been studied little in the current literature. In the existing experimental works, situations where all subjects are connected with each other through direct visual and auditory coupling, and social interaction affects their coordination, have been investigated. Here, we study coordination in human ensembles via a novel computer-based set-up that enables individuals to coordinate each other's motion from a distance so as to minimize the influence of social interaction. The proposed platform makes it possible to implement different visual interaction patterns among the players, so that participants take into consideration the motion of a designated subset of the others. This allows the evaluation of the exclusive effects on coordination of the structure of interconnections among the players and their own dynamics. Our set-up enables also the deployment of virtual players to investigate dyadic interaction between a human and a virtual agent, as well as group synchronization in mixed teams of human and virtual agents. We use this novel set-up to study coordination both in dyads and in groups over different structures of interconnections, with and without virtual agents. We find that, in dual interaction, virtual players manage to interact with participants in a human-like fashion, thus confirming findings in previous work. We also observe that, in group interaction, the level of coordination among humans in the absence of direct visual and auditory coupling depends on the structure of interconnections among participants. This confirms, as recently suggested in the literature, that different coordination levels are achieved over diverse visual pairings in the presence and in the absence of social interaction. We present preliminary experimental results on the effect on group coordination of deploying virtual computer agents in the human ensemble.

Marco Coraggio, Martin Homer, Oliver D. Payton et al.

Atomic force microscopes have proved to be fundamental research tools in many situations where a gentle imaging process is required, and in a variety of environmental conditions, such as the study of biological samples. Among the possible modes of operation, intermittent contact mode is one that causes less wear to both the sample and the instrument; therefore, it is ideal when imaging soft samples. However, intermittent contact mode is not particularly fast when compared to other imaging strategies. In this paper, we introduce three enhanced control approaches, applied at both the dither and z-axis piezos, to address the limitations of existing control schemes. Our proposed strategies are able to eliminate different image artefacts, automatically adapt scan speed to the sample being scanned and predict its features in real time. The result is that both the image quality and the scan time are improved.

Louis Kempton, Guido Herrmann, Mario di Bernardo

We show that a network can self-organize its structure in a completely distributed manner in order to optimize its synchronizability whilst satisfying the local constraints: non-negativity of edge weights, and maximum weighted degree of nodes. A novel multilayer approach is presented which uses a distributed strategy to estimate two spectral functions of the graph Laplacian, the algebraic connectivity $λ_2$ and the eigenratio $r = λ_n / λ_2$ . These local estimates are then used to evolve the edge weights so as to maximize $λ_2$, or minimize $r$ and, hence, achieve an optimal structure.