Marco Coraggio

Marco Coraggio, Pietro DeLellis, Mario di Bernardo

Complex networks are a successful framework to describe collective behaviour in many applications, but a notable gap remains in the current literature, that of proving asymptotic convergence in networks of piecewise-smooth systems. Indeed, a wide variety of physical systems display discontinuous dynamics that change abruptly, including dry friction mechanical oscillators, electrical power converters, and biological neurons. In this paper, we study how to enforce global asymptotic state-synchronization in these networks. Specifically, we propose the addition of a distributed discontinuous coupling action to the commonly used diffusive coupling protocol. Moreover, we provide analytical estimates of the thresholds on the coupling gains required for convergence, and highlight the importance of a new connectivity measure, which we named minimum density. The theoretical results are illustrated by a set of representative examples.

Francesco De Lellis, Marco Coraggio, Giovanni Russo et al.

One of the major challenges in Deep Reinforcement Learning for control is the need for extensive training to learn the policy. Motivated by this, we present the design of the Control-Tutored Deep Q-Networks (CT-DQN) algorithm, a Deep Reinforcement Learning algorithm that leverages a control tutor, i.e., an exogenous control law, to reduce learning time. The tutor can be designed using an approximate model of the system, without any assumption about the knowledge of the system's dynamics. There is no expectation that it will be able to achieve the control objective if used stand-alone. During learning, the tutor occasionally suggests an action, thus partially guiding exploration. We validate our approach on three scenarios from OpenAI Gym: the inverted pendulum, lunar lander, and car racing. We demonstrate that CT-DQN is able to achieve better or equivalent data efficiency with respect to the classic function approximation solutions.

Francesco De Lellis, Marco Coraggio, Giovanni Russo et al.

In addressing control problems such as regulation and tracking through reinforcement learning, it is often required to guarantee that the acquired policy meets essential performance and stability criteria such as a desired settling time and steady-state error prior to deployment. Motivated by this necessity, we present a set of results and a systematic reward shaping procedure that (i) ensures the optimal policy generates trajectories that align with specified control requirements and (ii) allows to assess whether any given policy satisfies them. We validate our approach through comprehensive numerical experiments conducted in two representative environments from OpenAI Gym: the Inverted Pendulum swing-up problem and the Lunar Lander. Utilizing both tabular and deep reinforcement learning methods, our experiments consistently affirm the efficacy of our proposed framework, highlighting its effectiveness in ensuring policy adherence to the prescribed control requirements.

Francesco De Lellis, Marco Coraggio, Nathan C. Foster et al.

We present a data-driven control architecture for modifying the kinematics of robots and artificial avatars to encode specific information such as the presence or not of an emotion in the movements of an avatar or robot driven by a human operator. We validate our approach on an experimental dataset obtained during the reach-to-grasp phase of a pick-and-place task.

Angelo Di Porzio, Marco Coraggio

The deployment of autonomous virtual avatars (in extended reality) and robots in human group activities -- such as rehabilitation therapy, sports, and manufacturing -- is expected to increase as these technologies become more pervasive. Designing cognitive architectures and control strategies to drive these agents requires realistic models of human motion. Furthermore, recent research has shown that each person exhibits a unique velocity signature, highlighting how individual motor behaviors are both rich in variability and internally consistent. However, existing models only provide simplified descriptions of human motor behavior, hindering the development of effective cognitive architectures. In this work, we first show that motion amplitude provides a valid and complementary characterization of individual motor signatures. Then, we propose a fully data-driven approach, based on long short-term memory neural networks, to generate original motion that captures the unique features of specific individuals. We validate the architecture using real human data from participants performing spontaneous oscillatory motion. Extensive analyses show that state-of-the-art Kuramoto-like models fail to replicate individual motor signatures, whereas our model accurately reproduces the velocity distribution and amplitude envelopes of the individual it was trained on, while remaining distinct from others.

F. De Lellis, M. Coraggio, G. Russo et al.

We present an architecture where a feedback controller derived on an approximate model of the environment assists the learning process to enhance its data efficiency. This architecture, which we term as Control-Tutored Q-learning (CTQL), is presented in two alternative flavours. The former is based on defining the reward function so that a Boolean condition can be used to determine when the control tutor policy is adopted, while the latter, termed as probabilistic CTQL (pCTQL), is instead based on executing calls to the tutor with a certain probability during learning. Both approaches are validated, and thoroughly benchmarked against Q-Learning, by considering the stabilization of an inverted pendulum as defined in OpenAI Gym as a representative problem.

Ricardo Cardona-Rivera, Francesco Lo Iudice, Antonio Grotta et al.

The problem of partitioning a power grid into a set of microgrids, or islands, is of interest for both the design of future smart grids, and as a last resort to restore power dispatchment in sections of a grid affected by an extreme failure. In the literature this problem is usually solved by turning it into a combinatorial optimization problem, often solved through generic heruristic methods such as Genetic Algorithms or Tabu Search. In this paper, we take a different route and obtain the grid partition by exploiting the synchronization dynamics of a cyberlayer of Kuramoto oscillators, each parameterized as a rough approximation of the dynamics of the grid's node it corresponds to. We present first a centralised algorithm and then a decentralised strategy. In the former, nodes are aggregated based on their internode synchronization times while in the latter they exploit synchronization of the oscillators in the cyber layer to selforganise into islands. Our preliminary results show that the heuristic synchronization based algorithms do converge towards partitions that are comparable to those obtained via other more cumbersome and computationally expensive optimization-based methods.

Davide Marchese, Marco Coraggio, S. John Hogan et al.

The Painlevé paradox is a phenomenon that causes instability in mechanical systems subjects to unilateral constraints. While earlier studies were mostly focused on abstract theoretical settings, recent work confirmed the occurrence of the paradox in realistic set-ups. In this paper, we investigate the dynamics and presence of the Painlevé phenomenon in a twolinks robot in contact with a moving belt, through a bifurcation study. Then, we use the results of this analysis to inform the design of control strategies able to keep the robot sliding on the belt and avoid the onset of undesired lift-off. To this aim, through numerical simulations, we synthesise and compare a PID strategy and a hybrid force/motion control scheme, finding that the latter is able to guarantee better performance and avoid the onset of bouncing motion due to the Painlevé phenomenon.

Marco Coraggio, Martin Homer, Oliver D. Payton et al.

Atomic force microscopes have proved to be fundamental research tools in many situations where a gentle imaging process is required, and in a variety of environmental conditions, such as the study of biological samples. Among the possible modes of operation, intermittent contact mode is one that causes less wear to both the sample and the instrument; therefore, it is ideal when imaging soft samples. However, intermittent contact mode is not particularly fast when compared to other imaging strategies. In this paper, we introduce three enhanced control approaches, applied at both the dither and z-axis piezos, to address the limitations of existing control schemes. Our proposed strategies are able to eliminate different image artefacts, automatically adapt scan speed to the sample being scanned and predict its features in real time. The result is that both the image quality and the scan time are improved.