Pietro Ferraro

Andrew Cullen, Pietro Ferraro, Christopher King et al.

Recently, Directed Acyclic Graph (DAG) based Distributed Ledgers have been proposed for various applications in the smart mobility domain [1]. While many application studies have been described in the literature, an open problem in the DLT community concerns the lack of mathematical models describing their behaviour, and their validation. Building on a previous work in [1], we present, in this paper, a fluid based approximation for the IOTA Foundation DAG based DLT that incorporates varying transaction delays. This extension, namely the inclusion of varying delays, is important for feedback control applications (such as transactive control [2]). Extensive simulations are presented to illustrate the efficacy of our approach.

Haozhe Tian, Homayoun Hamedmoghadam, Robert Shorten et al.

Reinforcement Learning (RL) is a powerful method for controlling dynamic systems, but its learning mechanism can lead to unpredictable actions that undermine the safety of critical systems. Here, we propose RL with Adaptive Regularization (RL-AR), an algorithm that enables safe RL exploration by combining the RL policy with a policy regularizer that hard-codes the safety constraints. RL-AR performs policy combination via a "focus module," which determines the appropriate combination depending on the state--relying more on the safe policy regularizer for less-exploited states while allowing unbiased convergence for well-exploited states. In a series of critical control applications, we demonstrate that RL-AR not only ensures safety during training but also achieves a return competitive with the standards of model-free RL that disregards safety.

Anna Soligo, Pietro Ferraro, David Boyle

Interpretability is crucial for ensuring RL systems align with human values. However, it remains challenging to achieve in complex decision making domains. Existing methods frequently attempt interpretability at the level of fundamental model units, such as neurons or decision nodes: an approach which scales poorly to large models. Here, we instead propose an approach to interpretability at the level of functional modularity. We show how encouraging sparsity and locality in network weights leads to the emergence of functional modules in RL policy networks. To detect these modules, we develop an extended Louvain algorithm which uses a novel `correlation alignment' metric to overcome the limitations of standard network analysis techniques when applied to neural network architectures. Applying these methods to 2D and 3D MiniGrid environments reveals the consistent emergence of distinct navigational modules for different axes, and we further demonstrate how these functions can be validated through direct interventions on network weights prior to inference.

Haozhe Tian, Qiyu Rao, Nina Moutonnet et al.

Matched filters are widely used to localise signal patterns due to their high efficiency and interpretability. However, their effectiveness deteriorates for low signal-to-noise ratio (SNR) signals, such as those recorded on edge devices, where prominent noise patterns can closely resemble the target within the limited length of the filter. One example is the ear-electrocardiogram (ear-ECG), where the cardiac signal is attenuated and heavily corrupted by artefacts. To address this, we propose the Sequential Matched Filter (SMF), a paradigm that replaces the conventional single matched filter with a sequence of filters designed by a Reinforcement Learning agent. By formulating filter design as a sequential decision-making process, SMF adaptively design signal-specific filter sequences that remain fully interpretable by revealing key patterns driving the decision-making. The proposed SMF framework has strong potential for reliable and interpretable clinical decision support, as demonstrated by its state-of-the-art R-peak detection and physiological state classification performance on two challenging real-world ECG datasets. The proposed formulation can also be extended to a broad range of applications that require accurate pattern localisation from noise-corrupted signals.

Haozhe Tian, Pietro Ferraro, Robert Shorten et al.

The application of message-passing Graph Neural Networks has been a breakthrough for important network science problems. However, the competitive performance often relies on using handcrafted structural features as inputs, which increases computational cost and introduces bias into the otherwise purely data-driven network representations. Here, we eliminate the need for handcrafted features by introducing an attention mechanism and utilizing message-iteration profiles, in addition to an effective algorithmic approach to generate a structurally diverse training set of small synthetic networks. Thereby, we build an expressive message-passing framework and use it to efficiently solve the NP-hard problem of Network Dismantling, virtually equivalent to vital node identification, with significant real-world applications. Trained solely on diversified synthetic networks, our proposed model -- MIND: Message Iteration Network Dismantler -- generalizes to large, unseen real networks with millions of nodes, outperforming state-of-the-art network dismantling methods. Increased efficiency and generalizability of the proposed model can be leveraged beyond dismantling in a range of complex network problems.

Lianna Zhao, Luigi Vigneri, Andrew Cullen et al.

Access control is a fundamental component of the design of distributed ledgers, influencing many aspects of their design, such as fairness, efficiency, traditional notions of network security, and adversarial attacks such as Denial-of-Service (DoS) attacks. In this work, we consider the security of a recently proposed access control protocol for Directed Acyclic Graph-based distributed ledgers. We present a number of attack scenarios and potential vulnerabilities of the protocol and introduce a number of additional features which enhance its resilience. Specifically, a blacklisting algorithm, which is based on a reputation-weighted threshold, is introduced to handle both spamming and multi-rate malicious attackers. The introduction of a solidification request component is also introduced to ensure the fairness and consistency of network in the presence of attacks. Finally, a timestamp component is also introduced to maintain the consistency of the network in the presence of multi-rate attackers. Simulations to illustrate the efficacy and robustness of the revised protocol are also described.

Meghana Rathi, Pietro Ferraro, Giovanni Russo

In this paper we propose a new approach to complement reinforcement learning (RL) with model-based control (in particular, Model Predictive Control - MPC). We introduce an algorithm, the MPC augmented RL (MPRL) that combines RL and MPC in a novel way so that they can augment each other's strengths. We demonstrate the effectiveness of the MPRL by letting it play against the Atari game Pong. For this task, the results highlight how MPRL is able to outperform both RL and MPC when these are used individually.

Roman Overko, Rodrigo H. Ordonez-Hurtado, Sergiy Zhuk et al.

We introduce a permissioned distributed ledger technology (DLT) design for crowdsourced smart mobility applications. This architecture is based on a directed acyclic graph architecture (similar to the IOTA tangle) and uses both Proof-of-Work and Proof-of-Position mechanisms to provide protection against spam attacks and malevolent actors. In addition to enabling individuals to retain ownership of their data and to monetize it, the architecture also is suitable for distributed privacy-preserving machine learning algorithms, is lightweight, and can be implemented in simple internet-of-things (IoT) devices. To demonstrate its efficacy, we apply this framework to reinforcement learning settings where a third party is interested in acquiring information from agents. In particular, one may be interested in sampling an unknown vehicular traffic flow in a city, using a DLT-type architecture and without perturbing the density, with the idea of realizing a set of virtual tokens as surrogates of real vehicles to explore geographical areas of interest. These tokens, whose authenticated position determines write access to the ledger, are thus used to emulate the probing actions of commanded (real) vehicles on a given planned route by "jumping" from a passing-by vehicle to another to complete the planned trajectory. Consequently, the environment stays unaffected (i.e., the autonomy of participating vehicles is not influenced by the algorithm), regardless of the number of emitted tokens. The design of such a DLT architecture is presented, and numerical results from large-scale simulations are provided to validate the proposed approach.

Andrew Cullen, Pietro Ferraro, Christopher King et al.

Directed Acyclic Graph (DAG) based Distributed Ledgers can be useful in a number of applications in the IoT domain. A distributed ledger should serve as an immutable and irreversible record of transactions, however, a DAG structure is a more complicated mathematical object than its blockchain counterparts, and as a result, providing guarantees of immutability and irreversibility is more involved. In this paper, we analyse a commonly discussed attack scenario known as a parasite chain attack for the IOTA Foundation DAG based ledger. We analyse the efficacy of IOTA core MCMC algorithm using a matrix model and present an extension which improves the ledger resistance to these attacks.

Pietro Ferraro, Christopher King, Robert Shorten

Directed Acylic Graphs (DAGs) are emerging as an attractive alternative to traditional blockchain architectures for distributed ledger technology (DLT). In particular DAG ledgers with stochastic attachment mechanisms potentially offer many advantages over blockchain, including scalability and faster transaction speeds. However, the random nature of the attachment mechanism coupled with the requirement of protection against double-spend transactions leaves open the possibility that not all transactions will be eventually validated. Such transactions are said to be orphaned, and will never be validated. Our principal contribution is to propose a simple modification to the attachment mechanism for the Tangle (the IOTA DAG architecture). This modification ensures that all transactions are validated in finite time, and preserves essential features of the popular Monte-Carlo selection algorithm. In order to demonstrate these results we derive a fluid approximation for the Tangle (in the limit of infinite arrival rate) and prove that this fluid model exhibits the desired behavior. We also present simulations which validate the results for finite arrival rates.

Pietro Ferraro, Christopher King, Robert Shorten

This paper describes how Distributed Ledger Technologies can be used to design a class of cyber-physical systems, as well as to enforce social contracts and to orchestrate the behaviour of agents trying to access a shared resource. The first part of the paper analyses the advantages and disadvantages of using Distributed Ledger Technologies architectures to implement certain control systems in an Internet of Things (IoT) setting, and then focuses on a specific type of DLT based on a Directed Acyclic Graph. In this setting we propose a set of delay differential equations to describe the dynamical behaviour of the Tangle, an IoT-inspired Directed Acyclic Graph designed for the cryptocurrency IOTA. The second part proposes an application of Distributed Ledger Technologies as a mechanism for dynamic deposit pricing, wherein the deposit of digital currency is used to orchestrate access to a network of shared resources. The pricing signal is used as a mechanism to enforce the desired level of compliance according to a predetermined set of rules. After presenting an illustrative example, we analyze the control system and provide sufficient conditions for the stability of the network.