José M. Amigó

José M. Amigó, Roberto Dale

Algebraic representations of time series are symbolic representations whose symbols belong to a finite group. Precisely, the framework of the present paper is the analysis of coupled time series in algebraic representations and, more generally, group-valued time series. The prototype of an algebraic representation is an ordinal representation, whose symbols are permutations, also called ordinal patterns in the context of time series analysis. In fact, permutations, endowed with function composition, build a group called a symmetric group. A simple way to harness the algebraic structure of the alphabet in such cases is the concept of transcript from one group element to another. Since transcripts involve two group elements, they are very suitable for studying couplings between time series in the same algebraic representation. In this paper, we outline several existing entropic and algebraic transcript-based tools for analyzing coupled time series and systems. In addition to entropy, the entropic tools include divergence, statistical complexity and mutual information. The algebraic tools comprise order classes and, most recently, the Cayley and Kendall distances. We use the detection of generalized synchronization in a well-studied coupled system to compare the performances of some of those tools. To this end, we also provide an alternative tool called the similarity distance between times series, which is a mean Kendall distance. We found that the novel similarity distance outperforms the other tools tested.

Adrián Hernández, Gilles Millerioux, José M. Amigó

In the past years, deep learning models have been successfully applied in several cognitive tasks. Originally inspired by neuroscience, these models are specific examples of differentiable programs. In this paper we define and motivate differentiable programming, as well as specify some program characteristics that allow us to incorporate the structure of the problem in a differentiable program. We analyze different types of differentiable programs, from more general to more specific, and evaluate, for a specific problem with a graph dataset, its structure and knowledge with several differentiable programs using those characteristics. Finally, we discuss some inherent limitations of deep learning and differentiable programs, which are key challenges in advancing artificial intelligence, and then analyze possible solutions

Paweł Pilarczyk, Grzegorz Graff, José M. Amigó et al.

We introduce an entropy-based classification method for pairs of sequences (ECPS) for quantifying mutual dependencies in heart rate and beat-to-beat blood pressure recordings. The purpose of the method is to build a classifier for data in which each item consists of the two intertwined data series taken for each subject. The method is based on ordinal patterns, and uses entropy-like indices. Machine learning is used to select a subset of indices most suitable for our classification problem in order to build an optimal yet simple model for distinguishing between patients suffering from obstructive sleep apnea and a control group.

Salomé A. Sepúveda Fontaine, José M. Amigó

Since its origin in the thermodynamics of the 19th century, the concept of entropy has also permeated other fields of physics and mathematics, such as Classical and Quantum Statistical Mechanics, Information Theory, Probability Theory, Ergodic Theory and the Theory of Dynamical Systems. Specifically, we are referring to the classical entropies: the Boltzmann-Gibbs, von Neumann, Shannon, Kolmogorov-Sinai and topological entropies. In addition to their common name, which is historically justified (as we briefly describe in this review), other commonality of the classical entropies is the important role that they have played and are still playing in the theory and applications of their respective fields and beyond. Therefore, it is not surprising that, in the course of time, many other instances of the overarching concept of entropy have been proposed, most of them tailored to specific purposes. Following the current usage, we will refer to all of them, whether classical or new, simply as entropies. Precisely, the subject of this review is their applications in data analysis and machine learning. The reason for these particular applications is that entropies are very well suited to characterize probability mass distributions, typically generated by finite-state processes or symbolized signals. Therefore, we will focus on entropies defined as positive functionals on probability mass distributions and provide an axiomatic characterization that goes back to Shannon and Khinchin. Given the plethora of entropies in the literature, we have selected a representative group, including the classical ones. The applications summarized in this review finely illustrate the power and versatility of entropy in data analysis and machine learning.

Adrián Hernández, José M. Amigó

Differentiable programming is the combination of classical neural networks modules with algorithmic ones in an end-to-end differentiable model. These new models, that use automatic differentiation to calculate gradients, have new learning capabilities (reasoning, attention and memory). In this tutorial, aimed at researchers in nonlinear systems with prior knowledge of deep learning, we present this new programming paradigm, describe some of its new features such as attention mechanisms, and highlight the benefits they bring. Then, we analyse the uses and limitations of traditional deep learning models in the modeling and prediction of dynamical systems. Here, a dynamical system is meant to be a set of state variables that evolve in time under general internal and external interactions. Finally, we review the advantages and applications of differentiable programming to dynamical systems.