Event2Vec: A Geometric Approach to Learning Composable Representations of Event Sequences
This work addresses the challenge of interpretable and efficient representation learning for event sequences, which is incremental by building on geometric insights in neural representations.
The authors tackled the problem of learning representations of discrete event sequences by introducing Event2Vec, a framework that uses geometric approaches to ensure composable embeddings, with theoretical analysis showing convergence to an additive structure and experiments validating improved performance, especially with hyperbolic geometry for hierarchical data.
The study of neural representations, both in biological and artificial systems, is increasingly revealing the importance of geometric and topological structures. Inspired by this, we introduce Event2Vec, a novel framework for learning representations of discrete event sequences. Our model leverages a simple, additive recurrent structure to learn composable, interpretable embeddings. We provide a theoretical analysis demonstrating that, under specific training objectives, our model's learned representations in a Euclidean space converge to an ideal additive structure. This ensures that the representation of a sequence is the vector sum of its constituent events, a property we term the linear additive hypothesis. To address the limitations of Euclidean geometry for hierarchical data, we also introduce a variant of our model in hyperbolic space, which is naturally suited to embedding tree-like structures with low distortion. We present experiments to validate our hypothesis and demonstrate the benefits of each geometry, highlighting the improved performance of the hyperbolic model on hierarchical event sequences.