Rep the Set: Neural Networks for Learning Set Representations
This addresses a fundamental limitation in machine learning for domains where data naturally decomposes into sets, such as text and graphs, offering a novel solution for set-based representations.
The paper tackles the problem of learning from data represented as sets of vectors, which conventional ML algorithms struggle with due to varying cardinality and lack of ordering, by introducing RepSet, a neural network architecture that achieves performance better or comparable to state-of-the-art on tasks like text categorization and graph classification.
In several domains, data objects can be decomposed into sets of simpler objects. It is then natural to represent each object as the set of its components or parts. Many conventional machine learning algorithms are unable to process this kind of representations, since sets may vary in cardinality and elements lack a meaningful ordering. In this paper, we present a new neural network architecture, called RepSet, that can handle examples that are represented as sets of vectors. The proposed model computes the correspondences between an input set and some hidden sets by solving a series of network flow problems. This representation is then fed to a standard neural network architecture to produce the output. The architecture allows end-to-end gradient-based learning. We demonstrate RepSet on classification tasks, including text categorization, and graph classification, and we show that the proposed neural network achieves performance better or comparable to state-of-the-art algorithms.