Deep Message Passing on Sets
This work addresses the challenge of exploiting intra-collection relationships in set data, which is important for machine learning applications involving sets, but it appears incremental as it builds on existing graph and denoising methods.
The authors tackled the problem of learning over set input data by introducing Deep Message Passing on Sets (DMPS), a method that incorporates relational learning for sets, achieving competitive or state-of-the-art results on synthetic and real-world datasets.
Modern methods for learning over graph input data have shown the fruitfulness of accounting for relationships among elements in a collection. However, most methods that learn over set input data use only rudimentary approaches to exploit intra-collection relationships. In this work we introduce Deep Message Passing on Sets (DMPS), a novel method that incorporates relational learning for sets. DMPS not only connects learning on graphs with learning on sets via deep kernel learning, but it also bridges message passing on sets and traditional diffusion dynamics commonly used in denoising models. Based on these connections, we develop two new blocks for relational learning on sets: the set-denoising block and the set-residual block. The former is motivated by the connection between message passing on general graphs and diffusion-based denoising models, whereas the latter is inspired by the well-known residual network. In addition to demonstrating the interpretability of our model by learning the true underlying relational structure experimentally, we also show the effectiveness of our approach on both synthetic and real-world datasets by achieving results that are competitive with or outperform the state-of-the-art.