Representation Learning with Multisets
This addresses representation learning for multisets, which is important for applications involving sets or collections, but appears incremental relative to existing set-based methods.
The paper tackles the problem of learning permutation invariant representations that capture flexible containment notions for multisets, formalizing it with measure theory and training on predicting symmetric difference/intersection sizes. The model outperforms DeepSets-based approaches on containment prediction and symmetric difference/intersection size prediction while learning meaningful representations.
We study the problem of learning permutation invariant representations that can capture "flexible" notions of containment. We formalize this problem via a measure theoretic definition of multisets, and obtain a theoretically-motivated learning model. We propose training this model on a novel task: predicting the size of the symmetric difference (or intersection) between pairs of multisets. We demonstrate that our model not only performs very well on predicting containment relations (and more effectively predicts the sizes of symmetric differences and intersections than DeepSets-based approaches with unconstrained object representations), but that it also learns meaningful representations.