Set Cross Entropy: Likelihood-based Permutation Invariant Loss Function for Probability Distributions
This work addresses the challenge of set reconstruction in neural networks, offering a method that avoids complex architectures or sequential algorithms, though it appears incremental as it builds on existing set-based approaches.
The authors tackled the problem of reconstructing sets of elements without order constraints by proposing Set Cross Entropy, a permutation-invariant loss function, and demonstrated its effectiveness in object reconstruction and rule learning tasks.
We propose a permutation-invariant loss function designed for the neural networks reconstructing a set of elements without considering the order within its vector representation. Unlike popular approaches for encoding and decoding a set, our work does not rely on a carefully engineered network topology nor by any additional sequential algorithm. The proposed method, Set Cross Entropy, has a natural information-theoretic interpretation and is related to the metrics defined for sets. We evaluate the proposed approach in two object reconstruction tasks and a rule learning task.