Learning Representations of Sets through Optimized Permutations
This addresses a bottleneck in set modeling for tasks like classification and visual question answering, though it appears incremental as it builds on existing set representation methods.
The paper tackles the challenge of learning permutation-invariant set representations by introducing a Permutation-Optimisation module that learns to permute sets end-to-end, achieving state-of-the-art results on four datasets including number sorting and visual question answering.
Representations of sets are challenging to learn because operations on sets should be permutation-invariant. To this end, we propose a Permutation-Optimisation module that learns how to permute a set end-to-end. The permuted set can be further processed to learn a permutation-invariant representation of that set, avoiding a bottleneck in traditional set models. We demonstrate our model's ability to learn permutations and set representations with either explicit or implicit supervision on four datasets, on which we achieve state-of-the-art results: number sorting, image mosaics, classification from image mosaics, and visual question answering.