FSPool: Learning Set Representations with Featurewise Sort Pooling
This addresses a specific issue in set representation learning for machine learning researchers, offering an incremental improvement over traditional methods.
The paper tackles the responsibility problem in set prediction models by introducing FSPool, a featurewise sort pooling method, which improves reconstruction and representation quality on toy polygon and set MNIST datasets, and enhances accuracy and convergence speed when integrated into existing set encoders.
Traditional set prediction models can struggle with simple datasets due to an issue we call the responsibility problem. We introduce a pooling method for sets of feature vectors based on sorting features across elements of the set. This can be used to construct a permutation-equivariant auto-encoder that avoids this responsibility problem. On a toy dataset of polygons and a set version of MNIST, we show that such an auto-encoder produces considerably better reconstructions and representations. Replacing the pooling function in existing set encoders with FSPool improves accuracy and convergence speed on a variety of datasets.