Vector Embeddings with Subvector Permutation Invariance using a Triplet Enhanced Autoencoder
This work addresses the problem of creating permutation-invariant vector embeddings for applications requiring robust representations, potentially benefiting fields where data order within subvectors is not meaningful.
This paper tackles the problem of creating vector embeddings that are invariant to subvector permutations. By enhancing an autoencoder with triplet loss, the authors achieve embeddings that are nearly invariant to such permutations, which can then be used to improve detection accuracy in downstream tasks like classification and clustering.
The use of deep neural network (DNN) autoencoders (AEs) has recently exploded due to their wide applicability. However, the embedding representation produced by a standard DNN AE that is trained to minimize only the reconstruction error does not always reveal more subtle patterns in the data. Sometimes, the autoencoder needs further direction in the form of one or more additional loss functions. In this paper, we use an autoencoder enhanced with triplet loss to promote the clustering of vectors that are related through permutations of constituent subvectors. With this approach, we can create an embedding of the vector that is nearly invariant to such permutations. We can then use these invariant embeddings as inputs to other problems, like classification and clustering, and improve detection accuracy in those problems.