Cycled Compositional Learning between Images and Text
This addresses the challenge of image-text composition for fashion applications, representing an incremental improvement with a novel method for a known bottleneck.
The paper tackles the problem of measuring semantic distance between composed image-text embeddings by proposing a Cycled Composition Network, which achieved first place in the Fashion IQ 2020 challenge.
We present an approach named the Cycled Composition Network that can measure the semantic distance of the composition of image-text embedding. First, the Composition Network transit a reference image to target image in an embedding space using relative caption. Second, the Correction Network calculates a difference between reference and retrieved target images in the embedding space and match it with a relative caption. Our goal is to learn a Composition mapping with the Composition Network. Since this one-way mapping is highly under-constrained, we couple it with an inverse relation learning with the Correction Network and introduce a cycled relation for given Image We participate in Fashion IQ 2020 challenge and have won the first place with the ensemble of our model.