"I Know It When I See It": Mood Spaces for Connecting and Expressing Visual Concepts
This addresses the challenge for users in creative or design fields who need to convey vague visual ideas, though it appears incremental as it builds on pre-trained features and affinity relationships.
The paper tackles the problem of expressing abstract visual concepts that are hard to define but recognizable, by proposing a Mood Space that factors out irrelevant features and connects images to bring relevant concepts closer, achieving a 50-100x compression of pre-trained features and enabling operations like object averaging and pose transfer in under a minute with few exemplars.
Expressing complex concepts is easy when they can be labeled or quantified, but many ideas are hard to define yet instantly recognizable. We propose a Mood Board, where users convey abstract concepts with examples that hint at the intended direction of attribute changes. We compute an underlying Mood Space that 1) factors out irrelevant features and 2) finds the connections between images, thus bringing relevant concepts closer. We invent a fibration computation to compress/decompress pre-trained features into/from a compact space, 50-100x smaller. The main innovation is learning to mimic the pairwise affinity relationship of the image tokens across exemplars. To focus on the coarse-to-fine hierarchical structures in the Mood Space, we compute the top eigenvector structure from the affinity matrix and define a loss in the eigenvector space. The resulting Mood Space is locally linear and compact, allowing image-level operations, such as object averaging, visual analogy, and pose transfer, to be performed as a simple vector operation in Mood Space. Our learning is efficient in computation without any fine-tuning, needs only a few (2-20) exemplars, and takes less than a minute to learn.