"Does it come in black?" CLIP-like models are zero-shot recommenders
This addresses a specific need in e-commerce for more flexible product discovery, though it is incremental as it adapts existing CLIP models to a new application.
The paper tackles the problem of enabling users to explore item recommendations along specific dimensions, such as color, in online shopping, and demonstrates that CLIP-based models can perform this comparative recommendation task in a zero-shot manner with GradREC, achieving competitive performance on fashion datasets.
Product discovery is a crucial component for online shopping. However, item-to-item recommendations today do not allow users to explore changes along selected dimensions: given a query item, can a model suggest something similar but in a different color? We consider item recommendations of the comparative nature (e.g. "something darker") and show how CLIP-based models can support this use case in a zero-shot manner. Leveraging a large model built for fashion, we introduce GradREC and its industry potential, and offer a first rounded assessment of its strength and weaknesses.