SHIFT15M: Fashion-specific dataset for set-to-set matching with several distribution shifts
This provides a domain-specific resource for evaluating set-to-set matching models under distribution shifts, which is incremental as it focuses on dataset creation rather than novel methods.
The paper tackles the problem of set-to-set matching in real-world scenarios where training and test data distributions differ, by introducing the SHIFT15M dataset for fashion-specific evaluation, and benchmark experiments show performance drops in naive methods due to distribution shift.
This paper addresses the problem of set-to-set matching, which involves matching two different sets of items based on some criteria, especially in the case of high-dimensional items like images. Although neural networks have been applied to solve this problem, most machine learning-based approaches assume that the training and test data follow the same distribution, which is not always true in real-world scenarios. To address this limitation, we introduce SHIFT15M, a dataset that can be used to evaluate set-to-set matching models when the distribution of data changes between training and testing. We conduct benchmark experiments that demonstrate the performance drop of naive methods due to distribution shift. Additionally, we provide software to handle the SHIFT15M dataset in a simple manner, with the URL for the software to be made available after publication of this manuscript. We believe proposed SHIFT15M dataset provide a valuable resource for evaluating set-to-set matching models under the distribution shift.