New York Smells: A Large Multimodal Dataset for Olfaction
This addresses the problem of limited olfactory data for AI researchers, providing a large-scale multimodal dataset to advance machine olfaction, though it is incremental as it builds on existing data collection efforts.
The authors tackled the lack of diverse multimodal olfactory training data by introducing New York Smells, a dataset with 7,000 smell-image pairs from 3,500 distinct objects, which is 70 times larger than existing datasets, and they found that visual data aids olfactory representation learning, outperforming hand-crafted features.
While olfaction is central to how animals perceive the world, this rich chemical sensory modality remains largely inaccessible to machines. One key bottleneck is the lack of diverse, multimodal olfactory training data collected in natural settings. We present New York Smells, a large dataset of paired image and olfactory signals captured ``in the wild.'' Our dataset contains 7,000 smell-image pairs from 3,500 distinct objects across indoor and outdoor environments, with approximately 70$\times$ more objects than existing olfactory datasets. Our benchmark has three tasks: cross-modal smell-to-image retrieval, recognizing scenes, objects, and materials from smell alone, and fine-grained discrimination between grass species. Through experiments on our dataset, we find that visual data enables cross-modal olfactory representation learning, and that our learned olfactory representations outperform widely-used hand-crafted features.