Data Augmentation via Latent Diffusion Models for Detecting Smell-Related Objects in Historical Artworks
This addresses a niche problem in art history and cultural heritage analysis where annotations are scarce and costly, but the approach is incremental as it applies existing diffusion models to a new domain.
The paper tackles the problem of detecting smell-related objects in historical artworks, which suffers from annotation sparsity and class imbalance, by using synthetic data generation via latent diffusion models. The result shows that incorporating synthetic data improves detection performance, with effectiveness even with small data amounts and potential for further enhancements through scaling.
Finding smell references in historic artworks is a challenging problem. Beyond artwork-specific challenges such as stylistic variations, their recognition demands exceptionally detailed annotation classes, resulting in annotation sparsity and extreme class imbalance. In this work, we explore the potential of synthetic data generation to alleviate these issues and enable accurate detection of smell-related objects. We evaluate several diffusion-based augmentation strategies and demonstrate that incorporating synthetic data into model training can improve detection performance. Our findings suggest that leveraging the large-scale pretraining of diffusion models offers a promising approach for improving detection accuracy, particularly in niche applications where annotations are scarce and costly to obtain. Furthermore, the proposed approach proves to be effective even with relatively small amounts of data, and scaling it up provides high potential for further enhancements.