SeeDS: Semantic Separable Diffusion Synthesizer for Zero-shot Food Detection
This work addresses the problem of detecting novel food objects in real-world scenarios for applications like food recommendation and dietary monitoring, representing an incremental advance in zero-shot detection methods.
The paper tackles zero-shot food detection by proposing the SeeDS framework, which uses semantic disentanglement and diffusion models to synthesize discriminative features, achieving state-of-the-art performance on food datasets like ZSFooD and UECFOOD-256, with effectiveness also shown on general datasets such as PASCAL VOC and MS COCO.
Food detection is becoming a fundamental task in food computing that supports various multimedia applications, including food recommendation and dietary monitoring. To deal with real-world scenarios, food detection needs to localize and recognize novel food objects that are not seen during training, demanding Zero-Shot Detection (ZSD). However, the complexity of semantic attributes and intra-class feature diversity poses challenges for ZSD methods in distinguishing fine-grained food classes. To tackle this, we propose the Semantic Separable Diffusion Synthesizer (SeeDS) framework for Zero-Shot Food Detection (ZSFD). SeeDS consists of two modules: a Semantic Separable Synthesizing Module (S$^3$M) and a Region Feature Denoising Diffusion Model (RFDDM). The S$^3$M learns the disentangled semantic representation for complex food attributes from ingredients and cuisines, and synthesizes discriminative food features via enhanced semantic information. The RFDDM utilizes a novel diffusion model to generate diversified region features and enhances ZSFD via fine-grained synthesized features. Extensive experiments show the state-of-the-art ZSFD performance of our proposed method on two food datasets, ZSFooD and UECFOOD-256. Moreover, SeeDS also maintains effectiveness on general ZSD datasets, PASCAL VOC and MS COCO. The code and dataset can be found at https://github.com/LanceZPF/SeeDS.