Visual Retrieval-Augmented Generation for Silhouette-Guided Animal Art
This work addresses the challenge of computational pareidolia for artists and designers, but the results are incremental with limited perceptual impact.
The paper introduces Visual-RAG, a framework that generates animal art from natural silhouettes by retrieving structurally similar animal shapes from a corpus of 28,586 silhouettes and using them to guide diffusion-based generation. A user study with 12 participants showed plausible interpretations but limited perceptual impact, with shape standardization critical for structural fidelity (inlier ratio drops to 13.4% without it).
Generative AI has advanced the ability to render photorealistic or artistic images, yet it remains limited in a key aspect of human creativity: interpreting ambiguous shapes. This phenomenon, rooted in pareidolia, allows humans to perceive meaningful forms in random patterns such as clouds, stones, or leaves. To computationally replicate this imaginative process, we introduce Visual Retrieval-Augmented Generation (Visual-RAG), a framework that generates animal art directly from natural silhouettes. Our method retrieves structurally similar animal shapes from a curated corpus of 28,586 high-quality silhouettes and uses them as reference exemplars to guide diffusion-based generation with ControlNet and IP-Adapter. Ablation studies confirm that shape Context with RANSAC provides the most accurate alignment, while removing shape standardization reduces the inlier ratio to just 13.4\%, underscoring the importance of structural fidelity in Visual-RAG. A user study with 12 participants evaluated the outputs in terms of aesthetics, silhouette fidelity, and overall impression. Results reveal that while Visual-RAG provides plausible interpretations, challenges remain in achieving high perceptual impact. This work lays the foundation for computational pareidolia, showing how machines can contribute to the early stages of imaginative discovery.