Style-Aware Gloss Control for Generative Non-Photorealistic Rendering
This work addresses the challenge of material appearance control in AI-generated art, offering incremental improvements in disentanglement for artists and designers.
The paper tackled the problem of controlling gloss and artistic style in generative non-photorealistic rendering by training an unsupervised model on a curated dataset, revealing a hierarchical latent space that disentangles gloss from other factors, and introducing a lightweight adapter for fine-grained control in image synthesis, with results showing improved disentanglement and controllability compared to previous models.
Humans can infer material characteristics of objects from their visual appearance, and this ability extends to artistic depictions, where similar perceptual strategies guide the interpretation of paintings or drawings. Among the factors that define material appearance, gloss, along with color, is widely regarded as one of the most important, and recent studies indicate that humans can perceive gloss independently of the artistic style used to depict an object. To investigate how gloss and artistic style are represented in learned models, we train an unsupervised generative model on a newly curated dataset of painterly objects designed to systematically vary such factors. Our analysis reveals a hierarchical latent space in which gloss is disentangled from other appearance factors, allowing for a detailed study of how gloss is represented and varies across artistic styles. Building on this representation, we introduce a lightweight adapter that connects our style- and gloss-aware latent space to a latent-diffusion model, enabling the synthesis of non-photorealistic images with fine-grained control of these factors. We compare our approach with previous models and observe improved disentanglement and controllability of the learned factors.