Lagrangian Neural Style Transfer for Fluids
This addresses challenges in visual effects production by making neural flow stylization practical and efficient, though it is an incremental improvement over existing grid-based techniques.
The paper tackles the problem of artistically controlling fluid simulations for visual effects by introducing a neural style transfer method from images to 3D fluids using a Lagrangian particle-based approach, resulting in improved temporal consistency, reduced computation time to less than an hour, and enhanced artistic control for smoke and liquids.
Artistically controlling the shape, motion and appearance of fluid simulations pose major challenges in visual effects production. In this paper, we present a neural style transfer approach from images to 3D fluids formulated in a Lagrangian viewpoint. Using particles for style transfer has unique benefits compared to grid-based techniques. Attributes are stored on the particles and hence are trivially transported by the particle motion. This intrinsically ensures temporal consistency of the optimized stylized structure and notably improves the resulting quality. Simultaneously, the expensive, recursive alignment of stylization velocity fields of grid approaches is unnecessary, reducing the computation time to less than an hour and rendering neural flow stylization practical in production settings. Moreover, the Lagrangian representation improves artistic control as it allows for multi-fluid stylization and consistent color transfer from images, and the generality of the method enables stylization of smoke and liquids likewise.