Neural Style Transfer for Computer Games
This addresses the challenge of real-time artistic stylization for game developers and players, though it is incremental as it builds on existing neural style transfer techniques.
The paper tackled the problem of applying neural style transfer to 3D computer games, which often causes artifacts when used as a post-processing effect, by developing a depth-aware method integrated into the rendering pipeline, resulting in temporally consistent stylized scenes that outperform state-of-the-art image and video methods.
Neural Style Transfer (NST) research has been applied to images, videos, 3D meshes and radiance fields, but its application to 3D computer games remains relatively unexplored. Whilst image and video NST systems can be used as a post-processing effect for a computer game, this results in undesired artefacts and diminished post-processing effects. Here, we present an approach for injecting depth-aware NST as part of the 3D rendering pipeline. Qualitative and quantitative experiments are used to validate our in-game stylisation framework. We demonstrate temporally consistent results of artistically stylised game scenes, outperforming state-of-the-art image and video NST methods.