Content Masked Loss: Human-Like Brush Stroke Planning in a Reinforcement Learning Painting Agent
This work addresses the problem of making robot painting agents produce more human-like stroke sequences for improved interpretability and artistic process, particularly for those interested in the artistic applications of AI.
This paper introduces Content Masked Loss, a new loss function for reinforcement learning painting agents. It uses an object detection model to prioritize important regions of the canvas, resulting in paintings where subject matter is detectable earlier in the stroke sequence compared to existing methods, as validated by 332 human evaluators, without sacrificing final painting quality.
The objective of most Reinforcement Learning painting agents is to minimize the loss between a target image and the paint canvas. Human painter artistry emphasizes important features of the target image rather than simply reproducing it (DiPaola 2007). Using adversarial or L2 losses in the RL painting models, although its final output is generally a work of finesse, produces a stroke sequence that is vastly different from that which a human would produce since the model does not have knowledge about the abstract features in the target image. In order to increase the human-like planning of the model without the use of expensive human data, we introduce a new loss function for use with the model's reward function: Content Masked Loss. In the context of robot painting, Content Masked Loss employs an object detection model to extract features which are used to assign higher weight to regions of the canvas that a human would find important for recognizing content. The results, based on 332 human evaluators, show that the digital paintings produced by our Content Masked model show detectable subject matter earlier in the stroke sequence than existing methods without compromising on the quality of the final painting. Our code is available at https://github.com/pschaldenbrand/ContentMaskedLoss.