Physically Embodied Deep Image Optimisation
This work addresses the problem of automating physical art creation for robotics and digital fabrication, but it is incremental as it builds on existing differentiable rendering and optimization techniques.
The paper tackled the problem of generating physical sketches from images by learning programs to control a drawing robot, achieving this through a differentiable rasteriser and deep networks to optimise drawing strokes that can be translated into G-code for robotic execution.
Physical sketches are created by learning programs to control a drawing robot. A differentiable rasteriser is used to optimise sets of drawing strokes to match an input image, using deep networks to provide an encoding for which we can compute a loss. The optimised drawing primitives can then be translated into G-code commands which command a robot to draw the image using drawing instruments such as pens and pencils on a physical support medium.