Modeling 3D Shapes by Reinforcement Learning
This addresses the challenge of automating 3D shape modeling for applications in computer graphics and design, but it is incremental as it builds on existing artist-based modeling techniques.
The paper tackled the problem of enabling machines to model 3D shapes like human modelers by proposing a two-step deep reinforcement learning framework that parses shapes into primitives and edits geometry, with experiments showing it produces regular and structure-aware mesh models.
We explore how to enable machines to model 3D shapes like human modelers using deep reinforcement learning (RL). In 3D modeling software like Maya, a modeler usually creates a mesh model in two steps: (1) approximating the shape using a set of primitives; (2) editing the meshes of the primitives to create detailed geometry. Inspired by such artist-based modeling, we propose a two-step neural framework based on RL to learn 3D modeling policies. By taking actions and collecting rewards in an interactive environment, the agents first learn to parse a target shape into primitives and then to edit the geometry. To effectively train the modeling agents, we introduce a novel training algorithm that combines heuristic policy, imitation learning and reinforcement learning. Our experiments show that the agents can learn good policies to produce regular and structure-aware mesh models, which demonstrates the feasibility and effectiveness of the proposed RL framework.