Growing 3D Artefacts and Functional Machines with Neural Cellular Automata
This enables simulated morphogenetic systems to create 3D artifacts and functional machines, though it's an incremental extension of existing NCA methods to a new domain.
The authors extended Neural Cellular Automata (NCAs) from 2D to 3D using 3D convolutions, enabling the growth of complex structures like castles and trees with over 3,000 blocks in Minecraft, and demonstrated regeneration of simple functional machines.
Neural Cellular Automata (NCAs) have been proven effective in simulating morphogenetic processes, the continuous construction of complex structures from very few starting cells. Recent developments in NCAs lie in the 2D domain, namely reconstructing target images from a single pixel or infinitely growing 2D textures. In this work, we propose an extension of NCAs to 3D, utilizing 3D convolutions in the proposed neural network architecture. Minecraft is selected as the environment for our automaton since it allows the generation of both static structures and moving machines. We show that despite their simplicity, NCAs are capable of growing complex entities such as castles, apartment blocks, and trees, some of which are composed of over 3,000 blocks. Additionally, when trained for regeneration, the system is able to regrow parts of simple functional machines, significantly expanding the capabilities of simulated morphogenetic systems. The code for the experiment in this paper can be found at: https://github.com/real-itu/3d-artefacts-nca.