Space of Functions Computed by Deep-Layered Machines

This work addresses a foundational problem in theoretical machine learning by analyzing the functional space of deep architectures, but it appears incremental as it builds on existing models without introducing new paradigms or broad applications.

The paper investigates the space of functions computed by random-layered machines, such as deep neural networks and Boolean circuits, finding that the distribution of Boolean functions is identical in recurrent and layer-dependent architectures. It characterizes this space in the large depth limit, showing that the macroscopic entropy of Boolean functions either monotonically increases or decreases with depth, depending on initial conditions and computing elements.