Geometry of Program Synthesis
This foundational work could impact all of ML/AI by providing a new geometric perspective on program synthesis, though it appears incremental as it builds on existing synthesis concepts.
The paper re-evaluates universal computation by viewing programs as singularities of analytic varieties or phases of Bayesian posteriors in synthesis problems, revealing new research directions in program synthesis including neural networks, and reports on simple experimental implementations.
We re-evaluate universal computation based on the synthesis of Turing machines. This leads to a view of programs as singularities of analytic varieties or, equivalently, as phases of the Bayesian posterior of a synthesis problem. This new point of view reveals unexplored directions of research in program synthesis, of which neural networks are a subset, for example in relation to phase transitions, complexity and generalisation. We also lay the empirical foundations for these new directions by reporting on our implementation in code of some simple experiments.