Bayesian Program Learning by Decompiling Amortized Knowledge
This work addresses program synthesis efficiency for AI systems, offering incremental improvements in search reduction.
The authors tackled the problem of inductive program synthesis by introducing a library learning approach that decompiles neural search policy knowledge to extract program components, resulting in faster domain proficiency and improved generalization, especially with fewer example solutions.
DreamCoder is an inductive program synthesis system that, whilst solving problems, learns to simplify search in an iterative wake-sleep procedure. The cost of search is amortized by training a neural search policy, reducing search breadth and effectively "compiling" useful information to compose program solutions across tasks. Additionally, a library of program components is learnt to compress and express discovered solutions in fewer components, reducing search depth. We present a novel approach for library learning that directly leverages the neural search policy, effectively "decompiling" its amortized knowledge to extract relevant program components. This provides stronger amortized inference: the amortized knowledge learnt to reduce search breadth is now also used to reduce search depth. We integrate our approach with DreamCoder and demonstrate faster domain proficiency with improved generalization on a range of domains, particularly when fewer example solutions are available.