A Survey of Machine Learning for Big Code and Naturalness
It provides a comprehensive overview for researchers in machine learning, programming languages, and software engineering, but is incremental as it synthesizes existing work.
The paper surveys probabilistic models of source code that leverage patterns in code, contrasting programming languages with natural languages and presenting a taxonomy to navigate the literature.
Research at the intersection of machine learning, programming languages, and software engineering has recently taken important steps in proposing learnable probabilistic models of source code that exploit code's abundance of patterns. In this article, we survey this work. We contrast programming languages against natural languages and discuss how these similarities and differences drive the design of probabilistic models. We present a taxonomy based on the underlying design principles of each model and use it to navigate the literature. Then, we review how researchers have adapted these models to application areas and discuss cross-cutting and application-specific challenges and opportunities.