Do People Prefer "Natural" code?
This addresses the problem of code readability and maintainability for developers, though it is incremental as it builds on existing theories of code bimodality.
The study investigated why code is highly repetitive despite programming languages allowing many ways to express computations, hypothesizing that it's due to code being bimodal (executed by machines and read by humans). They found that meaning-preserving transformations generally produce less common program structures, supporting the theory that repetitiveness is a deliberate preference, and showed alignment between language model scores and human preferences.
Natural code is known to be very repetitive (much more so than natural language corpora); furthermore, this repetitiveness persists, even after accounting for the simpler syntax of code. However, programming languages are very expressive, allowing a great many different ways (all clear and unambiguous) to express even very simple computations. So why is natural code repetitive? We hypothesize that the reasons for this lie in fact that code is bimodal: it is executed by machines, but also read by humans. This bimodality, we argue, leads developers to write code in certain preferred ways that would be familiar to code readers. To test this theory, we 1) model familiarity using a language model estimated over a large training corpus and 2) run an experiment applying several meaning preserving transformations to Java and Python expressions in a distinct test corpus to see if forms more familiar to readers (as predicted by the language models) are in fact the ones actually written. We find that these transformations generally produce program structures that are less common in practice, supporting the theory that the high repetitiveness in code is a matter of deliberate preference. Finally, 3) we use a human subject study to show alignment between language model score and human preference for the first time in code, providing support for using this measure to improve code.