Program Synthesis with Pragmatic Communication
This addresses the problem of ambiguous specifications for end-users in program synthesis, offering an incremental improvement by integrating pragmatic reasoning.
The paper tackles the ambiguity in program synthesis from user specifications by introducing a pragmatic communication model as an inductive bias, scoring programs based on consistency and whether a rational speaker would choose the specification to convey them. In a user study, participants communicated more effectively with the pragmatic synthesizer compared to a non-pragmatic one.
Program synthesis techniques construct or infer programs from user-provided specifications, such as input-output examples. Yet most specifications, especially those given by end-users, leave the synthesis problem radically ill-posed, because many programs may simultaneously satisfy the specification. Prior work resolves this ambiguity by using various inductive biases, such as a preference for simpler programs. This work introduces a new inductive bias derived by modeling the program synthesis task as rational communication, drawing insights from recursive reasoning models of pragmatics. Given a specification, we score a candidate program both on its consistency with the specification, and also whether a rational speaker would chose this particular specification to communicate that program. We develop efficient algorithms for such an approach when learning from input-output examples, and build a pragmatic program synthesizer over a simple grid-like layout domain. A user study finds that end-user participants communicate more effectively with the pragmatic program synthesizer over a non-pragmatic one.