Ambiguity Resolution with Human Feedback for Code Writing Tasks
This addresses the challenge for programmers and students in computer science education by providing an assistive tool to handle specification ambiguities, though it appears incremental in nature.
The paper tackled the problem of ambiguous natural language specifications in code writing tasks by developing a prototype system that suggests ambiguous inputs, seeks human feedback, and generates code to resolve ambiguities, showing efficacy in evaluation.
Specifications for code writing tasks are usually expressed in natural language and may be ambiguous. Programmers must therefore develop the ability to recognize ambiguities in task specifications and resolve them by asking clarifying questions. We present and evaluate a prototype system, based on a novel technique (ARHF: Ambiguity Resolution with Human Feedback), that (1) suggests specific inputs on which a given task specification may be ambiguous, (2) seeks limited human feedback about the code's desired behavior on those inputs, and (3) uses this feedback to generate code that resolves these ambiguities. We evaluate the efficacy of our prototype, and we discuss the implications of such assistive systems on Computer Science education.