ChatGPT Incorrectness Detection in Software Reviews
This addresses the reliability concerns of software practitioners using ChatGPT for tasks like library selection, though it is an incremental improvement focused on a specific domain.
The paper tackled the problem of detecting incorrect responses from ChatGPT in software engineering tasks, developing a tool called CID that uses iterative prompting with metamorphic text relationships to identify inconsistencies. In a benchmark study on library selection, CID achieved an F1-score of 0.74-0.75 for detecting incorrect responses.
We conducted a survey of 135 software engineering (SE) practitioners to understand how they use Generative AI-based chatbots like ChatGPT for SE tasks. We find that they want to use ChatGPT for SE tasks like software library selection but often worry about the truthfulness of ChatGPT responses. We developed a suite of techniques and a tool called CID (ChatGPT Incorrectness Detector) to automatically test and detect the incorrectness in ChatGPT responses. CID is based on the iterative prompting to ChatGPT by asking it contextually similar but textually divergent questions (using an approach that utilizes metamorphic relationships in texts). The underlying principle in CID is that for a given question, a response that is different from other responses (across multiple incarnations of the question) is likely an incorrect response. In a benchmark study of library selection, we show that CID can detect incorrect responses from ChatGPT with an F1-score of 0.74 - 0.75.