Decomposed Prompting to Answer Questions on a Course Discussion Board
This addresses the specific problem of automating question-answering for students in educational settings, but it is incremental as it builds on existing LLM techniques.
The researchers tackled the problem of automatically answering student questions on course discussion boards by developing a system that uses decomposed prompting to classify questions into four types and apply different answering strategies, achieving 81% classification accuracy with a GPT-3 variant.
We propose and evaluate a question-answering system that uses decomposed prompting to classify and answer student questions on a course discussion board. Our system uses a large language model (LLM) to classify questions into one of four types: conceptual, homework, logistics, and not answerable. This enables us to employ a different strategy for answering questions that fall under different types. Using a variant of GPT-3, we achieve $81\%$ classification accuracy. We discuss our system's performance on answering conceptual questions from a machine learning course and various failure modes.