Beyond Scores: A Modular RAG-Based System for Automatic Short Answer Scoring with FeedbackMenna Fateen, Bo Wang, Tsunenori Mine
Automatic short answer scoring (ASAS) helps reduce the grading burden on educators but often lacks detailed, explainable feedback. Existing methods in ASAS with feedback (ASAS-F) rely on fine-tuning language models with limited datasets, which is resource-intensive and struggles to generalize across contexts. Recent approaches using large language models (LLMs) have focused on scoring without extensive fine-tuning. However, they often rely heavily on prompt engineering and either fail to generate elaborated feedback or do not adequately evaluate it. In this paper, we propose a modular retrieval augmented generation based ASAS-F system that scores answers and generates feedback in strict zero-shot and few-shot learning scenarios. We design our system to be adaptable to various educational tasks without extensive prompt engineering using an automatic prompt generation framework. Results show an improvement in scoring accuracy by 9\% on unseen questions compared to fine-tuning, offering a scalable and cost-effective solution.
2.6LGAug 4, 2024
A multi-task deep learning approach for lane-level pavement performance prediction with segment-level dataBo Wang, Wenbo Zhang, Yunpeng LI
The elaborate pavement performance prediction is an important premise of implementing preventive maintenance. Our survey reveals that in practice, the pavement performance is usually measured at segment-level, where an unique performance value is obtained for all lanes within one segment of 1km length. It still lacks more elaborate performance analysis at lane-level due to costly data collection and difficulty in prediction modeling. Therefore, this study developed a multi-task deep learning approach to predict the lane-level pavement performance with a large amount of historical segment-level performance measurement data. The unified prediction framework can effectively address inherent correlation and differences across lanes. In specific, the prediction framework firstly employed an Long Short-Term Memory (LSTM) layer to capture the segment-level pavement deterioration pattern. Then multiple task-specific LSTM layers were designed based on number of lanes to capture lane-level differences in pavement performance. Finally, we concatenated multiple task-specific LSTM outputs with auxiliary features for further training and obtained the lane-level predictions after fully connected layer. The aforementioned prediction framework was validated with a real case in China. It revealed a better model performance regardless of one-way 2-lane, 3-lane, and 4-lane scenarios, all lower than 10% in terms of mean absolute percentage error. The proposed prediction framework also outperforms other ensemble learning and shallow machine learning methods in almost every lane.