Towards objectively evaluating the quality of generated medical summaries
This addresses the need for reliable evaluation in medical report summarization, where accuracy is critical, but it is incremental as it adapts existing metrics to a specific domain.
The paper tackled the problem of evaluating generated medical summaries by introducing a method based on counting facts to compute precision, recall, f-score, and accuracy, aiming for more objective and reproducible assessments.
We propose a method for evaluating the quality of generated text by asking evaluators to count facts, and computing precision, recall, f-score, and accuracy from the raw counts. We believe this approach leads to a more objective and easier to reproduce evaluation. We apply this to the task of medical report summarisation, where measuring objective quality and accuracy is of paramount importance.