Investigating Low-Cost LLM Annotation for~Spoken Dialogue Understanding Datasets
This work addresses a domain-specific bottleneck in spoken Task-Oriented Dialogue systems, offering incremental improvements for dataset annotation.
The paper tackled the problem of enhancing semantic representations in spoken dialogue datasets, which lag behind textual ones, by investigating low-cost LLM annotation methods; it assessed fine-tuning relevance, evaluated annotation knowledge capture, and highlighted semi-automatic implications.
In spoken Task-Oriented Dialogue (TOD) systems, the choice of the semantic representation describing the users' requests is key to a smooth interaction. Indeed, the system uses this representation to reason over a database and its domain knowledge to choose its next action. The dialogue course thus depends on the information provided by this semantic representation. While textual datasets provide fine-grained semantic representations, spoken dialogue datasets fall behind. This paper provides insights into automatic enhancement of spoken dialogue datasets' semantic representations. Our contributions are three fold: (1) assess the relevance of Large Language Model fine-tuning, (2) evaluate the knowledge captured by the produced annotations and (3) highlight semi-automatic annotation implications.