A Longitudinal Multi-modal Dataset for Dementia Monitoring and Diagnosis
This dataset addresses the lack of consistent longitudinal data for dementia monitoring, which is a problem for researchers and clinicians in healthcare, but it is incremental as it builds on existing speech and language analysis methods.
The authors tackled the problem of diagnosing and monitoring dementia by introducing a novel longitudinal multi-modal dataset collected from healthy controls and people with dementia over 56 sessions, establishing baselines for capturing changes in language across modalities.
Dementia affects cognitive functions of adults, including memory, language, and behaviour. Standard diagnostic biomarkers such as MRI are costly, whilst neuropsychological tests suffer from sensitivity issues in detecting dementia onset. The analysis of speech and language has emerged as a promising and non-intrusive technology to diagnose and monitor dementia. Currently, most work in this direction ignores the multi-modal nature of human communication and interactive aspects of everyday conversational interaction. Moreover, most studies ignore changes in cognitive status over time due to the lack of consistent longitudinal data. Here we introduce a novel fine-grained longitudinal multi-modal corpus collected in a natural setting from healthy controls and people with dementia over two phases, each spanning 28 sessions. The corpus consists of spoken conversations, a subset of which are transcribed, as well as typed and written thoughts and associated extra-linguistic information such as pen strokes and keystrokes. We present the data collection process and describe the corpus in detail. Furthermore, we establish baselines for capturing longitudinal changes in language across different modalities for two cohorts, healthy controls and people with dementia, outlining future research directions enabled by the corpus.