It's not what you said, it's how you said it: discriminative perception of speech as a multichannel communication system
This research addresses how humans perceive speech in multichannel communication, offering insights into cognitive processing, but it is incremental as it builds on existing work in prosody and context.
The study tackled how listeners discriminate between true utterances and contextually mismatched ones with identical words, finding that non-lexical cues (e.g., prosody) are more informative than lexical content for this task.
People convey information extremely effectively through spoken interaction using multiple channels of information transmission: the lexical channel of what is said, and the non-lexical channel of how it is said. We propose studying human perception of spoken communication as a means to better understand how information is encoded across these channels, focusing on the question 'What characteristics of communicative context affect listener's expectations of speech?'. To investigate this, we present a novel behavioural task testing whether listeners can discriminate between the true utterance in a dialogue and utterances sampled from other contexts with the same lexical content. We characterize how perception - and subsequent discriminative capability - is affected by different degrees of additional contextual information across both the lexical and non-lexical channel of speech. Results demonstrate that people can effectively discriminate between different prosodic realisations, that non-lexical context is informative, and that this channel provides more salient information than the lexical channel, highlighting the importance of the non-lexical channel in spoken interaction.