On the difficulty of a distributional semantics of spoken language
This work addresses the challenge of learning distributional semantics for spoken language, which is incremental as it builds on existing methods from written text.
The study tackled the problem of adapting unsupervised semantic representation learning from written to spoken language, finding that while simple models achieved some success on synthetic speech, results on natural human speech were inconclusive.
In the domain of unsupervised learning most work on speech has focused on discovering low-level constructs such as phoneme inventories or word-like units. In contrast, for written language, where there is a large body of work on unsupervised induction of semantic representations of words, whole sentences and longer texts. In this study we examine the challenges of adapting these approaches from written to spoken language. We conjecture that unsupervised learning of the semantics of spoken language becomes feasible if we abstract from the surface variability. We simulate this setting with a dataset of utterances spoken by a realistic but uniform synthetic voice. We evaluate two simple unsupervised models which, to varying degrees of success, learn semantic representations of speech fragments. Finally we present inconclusive results on human speech, and discuss the challenges inherent in learning distributional semantic representations on unrestricted natural spoken language.