Learning Word Embeddings from Speech
This addresses the problem of extracting semantic information from speech without relying on text or images, which can be expensive to collect, though it is incremental in method.
The paper tackles unsupervised learning of semantic vector representations directly from raw speech audio segments, achieving competitive results on 13 word similarity benchmarks compared to GloVe.
In this paper, we propose a novel deep neural network architecture, Sequence-to-Sequence Audio2Vec, for unsupervised learning of fixed-length vector representations of audio segments excised from a speech corpus, where the vectors contain semantic information pertaining to the segments, and are close to other vectors in the embedding space if their corresponding segments are semantically similar. The design of the proposed model is based on the RNN Encoder-Decoder framework, and borrows the methodology of continuous skip-grams for training. The learned vector representations are evaluated on 13 widely used word similarity benchmarks, and achieved competitive results to that of GloVe. The biggest advantage of the proposed model is its capability of extracting semantic information of audio segments taken directly from raw speech, without relying on any other modalities such as text or images, which are challenging and expensive to collect and annotate.