A Technique for Isolating Lexically-Independent Phonetic Dependencies in Generative CNNs
This addresses the open question of lexically-invariant generalization in DNNs for researchers in speech processing and machine learning, though it appears incremental as it builds on existing CNN methods with a specific probing technique.
The study tackled the problem of whether deep neural networks can represent phonotactic generalizations independent of lexical learning by investigating generative CNNs trained on raw audio waveforms, and found that convolutional layers can generalize phonetic dependencies beyond lexical constraints, as shown by outputs generated with randomized feature maps being equally biased by phonotactic restrictions.
The ability of deep neural networks (DNNs) to represent phonotactic generalizations derived from lexical learning remains an open question. This study (1) investigates the lexically-invariant generalization capacity of generative convolutional neural networks (CNNs) trained on raw audio waveforms of lexical items and (2) explores the consequences of shrinking the fully-connected layer (FC) bottleneck from 1024 channels to 8 before training. Ultimately, a novel technique for probing a model's lexically-independent generalizations is proposed that works only under the narrow FC bottleneck: generating audio outputs by bypassing the FC and inputting randomized feature maps into the convolutional block. These outputs are equally biased by a phonotactic restriction in training as are outputs generated with the FC. This result shows that the convolutional layers can dynamically generalize phonetic dependencies beyond lexically-constrained configurations learned by the FC.