Dynamic Cell Structure via Recursive-Recurrent Neural Networks
This work addresses the need for more efficient and adaptive neural architecture search in recurrent neural networks, though it appears incremental as it builds on existing methods.
The paper tackles the problem of static neural architecture search in recurrent settings by proposing a dynamic algorithm that customizes cell structures per data sample and time step, achieving better prediction accuracy than GRU on language modeling and sentiment analysis datasets.
In a recurrent setting, conventional approaches to neural architecture search find and fix a general model for all data samples and time steps. We propose a novel algorithm that can dynamically search for the structure of cells in a recurrent neural network model. Based on a combination of recurrent and recursive neural networks, our algorithm is able to construct customized cell structures for each data sample and time step, allowing for a more efficient architecture search than existing models. Experiments on three common datasets show that the algorithm discovers high-performance cell architectures and achieves better prediction accuracy compared to the GRU structure for language modelling and sentiment analysis.