DartsReNet: Exploring new RNN cells in ReNet architectures
This work addresses image classification challenges by enhancing ReNet architectures, though it is incremental as it builds on existing NAS and ReNet methods.
The paper tackled the limitation of standard RNN cells for 2D image data by using DARTS to discover new RNN cells for ReNet architectures, resulting in improved performance on CIFAR-10 and SVHN datasets compared to GRU and LSTM cells.
We present new Recurrent Neural Network (RNN) cells for image classification using a Neural Architecture Search (NAS) approach called DARTS. We are interested in the ReNet architecture, which is a RNN based approach presented as an alternative for convolutional and pooling steps. ReNet can be defined using any standard RNN cells, such as LSTM and GRU. One limitation is that standard RNN cells were designed for one dimensional sequential data and not for two dimensions like it is the case for image classification. We overcome this limitation by using DARTS to find new cell designs. We compare our results with ReNet that uses GRU and LSTM cells. Our found cells outperform the standard RNN cells on CIFAR-10 and SVHN. The improvements on SVHN indicate generalizability, as we derived the RNN cell designs from CIFAR-10 without performing a new cell search for SVHN.