AutoCaption: Image Captioning with Neural Architecture Search
This work addresses the challenge of designing efficient and effective deep neural networks for image captioning, which is a problem for computer vision and natural language processing researchers.
This paper tackles the problem of automatically designing the decoder module for image captioning models. They propose AutoCaption, which uses Neural Architecture Search (NAS) to design an AutoRNN decoder. Experiments on the MSCOCO dataset show that AutoCaption achieves better performance than traditional hand-designed methods.
Image captioning transforms complex visual information into abstract natural language for representation, which can help computers understanding the world quickly. However, due to the complexity of the real environment, it needs to identify key objects and realize their connections, and further generate natural language. The whole process involves a visual understanding module and a language generation module, which brings more challenges to the design of deep neural networks than other tasks. Neural Architecture Search (NAS) has shown its important role in a variety of image recognition tasks. Besides, RNN plays an essential role in the image captioning task. We introduce a AutoCaption method to better design the decoder module of the image captioning where we use the NAS to design the decoder module called AutoRNN automatically. We use the reinforcement learning method based on shared parameters for automatic design the AutoRNN efficiently. The search space of the AutoCaption includes connections between the layers and the operations in layers both, and it can make AutoRNN express more architectures. In particular, RNN is equivalent to a subset of our search space. Experiments on the MSCOCO datasets show that our AutoCaption model can achieve better performance than traditional hand-design methods.