Enabling My Robot To Play Pictionary : Recurrent Neural Networks For Sketch Recognition
This work addresses the problem of real-time sketch recognition for applications like interactive games, but it is incremental as it builds on existing RNN methods.
The paper tackled sketch object recognition by proposing a recurrent neural network architecture that exploits sequential stroke data, achieving state-of-the-art results on a large database of freehand sketches across many categories.
Freehand sketching is an inherently sequential process. Yet, most approaches for hand-drawn sketch recognition either ignore this sequential aspect or exploit it in an ad-hoc manner. In our work, we propose a recurrent neural network architecture for sketch object recognition which exploits the long-term sequential and structural regularities in stroke data in a scalable manner. Specifically, we introduce a Gated Recurrent Unit based framework which leverages deep sketch features and weighted per-timestep loss to achieve state-of-the-art results on a large database of freehand object sketches across a large number of object categories. The inherently online nature of our framework is especially suited for on-the-fly recognition of objects as they are being drawn. Thus, our framework can enable interesting applications such as camera-equipped robots playing the popular party game Pictionary with human players and generating sparsified yet recognizable sketches of objects.