Towards an IMU-based Pen Online Handwriting Recognizer
This enables handwriting recognition on paper for users needing portable digitization, though it is incremental as it adapts existing neural methods to new sensor data.
The paper tackles the problem of online handwriting recognition without a writing surface by using an IMU-equipped pen, achieving character error rates of 17.97% on seen words and 17.08% on unseen words without a dictionary or language model.
Most online handwriting recognition systems require the use of specific writing surfaces to extract positional data. In this paper we present a online handwriting recognition system for word recognition which is based on inertial measurement units (IMUs) for digitizing text written on paper. This is obtained by means of a sensor-equipped pen that provides acceleration, angular velocity, and magnetic forces streamed via Bluetooth. Our model combines convolutional and bidirectional LSTM networks, and is trained with the Connectionist Temporal Classification loss that allows the interpretation of raw sensor data into words without the need of sequence segmentation. We use a dataset of words collected using multiple sensor-enhanced pens and evaluate our model on distinct test sets of seen and unseen words achieving a character error rate of 17.97% and 17.08%, respectively, without the use of a dictionary or language model