Online Handwriting Recognition Using Encoder-Decoder Model

In this paper we propose a system that is capable of recognizing raw online handwritten data. The system consists of an advanced type of neural network known as LSTM (Long Short Term Memory) Encoder-decoder combined with a customized attention mechanism layer. The attention mechanism has greatly enhanced the system performance from a low character level accuracy of 53% to an excellent accuracy of 96%. Moreover, the system involves a segmentation algorithm designed to divide the sentences into segments of lines. For the training and testing we employ the IAM On-Line Handwriting database, the source can be found here [1]. The accuracy can be improved even further by integrating our system with a language model to spell check the outputs.