Long Short-Term Memory (LSTM) can learn algorithms for temporal pattern processing not learnable by alternative recurrent neural networks (RNNs) or other methods such as Hidden Markov Models (HMMs) and symbolic grammar learning (SGL). Here we present tasks involving arithmetic operations on continual input streams that even LSTM cannot solve. But an LSTM variant based on `forget gates,' a recent extension, has superior arithmetic capabilities and does solve the tasks.
|Original language||English (US)|
|Title of host publication||Proceedings of the International Joint Conference on Neural Networks|
|Publisher||IEEEPiscataway, NJ, United States|
|Number of pages||6|
|State||Published - Jan 1 2000|