Neural processing of complex continual input streams

Felix A. Gers; Jurgen Schmidhuber

Neural processing of complex continual input streams

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Long Short-Term Memory (LSTM) can learn algorithms for temporal pattern processing not learnable by alternative recurrent neural networks (RNNs) or other methods such as Hidden Markov Models (HMMs) and symbolic grammar learning (SGL). Here we present tasks involving arithmetic operations on continual input streams that even LSTM cannot solve. But an LSTM variant based on `forget gates,' a recent extension, has superior arithmetic capabilities and does solve the tasks.

Original language	English (US)
Title of host publication	Proceedings of the International Joint Conference on Neural Networks
Publisher	IEEEPiscataway, NJ, United States
Pages	557-562
Number of pages	6
State	Published - Jan 1 2000
Externally published	Yes

Cite this

@inproceedings{d7af284dd0954254ab4d8bd6d657b8f1,

title = "Neural processing of complex continual input streams",

abstract = "Long Short-Term Memory (LSTM) can learn algorithms for temporal pattern processing not learnable by alternative recurrent neural networks (RNNs) or other methods such as Hidden Markov Models (HMMs) and symbolic grammar learning (SGL). Here we present tasks involving arithmetic operations on continual input streams that even LSTM cannot solve. But an LSTM variant based on `forget gates,' a recent extension, has superior arithmetic capabilities and does solve the tasks.",

author = "Gers, {Felix A.} and Jurgen Schmidhuber",

note = "Generated from Scopus record by KAUST IRTS on 2022-09-14",

year = "2000",

month = jan,

day = "1",

language = "English (US)",

pages = "557--562",

booktitle = "Proceedings of the International Joint Conference on Neural Networks",

publisher = "IEEEPiscataway, NJ, United States",

}

TY - GEN

T1 - Neural processing of complex continual input streams

AU - Gers, Felix A.

AU - Schmidhuber, Jurgen

N1 - Generated from Scopus record by KAUST IRTS on 2022-09-14

PY - 2000/1/1

Y1 - 2000/1/1

N2 - Long Short-Term Memory (LSTM) can learn algorithms for temporal pattern processing not learnable by alternative recurrent neural networks (RNNs) or other methods such as Hidden Markov Models (HMMs) and symbolic grammar learning (SGL). Here we present tasks involving arithmetic operations on continual input streams that even LSTM cannot solve. But an LSTM variant based on `forget gates,' a recent extension, has superior arithmetic capabilities and does solve the tasks.

AB - Long Short-Term Memory (LSTM) can learn algorithms for temporal pattern processing not learnable by alternative recurrent neural networks (RNNs) or other methods such as Hidden Markov Models (HMMs) and symbolic grammar learning (SGL). Here we present tasks involving arithmetic operations on continual input streams that even LSTM cannot solve. But an LSTM variant based on `forget gates,' a recent extension, has superior arithmetic capabilities and does solve the tasks.

UR - http://www.scopus.com/inward/record.url?scp=0033699183&partnerID=8YFLogxK

M3 - Conference contribution

SP - 557

EP - 562

BT - Proceedings of the International Joint Conference on Neural Networks

PB - IEEEPiscataway, NJ, United States

ER -

Neural processing of complex continual input streams

Abstract

Other files and links

Fingerprint

Cite this