Improving long-term online prediction with decoupled extended Kalman filters

Juan A. Pérez-Ortiz; Jürgen Schmidhuber; Felix A. Gers; Douglas Eck

doi:10.1007/3-540-46084-5_171

Improving long-term online prediction with decoupled extended Kalman filters

Juan A. Pérez-Ortiz, Jürgen Schmidhuber, Felix A. Gers, Douglas Eck

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

5 Scopus citations

Abstract

Long Short-Term Memory (LSTM) recurrent neural networks (RNNs) outperform traditional RNNs when dealing with sequences involving not only short-term but also long-term dependencies. The decoupled extended Kalman filter learning algorithm (DEKF) works well in online environments and reduces significantly the number of training steps when compared to the standard gradient-descent algorithms. Previous work on LSTM, however, has always used a form of gradient descent and has not focused on true online situations. Here we combine LSTM with DEKF and show that this new hybrid improves upon the original learning algorithm when applied to online processing. © Springer-Verlag Berlin Heidelberg 2002.

Original language	English (US)
Title of host publication	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Publisher	Springer Verlag
Pages	1055-1060
Number of pages	6
ISBN (Print)	9783540440741
DOIs	https://doi.org/10.1007/3-540-46084-5_171
State	Published - Jan 1 2002
Externally published	Yes

Access to Document

10.1007/3-540-46084-5_171

Cite this

@inproceedings{026d980c195c47d1aabea1c62cf52676,

title = "Improving long-term online prediction with decoupled extended Kalman filters",

abstract = "Long Short-Term Memory (LSTM) recurrent neural networks (RNNs) outperform traditional RNNs when dealing with sequences involving not only short-term but also long-term dependencies. The decoupled extended Kalman filter learning algorithm (DEKF) works well in online environments and reduces significantly the number of training steps when compared to the standard gradient-descent algorithms. Previous work on LSTM, however, has always used a form of gradient descent and has not focused on true online situations. Here we combine LSTM with DEKF and show that this new hybrid improves upon the original learning algorithm when applied to online processing. {\textcopyright} Springer-Verlag Berlin Heidelberg 2002.",

author = "P{\'e}rez-Ortiz, {Juan A.} and J{\"u}rgen Schmidhuber and Gers, {Felix A.} and Douglas Eck",

note = "Generated from Scopus record by KAUST IRTS on 2022-09-14",

year = "2002",

month = jan,

day = "1",

doi = "10.1007/3-540-46084-5_171",

language = "English (US)",

isbn = "9783540440741",

pages = "1055--1060",

booktitle = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

address = "Germany",

}

Improving long-term online prediction with decoupled extended Kalman filters. / Pérez-Ortiz, Juan A.; Schmidhuber, Jürgen; Gers, Felix A. et al.
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer Verlag, 2002. p. 1055-1060.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Improving long-term online prediction with decoupled extended Kalman filters

AU - Pérez-Ortiz, Juan A.

AU - Schmidhuber, Jürgen

AU - Gers, Felix A.

AU - Eck, Douglas

N1 - Generated from Scopus record by KAUST IRTS on 2022-09-14

PY - 2002/1/1

Y1 - 2002/1/1

N2 - Long Short-Term Memory (LSTM) recurrent neural networks (RNNs) outperform traditional RNNs when dealing with sequences involving not only short-term but also long-term dependencies. The decoupled extended Kalman filter learning algorithm (DEKF) works well in online environments and reduces significantly the number of training steps when compared to the standard gradient-descent algorithms. Previous work on LSTM, however, has always used a form of gradient descent and has not focused on true online situations. Here we combine LSTM with DEKF and show that this new hybrid improves upon the original learning algorithm when applied to online processing. © Springer-Verlag Berlin Heidelberg 2002.

AB - Long Short-Term Memory (LSTM) recurrent neural networks (RNNs) outperform traditional RNNs when dealing with sequences involving not only short-term but also long-term dependencies. The decoupled extended Kalman filter learning algorithm (DEKF) works well in online environments and reduces significantly the number of training steps when compared to the standard gradient-descent algorithms. Previous work on LSTM, however, has always used a form of gradient descent and has not focused on true online situations. Here we combine LSTM with DEKF and show that this new hybrid improves upon the original learning algorithm when applied to online processing. © Springer-Verlag Berlin Heidelberg 2002.

UR - http://link.springer.com/10.1007/3-540-46084-5_171

UR - http://www.scopus.com/inward/record.url?scp=84892520259&partnerID=8YFLogxK

U2 - 10.1007/3-540-46084-5_171

DO - 10.1007/3-540-46084-5_171

M3 - Conference contribution

SN - 9783540440741

SP - 1055

EP - 1060

BT - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

PB - Springer Verlag

ER -

Improving long-term online prediction with decoupled extended Kalman filters

Abstract

Access to Document

Other files and links

Fingerprint

Cite this