Unsupervised learning in LSTM recurrent neural networks

Magdalena Klapper-Rybicka; Nicol N. Schraudolph; Jürgen Schmidhuber

doi:10.1007/3-540-44668-0_95

Unsupervised learning in LSTM recurrent neural networks

Magdalena Klapper-Rybicka, Nicol N. Schraudolph, Jürgen Schmidhuber

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

21 Scopus citations

Abstract

While much work has been done on unsupervised learning in feed forward neural network architectures, its potential with (theoretically more powerful) recurrent networks and time-varying inputs has rarely been explored. Here we train Long Short-Term Memory (LSTM) recurrent networks to maximize two information-theoretic objectives for unsupervised learning: Binary Information Gain Optimization (BINGO) and Nonparametric Entropy Optimization (NEO). LSTM learns to discriminate different types of temporal sequences and group them according to a variety of features.

Original language	English (US)
Title of host publication	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Publisher	Springer [email protected]
Pages	684-691
Number of pages	8
ISBN (Print)	3540424865
DOIs	https://doi.org/10.1007/3-540-44668-0_95
State	Published - Jan 1 2001
Externally published	Yes

Access to Document

10.1007/3-540-44668-0_95

Cite this

@inproceedings{780a05e65b27461e95a9b6f50bf26feb,

title = "Unsupervised learning in LSTM recurrent neural networks",

abstract = "While much work has been done on unsupervised learning in feed forward neural network architectures, its potential with (theoretically more powerful) recurrent networks and time-varying inputs has rarely been explored. Here we train Long Short-Term Memory (LSTM) recurrent networks to maximize two information-theoretic objectives for unsupervised learning: Binary Information Gain Optimization (BINGO) and Nonparametric Entropy Optimization (NEO). LSTM learns to discriminate different types of temporal sequences and group them according to a variety of features.",

author = "Magdalena Klapper-Rybicka and Schraudolph, {Nicol N.} and J{\"u}rgen Schmidhuber",

note = "Generated from Scopus record by KAUST IRTS on 2022-09-14",

year = "2001",

month = jan,

day = "1",

doi = "10.1007/3-540-44668-0_95",

language = "English (US)",

isbn = "3540424865",

pages = "684--691",

booktitle = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer [email protected]",

}

Unsupervised learning in LSTM recurrent neural networks. / Klapper-Rybicka, Magdalena; Schraudolph, Nicol N.; Schmidhuber, Jürgen.
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer [email protected], 2001. p. 684-691.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Unsupervised learning in LSTM recurrent neural networks

AU - Klapper-Rybicka, Magdalena

AU - Schraudolph, Nicol N.

AU - Schmidhuber, Jürgen

N1 - Generated from Scopus record by KAUST IRTS on 2022-09-14

PY - 2001/1/1

Y1 - 2001/1/1

N2 - While much work has been done on unsupervised learning in feed forward neural network architectures, its potential with (theoretically more powerful) recurrent networks and time-varying inputs has rarely been explored. Here we train Long Short-Term Memory (LSTM) recurrent networks to maximize two information-theoretic objectives for unsupervised learning: Binary Information Gain Optimization (BINGO) and Nonparametric Entropy Optimization (NEO). LSTM learns to discriminate different types of temporal sequences and group them according to a variety of features.

AB - While much work has been done on unsupervised learning in feed forward neural network architectures, its potential with (theoretically more powerful) recurrent networks and time-varying inputs has rarely been explored. Here we train Long Short-Term Memory (LSTM) recurrent networks to maximize two information-theoretic objectives for unsupervised learning: Binary Information Gain Optimization (BINGO) and Nonparametric Entropy Optimization (NEO). LSTM learns to discriminate different types of temporal sequences and group them according to a variety of features.

UR - http://link.springer.com/10.1007/3-540-44668-0_95

UR - http://www.scopus.com/inward/record.url?scp=84958950708&partnerID=8YFLogxK

U2 - 10.1007/3-540-44668-0_95

DO - 10.1007/3-540-44668-0_95

M3 - Conference contribution

SN - 3540424865

SP - 684

EP - 691

BT - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

PB - Springer [email protected]

ER -

Unsupervised learning in LSTM recurrent neural networks

Abstract

Access to Document

Other files and links

Fingerprint

Cite this