Evolving memory cell structures for sequence learning

Justin Bayer; Daan Wierstra; Julian Togelius; Jürgen Schmidhuber

doi:10.1007/978-3-642-04277-5_76

Evolving memory cell structures for sequence learning

Justin Bayer, Daan Wierstra, Julian Togelius, Jürgen Schmidhuber

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

78 Scopus citations

Abstract

Long Short-Term Memory (LSTM) is one of the best recent supervised sequence learning methods. Using gradient descent, it trains memory cells represented as differentiable computational graph structures. Interestingly, LSTM's cell structure seems somewhat arbitrary. In this paper we optimize its computational structure using a multi-objective evolutionary algorithm. The fitness function reflects the structure's usefulness for learning various formal languages. The evolved cells help to understand crucial features that aid sequence learning. © 2009 Springer Berlin Heidelberg.

Original language	English (US)
Title of host publication	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Pages	755-764
Number of pages	10
DOIs	https://doi.org/10.1007/978-3-642-04277-5_76
State	Published - Nov 27 2009
Externally published	Yes

ASJC Scopus subject areas

Theoretical Computer Science
General Computer Science

Access to Document

10.1007/978-3-642-04277-5_76

Cite this

@inproceedings{ee3a24c4cf474284859f54976bb7bbeb,

title = "Evolving memory cell structures for sequence learning",

abstract = "Long Short-Term Memory (LSTM) is one of the best recent supervised sequence learning methods. Using gradient descent, it trains memory cells represented as differentiable computational graph structures. Interestingly, LSTM's cell structure seems somewhat arbitrary. In this paper we optimize its computational structure using a multi-objective evolutionary algorithm. The fitness function reflects the structure's usefulness for learning various formal languages. The evolved cells help to understand crucial features that aid sequence learning. {\textcopyright} 2009 Springer Berlin Heidelberg.",

author = "Justin Bayer and Daan Wierstra and Julian Togelius and J{\"u}rgen Schmidhuber",

note = "Generated from Scopus record by KAUST IRTS on 2022-09-14",

year = "2009",

month = nov,

day = "27",

doi = "10.1007/978-3-642-04277-5_76",

language = "English (US)",

isbn = "3642042767",

pages = "755--764",

booktitle = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

}

TY - GEN

T1 - Evolving memory cell structures for sequence learning

AU - Bayer, Justin

AU - Wierstra, Daan

AU - Togelius, Julian

AU - Schmidhuber, Jürgen

N1 - Generated from Scopus record by KAUST IRTS on 2022-09-14

PY - 2009/11/27

Y1 - 2009/11/27

N2 - Long Short-Term Memory (LSTM) is one of the best recent supervised sequence learning methods. Using gradient descent, it trains memory cells represented as differentiable computational graph structures. Interestingly, LSTM's cell structure seems somewhat arbitrary. In this paper we optimize its computational structure using a multi-objective evolutionary algorithm. The fitness function reflects the structure's usefulness for learning various formal languages. The evolved cells help to understand crucial features that aid sequence learning. © 2009 Springer Berlin Heidelberg.

AB - Long Short-Term Memory (LSTM) is one of the best recent supervised sequence learning methods. Using gradient descent, it trains memory cells represented as differentiable computational graph structures. Interestingly, LSTM's cell structure seems somewhat arbitrary. In this paper we optimize its computational structure using a multi-objective evolutionary algorithm. The fitness function reflects the structure's usefulness for learning various formal languages. The evolved cells help to understand crucial features that aid sequence learning. © 2009 Springer Berlin Heidelberg.

UR - http://link.springer.com/10.1007/978-3-642-04277-5_76

UR - http://www.scopus.com/inward/record.url?scp=70450190492&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-04277-5_76

DO - 10.1007/978-3-642-04277-5_76

M3 - Conference contribution

SN - 3642042767

SP - 755

EP - 764

BT - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

ER -

Evolving memory cell structures for sequence learning

Abstract

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this