Online evolution of deep convolutional network for vision-based reinforcement learning

Jan Koutník, Jürgen Schmidhuber, Faustino Gomez

Research output: Chapter in Book/Report/Conference proceedingConference contribution

20 Scopus citations


Dealing with high-dimensional input spaces, like visual input, is a challenging task for reinforcement learning (RL). Neuroevolution (NE), used for continuous RL problems, has to either reduce the problem dimensionality by (1) compressing the representation of the neural network controllers or (2) employing a pre-processor (compressor) that transforms the high-dimensional raw inputs into low-dimensional features. In this paper we extend the approach in [16]. The Max-Pooling Convolutional Neural Network (MPCNN) compressor is evolved online, maximizing the distances between normalized feature vectors computed from the images collected by the recurrent neural network (RNN) controllers during their evaluation in the environment. These two interleaved evolutionary searches are used to find MPCNN compressors and RNN controllers that drive a race car in the TORCS racing simulator using only visual input. © 2014 Springer International Publishing Switzerland.
Original languageEnglish (US)
Title of host publicationLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Number of pages10
ISBN (Print)9783319088631
StatePublished - Jan 1 2014
Externally publishedYes


Dive into the research topics of 'Online evolution of deep convolutional network for vision-based reinforcement learning'. Together they form a unique fingerprint.

Cite this