On learning soccer strategies

Rafał Sałustowicz, Marco Wiering, Jürgen Schmidhuber

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Scopus citations


We use simulated soccer to study multiagent learning. Each team's players (agents) share action set and policy but may behave differently due to position-dependent inputs. AU agents making up a team are rewarded or punished collectively in case of goals. We conduct simulations with varying team sizes, and compare two learning algorithms: TD-Q learning with linear neural networks (TD-Q) and Probabilistic Incremental Program Evolution (PIPE). TD-Q is based on evaluation functions (EFs) mapping input/action pairs to expected reward, while PIPE searches policy space directly. PIPE uses an adaptive probability distribution to synthesize programs that calculate action probabilities from current inputs. Our results show that TD-Q has difficulties to learn appropriate shared EFs. PIPE, however, does not depend on EFs and finds good policies faster and more reliably.
Original languageEnglish (US)
Title of host publicationLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
PublisherSpringer Verlag
Number of pages6
ISBN (Print)3540636315
StatePublished - Jan 1 1997
Externally publishedYes


Dive into the research topics of 'On learning soccer strategies'. Together they form a unique fingerprint.

Cite this