This volume brings together recent theoretical work in Learning Classifier Systems (LCS), which is a Machine Learning technique combining Genetic Algorithms and Reinforcement Learning. It includes self-contained background chapters on related fields (reinforcement learning and evolutionary computation) tailored for a classifier systems audience and written by acknowledged authorities in their area - as well as a relevant historical original work by John Holland.

The Distinguished Dissertations series is published on behalf of the Conference of Professors and Heads of Computing and the British Computer Society, who annually select the best British PhD dissertations in computer science for publication. The dissertations are selected on behalf of the CPHC by a panel of eight academics. Each dissertation chosen makes a noteworthy contribution to the subject and reaches a high standard of exposition, placing all results clearly in the context of computer science as a whole. In this way computer scientists with significantly different interests are able to grasp the essentials - or even find a means of entry - to an unfamiliar research topic. Machine learning promises both to create machine intelligence and to shed light on natural intelligence. A fundamental issue for either endevour is that of credit assignment, which we can pose as follows: how can we credit individual components of a complex adaptive system for their often subtle effects on the world? For example, in a game of chess, how did each move (and the reasoning behind it) contribute to the outcome? This text studies aspects of credit assignment in learning classifier systems, which combine evolutionary algorithms with reinforcement learning methods to address a range of tasks from pattern classification to stochastic control to simulation of learning in animals. Credit assignment in classifier systems is complicated by two features: 1) their components are frequently modified by evolutionary search, and 2) components tend to interact. Classifier systems are re-examined from first principles and the result is, primarily, a formalization of learning in these systems, and a body of theoryrelating types of classifier systems, learning tasks, and credit assignment pathologies. Most significantly, it is shown that both of the main approaches have difficulties with certain tasks, which the other type does not.

This volume brings together recent theoretical work in Learning Classifier Systems (LCS), which is a Machine Learning technique combining Genetic Algorithms and Reinforcement Learning. It includes self-contained background chapters on related fields (reinforcement learning and evolutionary computation) tailored for a classifier systems audience and written by acknowledged authorities in their area - as well as a relevant historical original work by John Holland.

This volume includes extended and revised versions of the papers presented at the 9th and 10th International Workshops on Learning Classi?er Systems (IWLCS 2006 and IWLCS 2007). Both workshops were held in association with theGeneticandEvolutionaryComputationConference(GECCO).IWLCS2006 was held on July 8th, 2006, in Seattle, USA, during GECCO 2006.IWLCS 2007 was held on July 8th, 2007, in London, UK, during GECCO 2007. The IWLCS is the annual meeting of researchers wishing to discuss recent developments in learning classi?er systems (LCS). At the last IWLCS, the LCS researchers commemorated the 10th anniversary of the workshop and ackno- edged the contribution of Stewart Wilson to the ?eld. Following his proposal of the XCS classi?er system in 1995, research on LCS was reactivated leading to signi?cant contributions and promising perspectives. The annual IWLCS wo- shops are the proof of this fruitful research. We include an invited paper from Stewart Wilson. We greatly appreciate his contribution to the volume. The contents of this book are as follows. First, Bacardit, Bernado -Mansilla and Butz review LCS research over the past ten years and point out new ch- lenges and open issues in the LCS ?eld. Next, papers investigating knowledge representations are presented. Lanzi et al. analyze the evolution of XCS with symbolic representations using a novel method that identi?es useful substr- tures and tracks the emergence of optimal solutions. Ioannides and Browne investigate the scaling of LCSs using ternary and symbolic representations."

This book constitutes the thoroughly refereed joint post-proceedings of 3 consecutive International Workshops on Learning Classifier Systems that took place in Chicago, IL, USA in July 2003, in Seattle, WA, USA in June 2004, and in Washington, DC, USA in June 2005 - all hosted by the Genetic and Evolutionary Computation Conference, GECCO.

The 22 revised full papers presented were carefully reviewed and selected from the workshop contributions. The papers are organized in topical sections on knowledge representation, mechanisms, new directions, as well as application-oriented research and tools. The topics range from theoretical analysis of mechanisms to practical consideration for successful application of such techniques to everday datamining tasks.

Classifier systems are an intriguing approach to a broad range of machine learning problems, based on automated generation and evaluation of condi tion/action rules. Inreinforcement learning tasks they simultaneously address the two major problems of learning a policy and generalising over it (and re lated objects, such as value functions). Despite over 20 years of research, however, classifier systems have met with mixed success, for reasons which were often unclear. Finally, in 1995 Stewart Wilson claimed a long-awaited breakthrough with his XCS system, which differs from earlier classifier sys tems in a number of respects, the most significant of which is the way in which it calculates the value of rules for use by the rule generation system. Specifically, XCS (like most classifiersystems) employs a genetic algorithm for rule generation, and the way in whichit calculates rule fitness differsfrom earlier systems. Wilson described XCS as an accuracy-based classifiersystem and earlier systems as strength-based. The two differin that in strength-based systems the fitness of a rule is proportional to the return (reward/payoff) it receives, whereas in XCS it is a function of the accuracy with which return is predicted. The difference is thus one of credit assignment, that is, of how a rule's contribution to the system's performance is estimated. XCS is a Q learning system; in fact, it is a proper generalisation of tabular Q-learning, in which rules aggregate states and actions. In XCS, as in other Q-learners, Q-valuesare used to weightaction selection."