The 5th International Workshop on Learning Classi?er Systems (IWLCS2002) was held September 7-8, 2002, in Granada, Spain, during the 7th International Conference on Parallel Problem Solving from Nature (PPSN VII). We have included in this volume revised and extended versions of the papers presented at the workshop. In the ?rst paper, Browne introduces a new model of learning classi?er system, iLCS, and tests it on the Wisconsin Breast Cancer classi?cation problem. Dixon et al. present an algorithm for reducing the solutions evolved by the classi?er system XCS, so as to produce a small set of readily understandable rules. Enee and Barbaroux take a close look at Pittsburgh-style classi?er systems, focusing on the multi-agent problem known as El-farol. Holmes and Bilker investigate the effect that various types of missing data have on the classi?cation performance of learning classi?er systems. The two papers by Kovacs deal with an important theoretical issue in learning classi?er systems: the use of accuracy-based ?tness as opposed to the more traditional strength-based ?tness. In the ?rst paper, Kovacs introduces a strength-based version of XCS, called SB-XCS. The original XCS and the new SB-XCS are compared in the second paper, where - vacs discusses the different classes of solutions that XCS and SB-XCS tend to evolve.

Algorithmic learning theory is mathematics about computer programs which learn from experience. This involves considerable interaction between various mathematical disciplines including theory of computation, statistics, and c- binatorics. There is also considerable interaction with the practical, empirical ?elds of machine and statistical learning in which a principal aim is to predict, from past data about phenomena, useful features of future data from the same phenomena. The papers in this volume cover a broad range of topics of current research in the ?eld of algorithmic learning theory. We have divided the 29 technical, contributed papers in this volume into eight categories (corresponding to eight sessions) re?ecting this broad range. The categories featured are Inductive Inf- ence, Approximate Optimization Algorithms, Online Sequence Prediction, S- tistical Analysis of Unlabeled Data, PAC Learning & Boosting, Statistical - pervisedLearning, LogicBasedLearning, andQuery&ReinforcementLearning. Below we give a brief overview of the ?eld, placing each of these topics in the general context of the ?eld. Formal models of automated learning re?ect various facets of the wide range of activities that can be viewed as learning. A ?rst dichotomy is between viewing learning as an inde?nite process and viewing it as a ?nite activity with a de?ned termination. Inductive Inference models focus on inde?nite learning processes, requiring only eventual success of the learner to converge to a satisfactory conclusion.

This volume contains the papers presented at the 14th Annual Conference on Algorithmic Learning Theory (ALT 2003), which was held in Sapporo (Japan) duringOctober17-19,2003. Themainobjectiveoftheconferencewastoprovide an interdisciplinary forum for discussing the theoretical foundations of machine learning as well as their relevance to practical applications. The conference was co-locatedwiththe6thInternationalConferenceonDiscoveryScience(DS2003). The volume includes 19 technical contributions that were selected by the program committee from 37 submissions. It also contains the ALT 2003 invited talks presented by Naftali Tishby (Hebrew University, Israel) on "E?cient Data Representations that Preserve Information," by Thomas Zeugmann (University of Lub ] eck, Germany) on "Can Learning in the Limit be Done E?ciently?," and by Genshiro Kitagawa (Institute of Statistical Mathematics, Japan) on "S- nal Extraction and Knowledge Discovery Based on Statistical Modeling" (joint invited talk with DS 2003). Furthermore, this volume includes abstracts of the invitedtalksforDS2003presentedbyThomasEiter(ViennaUniversityofTe- nology, Austria) on "Abduction and the Dualization Problem" and by Akihiko Takano (National Institute of Informatics, Japan) on "Association Computation for Information Access. " The complete versions of these papers were published in the DS 2003 proceedings (Lecture Notes in Arti?cial Intelligence Vol. 2843). ALT has been awarding theE. MarkGoldAward for the most outstanding paper by a student author since 1999. This year the award was given to Sandra Zilles for her paper "Intrinsic Complexity of Uniform Learning. " This conference was the 14th in a series of annual conferences established in 1990. ContinuationoftheALTseriesissupervisedbyitssteeringcommittee, c- sisting of: Thomas Zeugmann (Univ."

The proceedings of ECML/PKDD2003 are published in two volumes: the P- ceedings of the 14th European Conference on Machine Learning (LNAI 2837) and the Proceedings of the 7th European Conference on Principles and Practice of Knowledge Discovery in Databases (LNAI 2838). The two conferences were held on September 22-26, 2003 in Cavtat, a small tourist town in the vicinity of Dubrovnik, Croatia. As machine learning and knowledge discovery are two highly related ?elds, theco-locationofbothconferencesisbene?cialforbothresearchcommunities.In Cavtat, ECML and PKDD were co-located for the third time in a row, following the successful co-location of the two European conferences in Freiburg (2001) and Helsinki (2002). The co-location of ECML2003 and PKDD2003 resulted in a joint program for the two conferences, including paper presentations, invited talks, tutorials, and workshops. Out of 332 submitted papers, 40 were accepted for publication in the ECML2003proceedings, and40wereacceptedforpublicationinthePKDD2003 proceedings. All the submitted papers were reviewed by three referees. In ad- tion to submitted papers, the conference program consisted of four invited talks, four tutorials, seven workshops, two tutorials combined with a workshop, and a discovery challenge

This volume contains papers presented at the 17th Annual Conference on Le- ning Theory (previously known as the Conference on Computational Learning Theory) held in Ban?, Canada from July 1 to 4, 2004. The technical program contained 43 papers selected from 107 submissions, 3 open problems selected from among 6 contributed, and 3 invited lectures. The invited lectures were given by Michael Kearns on 'Game Theory, Automated Trading and Social Networks', Moses Charikar on 'Algorithmic Aspects of - nite Metric Spaces', and Stephen Boyd on 'Convex Optimization, Semide?nite Programming, and Recent Applications'. These papers were not included in this volume. The Mark Fulk Award is presented annually for the best paper co-authored by a student. Thisyear theMark Fulk award wassupplemented with two further awards funded by the Machine Learning Journal and the National Information Communication Technology Centre, Australia (NICTA). We were therefore able toselectthreestudentpapersforprizes.ThestudentsselectedwereMagalieF- montforthesingle-authorpaper"ModelSelectionbyBootstrapPenalizationfor Classi?cation", Daniel Reidenbach for the single-author paper "On the Lear- bility of E-Pattern Languages over Small Alphabets", and Ran Gilad-Bachrach for the paper "Bayes and Tukey Meet at the Center Point" (co-authored with Amir Navot and Naftali Tishby).

Computational learning theory is one of the first attempts to construct a mathematical theory of a cognitive process. It has been a field of much interest and rapid growth in recent years. This text provides a framework for studying a variety of algorithmic processes, such as those currently in use for training artificial neural networks. The authors concentrate on an approximate model for learning and gradually develop the ideas of efficiency considerations. Finally, they consider applications of the theory to artificial neural networks. An abundance of exercises and an extensive list of references round out the text. This volume provides a comprehensive review of the topic, including information drawn from logic, probability, and complexity theory. It forms a solid introduction to the theory of comptutational learning suitable for a broad spectrum of graduate students from theoretical computer science to mathematics.

For some time, all branches of the military have used a wide range of sensors to provide data for many purposes, including surveillance, reconnoitring, target detection and battle damage assessment. Many nations have also attempted to utilise these sensors for civilian applications, such as crop monitoring, agricultural disease tracking, environmental diagnostics, cartography, ocean temperature profiling, urban planning, and the characterisation of the Ozone Hole above Antarctica. The recent convergence of several important technologies has made possible new, advanced, high performance, sensor based applications relying on the near-simultaneous fusion of data from an ensemble of different types of sensors. The book examines the underlying principles of sensor operation and data fusion, the techniques and technologies that enable the process, including the operation of 'fusion engines'. Fundamental theory and the enabling technologies of data fusion are presented in a systematic and accessible manner. Applications are discussed in the areas of medicine, meteorology, BDA and targeting, transportation, cartography, the environment, agriculture, and manufacturing and process control.

This book constitutes the refereed proceedings of the 15th Annual Conference on Computational Learning Theory, COLT 2002, held in Sydney, Australia, in July 2002.The 26 revised full papers presented were carefully reviewed and selected from 55 submissions. The papers are organized in topical sections on statistical learning theory, online learning, inductive inference, PAC learning, boosting, and other learning paradigms.

This book constitutes the refereed proceedings of the Third International Workshop on Multiple Classifier Systems, MCS 2002, held in Cagliari, Italy, in June 2002.The 29 revised full papers presented together with three invited papers were carefully reviewed and selected for inclusion in the volume. The papers are organized in topical sections on bagging and boosting, ensemble learning and neural networks, design methodologies, combination strategies, analysis and performance evaluation, and applications.

This volume contains papers presented at the joint 16th Annual Conference on Learning Theory (COLT) and the 7th Annual Workshop on Kernel Machines, heldinWashington, DC, USA, duringAugust24 27,2003.COLT, whichrecently merged with EuroCOLT, has traditionally been a meeting place for learning theorists. We hope that COLT will bene't from the collocation with the annual workshoponkernelmachines, formerlyheldasaNIPSpostconferenceworkshop. The technical program contained 47 papers selected from 92 submissions. All 47paperswerepresentedasposters;22ofthepaperswereadditionallypresented astalks.Therewerealsotwotargetareaswithinvitedcontributions.Incompu- tional game theory, atutorialentitled LearningTopicsinGame-TheoreticDe- sionMaking wasgivenbyMichaelLittman, andaninvitedpaperon AGeneral Class of No-Regret Learning Algorithms and Game-Theoretic Equilibria was contributed by Amy Greenwald. In natural language processing, a tutorial on Machine Learning Methods in Natural Language Processing was presented by Michael Collins, followed by two invited talks, Learning from Uncertain Data by Mehryar Mohri and Learning and Parsing Stochastic Uni?cation- Based Grammars by Mark Johnson. In addition to the accepted papers and invited presentations, we solicited short open problems that were reviewed and included in the proceedings. We hope that reviewed open problems might become a new tradition for COLT. Our goal was to select simple signature problems whose solutions are likely to inspire further research. For some of the problems the authors o?ered monetary rewards. Yoav Freund acted as the open problem area chair. The open problems were presented as posters at the conference."

This book constitutes the refereed proceedings of the Third International Conference on Machine Learning and Data Mining in Pattern Recognition, MLDM 2003, held in Leipzig, Germany, in July 2003. The 33 revised full papers presented together with two invited papers were carefully reviewed and selected from 75 submissions. The papers are organized in topical sections on decision trees; clustering and its applications; support vector machines; case-based reasoning; classification, retrieval, and feature Learning; discovery of frequent or sequential patterns; Bayesian models and methods; association rule mining; and applications.

This volume contains the papers presented at the 12th Annual Conference on Algorithmic Learning Theory (ALT 2001), which was held in Washington DC, USA, during November 25-28, 2001. The main objective of the conference is to provide an inter-disciplinary forum for the discussion of theoretical foundations of machine learning, as well as their relevance to practical applications. The conference was co-located with the Fourth International Conference on Discovery Science (DS 2001). The volume includes 21 contributed papers. These papers were selected by the program committee from 42 submissions based on clarity, signi?cance, o- ginality, and relevance to theory and practice of machine learning. Additionally, the volume contains the invited talks of ALT 2001 presented by Dana Angluin of Yale University, USA, Paul R. Cohen of the University of Massachusetts at Amherst, USA, and the joint invited talk for ALT 2001 and DS 2001 presented by Setsuo Arikawa of Kyushu University, Japan. Furthermore, this volume includes abstracts of the invited talks for DS 2001 presented by Lindley Darden and Ben Shneiderman both of the University of Maryland at College Park, USA. The complete versions of these papers are published in the DS 2001 proceedings (Lecture Notes in Arti?cial Intelligence Vol. 2226).

This book constitutes the refereed preceedings of the 13th European Conference on Machine Learning, ECML 2002, held in Helsinki, Finland in August 2002.The 41 revised full papers presented together with 4 invited contributions were carefully reviewed and selected from numerous submissions. Among the topics covered are computational discovery, search strategies, Classification, support vector machines, kernel methods, rule induction, linear learning, decision tree learning, boosting, collaborative learning, statistical learning, clustering, instance-based learning, reinforcement learning, multiagent learning, multirelational learning, Markov decision processes, active learning, etc.

This book presents the thoroughly refereed post-proceedings of the Second International Workshop on Intelligent Memory Systems, IMS 2000, held in Cambridge, MA, USA, in November 2000.The nine revised full papers and six poster papers presented were carefully reviewed and selected from 28 submissions. The papers cover a wide range of topics in intelligent memory computing; they are organized in topical sections on memory technology, processor and memory architecture, applications and operating systems, and compiler technology.

This book constitutes the refereed proceedings of the 11th International Conference on Inductive Logic Programming, ILP 2001, held in Strasbourg, France in September 2001.The 21 revised full papers presented were carefully reviewed and selected from 37 submissions. Among the topics addressed are data mining issues for multi-relational databases, supervised learning, inductive inference, Bayesian reasoning, learning refinement operators, neural network learning, constraint satisfaction, genetic algorithms, statistical machine learning, transductive inference, etc.

This book constitutes the thoroughly refereed post-proceedings of the 4th International Workshop on Learning Classifier Systems, IWLCS 2001, held in San Francisco, CA, USA, in July 2001.The 12 revised full papers presented together with a special paper on a formal description of ACS have gone through two rounds of reviewing and improvement. The first part of the book is devoted to theoretical issues of learning classifier systems including the influence of exploration strategy, self-adaptive classifier systems, and the use of classifier systems for social simulation. The second part is devoted to applications in various fields such as data mining, stock trading, and power distributionn networks.

Learning classi er systems are rule-based systems that exploit evolutionary c- putation and reinforcement learning to solve di cult problems. They were - troduced in 1978 by John H. Holland, the father of genetic algorithms, and since then they have been applied to domains as diverse as autonomous robotics, trading agents, and data mining. At the Second International Workshop on Learning Classi er Systems (IWLCS 99), held July 13, 1999, in Orlando, Florida, active researchers reported on the then current state of learning classi er system research and highlighted some of the most promising research directions. The most interesting contri- tions to the meeting are included in the book Learning Classi er Systems: From Foundations to Applications, published as LNAI 1813 by Springer-Verlag. The following year, the Third International Workshop on Learning Classi er Systems (IWLCS 2000), held September 15{16 in Paris, gave participants the opportunity to discuss further advances in learning classi er systems. We have included in this volume revised and extended versions of thirteen of the papers presented at the workshop.

This book constitutes the refereed proceedings of the 12th European Conference on Machine Learning, ECML 2001, held in Freiburg, Germany, in September 2001.The 50 revised full papers presented together with four invited contributions were carefully reviewed and selected from a total of 140 submissions. Among the topics covered are classifier systems, naive-Bayes classification, rule learning, decision tree-based classification, Web mining, equation discovery, inductive logic programming, text categorization, agent learning, backpropagation, reinforcement learning, sequence prediction, sequential decisions, classification learning, sampling, and semi-supervised learning.

In recent years machine learning has made its way from artificial intelligence into areas of administration, commerce, and industry. Data mining is perhaps the most widely known demonstration of this migration, complemented by less publicized applications of machine learning like adaptive systems in industry, financial prediction, medical diagnosis and the construction of user profiles for Web browsers.This book presents the capabilities of machine learning methods and ideas on how these methods could be used to solve real-world problems. The first ten chapters assess the current state of the art of machine learning, from symbolic concept learning and conceptual clustering to case-based reasoning, neural networks, and genetic algorithms. The second part introduces the reader to innovative applications of ML techniques in fields such as data mining, knowledge discovery, human language technology, user modeling, data analysis, discovery science, agent technology, finance, etc.

Driven by the requirements of a large number of practical and commercially - portant applications, the last decade has witnessed considerable advances in p- tern recognition. Better understanding of the design issues and new paradigms, such as the Support Vector Machine, have contributed to the development of - proved methods of pattern classi cation. However, while any performance gains are welcome, and often extremely signi cant from the practical point of view, it is increasingly more challenging to reach the point of perfection as de ned by the theoretical optimality of decision making in a given decision framework. The asymptoticity of gains that can be made for a single classi er is a re?- tion of the fact that any particular design, regardless of how good it is, simply provides just one estimate of the optimal decision rule. This observation has motivated the recent interest in Multiple Classi er Systems , which aim to make use of several designs jointly to obtain a better estimate of the optimal decision boundary and thus improve the system performance. This volume contains the proceedings of the international workshop on Multiple Classi er Systems held at Robinson College, Cambridge, United Kingdom (July 2{4, 2001), which was organized to provide a forum for researchers in this subject area to exchange views and report their latest results.

This book constitutes the refereed proceedings of the Third International Workshop on Ant Algorithms, ANTS 2002, held in Brussels, Belgium in September 2002.The 17 revised full papers, 11 short papers, and extended poster abstracts presented were carefully reviewed and selected from 52 submissions. The papers deal with theoretical and foundational aspects and a variety of new variants of ant algorithms as well as with a broad variety of optimization applications in networking and operations research. All in all, this book presents the state of the art in research and development in the emerging field of ant algorithms

This book constitutes the refereed proceedings of the Third International Conference on Intelligent Data Engineering and Automated Learning, IDEAL 2002, held in Manchester, UK in August 2002.The 89 revised papers presented were carefully reviewed and selected from more than 150 submissions. The book offers topical sections on data mining, knowledge engineering, text and document processing, internet applications, agent technology, autonomous mining, financial engineering, bioinformatics, learning systems, and pattern recognition.

The practical application of Genetic Algorithms to the solution of engineering problems, is rapidly becoming an established approach in the fields of control and signal processing. This book provides comprehensive coverage of the techniques involved, describing the intrinsic characteristics, advantages and constraints of genetic algorithms, as well as discussing genetic operations such as crossover, mutation and reinsertion. In addition, the principle of multiobjective optimization and computing parallelism are discussed. These features are fully illustrated by real-world applications. Also described is a newly proposed and unique hierarchical genetic algorithm designed to address the problems in determining system topology. For added value, a 3.5" disk accompanies the book, that provides the reader with an interactive Genetic Algorithms demonstration programme.

This volume contains all the papers presented at the Eleventh International C- ference on Algorithmic Learning Theory (ALT 2000) held at Coogee Holiday Inn, Sydney, Australia,11-13 December 2000. The conference was sponsored by the School of Computer Science and Engineering, University of New South Wales, and supported by the IFIP Working Group 1.4 on Computational Learning T- ory and the Computer Science Association (CSA) of Australia. In response to the call for papers 39 submissions were received on all aspects of algorithmic learning theory. Out of these 22 papers were accepted for p- sentation by the program committee. In addition, there were three invited talks by William Cohen (Whizbang Labs), Tom Dietterich (Oregon State Univeristy), and Osamu Watanabe (Tokyo Institute of Technology). This year's conference is the last in the millenium and eleventh overall in the ALT series. The ?rst ALT workshop was held in Tokyo in 1990. It was merged with the workshop on Analogical and Inductive Inference in 1994. The conf- ence focuses on all areas related to algorithmic learning theory, including (but not limited to) the design and analysis of learning algorithms, the theory of machine learning, computational logic of/for machine discovery, inductive inf- ence, learning via queries, new learning models, scienti?c discovery, learning by analogy, arti?cial and biological neural networks, pattern recognition, statistical learning, Bayesian/MDL estimation, inductive logic programming, data m- ing and knowledge discovery, and application of learning to biological sequence analysis. In the current conference there were papers from a variety of the above areas, refelecting both the theoretical as well as practical aspec

This volumecontains the papers presentedatthe SeventhInternationalC- ference on Logicfor Programmingand Automated Reasoning (LPAR 2000)held onReunionIsland, France,6 10November2000, followedbythe ReunionWo- shop on Implementation of Logic. Sixty-?ve papers were submitted to LPAR 2000 of which twenty-six papers were accepted. Submissions by the program committee members were not - lowed. There was a special category of experimental papers intended to describe implementations of systems, to report experiments with implemented systems, orto compareimplementedsystems.Eachof thesubmissionswasreviewedbyat least three program committee members and an electronic program committee meeting was held via the Internet. In addition to the refereed papers, this volume contains full papers by two of the four invited speakers, Georg Gottlob and Micha] el Rusinowitch, along with an extended abstract of Bruno Courcelle s invited lecture and an abstract of Erich Gr] adel s invited lecture. WewouldliketothankthemanypeoplewhohavemadeLPAR2000possible. We are grateful to the following groups and individuals: the program and or- nizing committees; the additional referees; the local arrangements chair Teodor Knapik; PascalManoury, who was in chargeof accommodation; Konstantin - rovin, whomaintainedthe programcommittee Webpage;andBillMcCune, who implemented the program committee management software."