The European Conference on Machine Learning (ECML) and the European Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD) were jointly organized this year for the ?fth time in a row, after some years of mutual independence before. After Freiburg (2001), Helsinki (2002), Cavtat (2003) and Pisa (2004), Porto received the 16th edition of ECML and the 9th PKDD in October 3-7. Having the two conferences together seems to be working well: 585 di?erent paper submissions were received for both events, which maintains the high s- mission standard of last year. Of these, 335 were submitted to ECML only, 220 to PKDD only and 30 to both. Such a high volume of scienti?c work required a tremendous e?ort from Area Chairs, Program Committee members and some additional reviewers. On average, PC members had 10 papers to evaluate, and Area Chairs had 25 papers to decide upon. We managed to have 3 highly qua- ?edindependentreviewsperpaper(withveryfewexceptions)andoneadditional overall input from one of the Area Chairs. After the authors' responses and the online discussions for many of the papers, we arrived at the ?nal selection of 40 regular papers for ECML and 35 for PKDD. Besides these, 32 others were accepted as short papers for ECML and 35 for PKDD. This represents a joint acceptance rate of around 13% for regular papers and 25% overall. We thank all involved for all the e?ort with reviewing and selection of papers. Besidesthecoretechnicalprogram, ECMLandPKDDhad6invitedspeakers, 10 workshops, 8 tutorials and a Knowledge Discovery Challenge.

This book constitutes the thoroughly refereed post-proceedings of the First VLDB 2006 International Workshop on Data Mining and Bioinformatics, VDMB 2006, held in Seoul, Korea in September 2006 in conjunction with VLDB 2006. The 15 revised full papers cover various topics in the areas of microarray data analysis, bioinformatics system and text retrieval, application of gene expression data, and sequence analysis.

This book constitutes the thoroughly refereed post-proceedings of the Third International Workshop on Machine Learning for Multimodal Interaction, MLMI 2006, held in Bethseda, MD, USA, in May 2006.

The 39 revised full papers presented together with 1 invited paper were carefully selected during two rounds of reviewing and revision. The papers are organized in topical sections on multimodal processing, image and video processing, HCI and applications, discourse and dialogue, speech and audio processing, and NIST meeting recognition evaluation.

This volume contains papers presented at the Eighteenth Annual Conference on Learning Theory (previously known as the Conference on Computational Learning Theory) held in Bertinoro, Italy from June 27 to 30, 2005. The technical program contained 45 papers selected from 120 submissions, 3 open problems selected from among 5 contributed, and 2 invited lectures. The invited lectures were given by Sergiu Hart on "Uncoupled Dynamics and Nash Equilibrium", and by Satinder Singh on "Rethinking State, Action, and Reward in Reinforcement Learning". These papers were not included in this volume. The Mark Fulk Award is presented annually for the best paper co-authored by a student. The student selected this year was Hadi Salmasian for the paper titled "The Spectral Method for General Mixture Models" co-authored with Ravindran Kannan and Santosh Vempala. The number of papers submitted to COLT this year was exceptionally high. In addition to the classical COLT topics, we found an increase in the number of submissions related to novel classi?cation scenarios such as ranking. This - crease re?ects a healthy shift towards more structured classi?cation problems, which are becoming increasingly relevant to practitioners.

This book constitutes the refereed proceedings of the 17th International Conference on Algorithmic Learning Theory, ALT 2006, held in Barcelona, Spain in October 2006, colocated with the 9th International Conference on Discovery Science, DS 2006.

The 24 revised full papers presented together with the abstracts of five invited papers were carefully reviewed and selected from 53 submissions. The papers are dedicated to the theoretical foundations of machine learning.

Volume III of the Transactions on Rough Sets (TRS) introduces advances in the theory and application of rough sets. These advances have far-reaching impli- tions in a number of researchareas such as approximate reasoning, bioinform- ics, computerscience, datamining, engineering(especially, computerengineering and signal analysis), intelligent systems, knowledge discovery, pattern recog- tion, machineintelligence, andvariousformsoflearning. This volumerevealsthe vigor, breadth and depth in research either directly or indirectly related to the rough sets theory introduced by Prof. Zdzis law Pawlak more than three decades ago. Evidence of this can be found in the seminal paper on data mining by Prof. Pawlak included in this volume. In addition, there are eight papers on the theory and application of rough sets as well as a presentation of a new version of the Rough Set Exploration System (RSES) tool set and an introduction to the Rough Set Database System (RSDS). Prof. Pawlak has contributed a pioneering paper on data mining to this v- ume. In this paper, it is shown that information ?ow in a ?ow graph is governed by Bayes' rule with a deterministic rather than a probabilistic interpretation. A cardinal feature of this paper is that it is self-contained inasmuch as it not only introduces a new viewof information?owbut alsoprovidesanintroduction to the basic concepts of ?ow graphs. The representation of information ?ow - troduced in this paper makes it possible to study di?erent relationships in data and establishes a basis for a new mathematical tool for data mining. Inadditionto thepaperbyProf

This book constitutes the refereed post-proceedings of the First PASCAL Machine Learning Challenges Workshop, MLCW 2005. 25 papers address three challenges: finding an assessment base on the uncertainty of predictions using classical statistics, Bayesian inference, and statistical learning theory; second, recognizing objects from a number of visual object classes in realistic scenes; third, recognizing textual entailment addresses semantic analysis of language to form a generic framework for applied semantic inference in text understanding.

With its intuitive yet rigorous approach to machine learning, this text provides students with the fundamental knowledge and practical tools needed to conduct research and build data-driven products. The authors prioritize geometric intuition and algorithmic thinking, and include detail on all the essential mathematical prerequisites, to offer a fresh and accessible way to learn. Practical applications are emphasized, with examples from disciplines including computer vision, natural language processing, economics, neuroscience, recommender systems, physics, and biology. Over 300 color illustrations are included and have been meticulously designed to enable an intuitive grasp of technical concepts, and over 100 in-depth coding exercises (in Python) provide a real understanding of crucial machine learning algorithms. A suite of online resources including sample code, data sets, interactive lecture slides, and a solutions manual are provided online, making this an ideal text both for graduate courses on machine learning and for individual reference and self-study.

This important work describes recent theoretical advances in the study of artificial neural networks. It explores probabilistic models of supervised learning problems, and addresses the key statistical and computational questions. Chapters survey research on pattern classification with binary-output networks, including a discussion of the relevance of the Vapnik Chervonenkis dimension, and of estimates of the dimension for several neural network models. In addition, Anthony and Bartlett develop a model of classification by real-output networks, and demonstrate the usefulness of classification with a "large margin." The authors explain the role of scale-sensitive versions of the Vapnik Chervonenkis dimension in large margin classification, and in real prediction. Key chapters also discuss the computational complexity of neural network learning, describing a variety of hardness results, and outlining two efficient, constructive learning algorithms. The book is self-contained and accessible to researchers and graduate students in computer science, engineering, and mathematics.

This book constitutes the refereed proceedings of the International Workshop on Data Mining for Biomedical Applications, BioDM 2006, held in Singapore in conjunction with the 10th Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD 2006). The 14 revised full papers presented together with one keynote talk were carefully reviewed and selected from 35 submissions. The papers are organized in topical sections

This collection of articles is devoted to fuzzy as well as rough set theories. Both theoriesarebasedonrigorousideas, methodsandtechniquesinlogic, mathem- ics, and computer science for treating problems for which approximate solutions are possible only, due to their inherent ambiguity, vagueness, incompleteness, etc. Vast areas of decision making, data mining, knowledge discovery in data, approximatereasoning, etc., aresuccessfully exploredusing methods workedout within fuzzy and rough paradigms. By the very nature of fuzzy and rough paradigms, outlined above, they are related to distinct logical schemes: it is well-known that rough sets are related to modal logicsS5andS4(Orl owska, E., Modal logics in the theory of infor- tion systems, Z. Math. Logik Grund. Math. 30, 1984, pp. 213 ?.; Vakarelov, D., Modal logics for knowledgerepresentationsystems, LNCS 363,1989, pp. 257?.) and to ?nitely-valued logics (Pagliani, P., Rough set theory and logic-algebraic structures. In Incomplete Information: Rough Set Analysis, Orlo wska, E., ed., Physica/Springer, 1998, pp. 109 ?.; Polkowski, L. A note on 3-valued rough logic accepting decision rules, Fundamenta Informaticae 61, to appear). Fuzzy sets are related to in?nitely-valued logics (fuzzy membership to degree r? 0,1]expressingtruthdegreer)(Goguen, J.A., Thelogicofinexactconcepts, Synthese18/19,1968-9, pp.325?.;Pavelka, J., OnfuzzylogicI, II, III, Z. Math. Logik Grund. Math. 25, 1979, pp. 45 ?., pp. 119 ?., pp. 454 ?.; Dubois, D., Prade, H., Possibility Theory, Plenum Press, 1988; Haj ek, P., Metamathematics of Fuzzy Logic, Kluw

The proceedings of ECML/PKDD 2004 are published in two separate, albeit - tertwined, volumes: theProceedingsofthe 15thEuropeanConferenceonMac- ne Learning (LNAI 3201) and the Proceedings of the 8th European Conferences on Principles and Practice of Knowledge Discovery in Databases (LNAI 3202). The two conferences were co-located in Pisa, Tuscany, Italy during September 20-24, 2004. It was the fourth time in a row that ECML and PKDD were co-located. - ter the successful co-locations in Freiburg (2001), Helsinki (2002), and Cavtat- Dubrovnik (2003), it became clear that researchersstrongly supported the or- nization of a major scienti?c event about machine learning and data mining in Europe. We are happy to provide some statistics about the conferences. 581 di?erent papers were submitted to ECML/PKDD (about a 75% increase over 2003); 280 weresubmittedtoECML2004only,194weresubmittedtoPKDD2004only, and 107weresubmitted to both.Aroundhalfofthe authorsforsubmitted papersare from outside Europe, which is a clear indicator of the increasing attractiveness of ECML/PKDD. The Program Committee members were deeply involved in what turned out to be a highly competitive selection process. We assigned each paper to 3 - viewers, deciding on the appropriate PC for papers submitted to both ECML and PKDD. As a result, ECML PC members reviewed 312 papers and PKDD PC members reviewed 269 papers. We accepted for publication regular papers (45 for ECML 2004 and 39 for PKDD 2004) and short papers that were as- ciated with poster presentations (6 for ECML 2004 and 9 for PKDD 2004). The globalacceptance ratewas14.5%for regular papers(17% if we include the short paper

We would like to present, with great pleasure, the ?rst volume of a new jo- nal, Transactions on Rough Sets. This journal, part of the new journal subline in the Springer-Verlag series Lecture Notes in Computer Science, is devoted to the entire spectrum of rough set related issues, starting from logical and ma- ematical foundations of rough sets, through all aspects of rough set theory and its applications, data mining, knowledge discovery and intelligent information processing, to relations between rough sets and other approaches to uncertainty, vagueness, and incompleteness, such as fuzzy sets, theory of evidence, etc. The ?rst, pioneering papers on rough sets, written by the originator of the idea, ProfessorZdzis lawPawlak, werepublishedintheearly1980s.Weareproud to dedicate this volume to our mentor, Professor Zdzis law Pawlak, who kindly enriched this volume with his contribution on philosophical, logical, and mat- matical foundations of roughset theory. In his paper Professor Pawlakshows all over again the underlying ideas of rough set theory as well as its relations with Bayes' theorem, con?ict analysis, ?ow graphs, decision networks, and decision rules.

The fusion of di?erent information sourcesis a persistent and intriguing issue. It hasbeenaddressedforcenturiesinvariousdisciplines, includingpoliticalscience, probability and statistics, system reliability assessment, computer science, and distributed detection in communications. Early seminal work on fusion was c- ried out by pioneers such as Laplace and von Neumann. More recently, research activities in information fusion have focused on pattern recognition. During the 1990s, classi?erfusionschemes, especiallyattheso-calleddecision-level, emerged under a plethora of di?erent names in various scienti?c communities, including machine learning, neural networks, pattern recognition, and statistics. The d- ferent nomenclatures introduced by these communities re?ected their di?erent perspectives and cultural backgrounds as well as the absence of common forums and the poor dissemination of the most important results. In 1999, the ?rst workshop on multiple classi?er systems was organized with the main goal of creating a common international forum to promote the diss- ination of the results achieved in the diverse communities and the adoption of a common terminology, thus giving the di?erent perspectives and cultural ba- grounds some concrete added value. After ?ve meetings of this workshop, there is strong evidence that signi?cant steps have been made towards this goal. - searchers from these diverse communities successfully participated in the wo- shops, and world experts presented surveys of the state of the art from the perspectives of their communities to aid cross-fertilizat

In this volume we present the accepted contributions for the 7th European C- ference on Genetic Programming (EuroGP 2004). The conference took place on 5 7 April 2004 in Portugal at the University of Coimbra, in the Department of Mathematics in Pra, ca Dom Dinis, located on the hill above the old town. EuroGP is a well-established conference and the sole one exclusively de- ted to Genetic Programming. Previous proceedings have all been published by Springer-Verlag in the LNCS series. EuroGP began as an international wor- hop in Paris, France in 1998 (14 15 April, LNCS 1391). Subsequently the wor- hop was held in G] oteborg, Sweden in 1999 (26 27 May, LNCS 1598) and then EuroGP became an annual conference: in 2000 in Edinburgh, UK (15 16 April, LNCS 1802), in 2001 at Lake Como, Italy (18 19 April, LNCS 2038), in 2002 in Kinsale, Ireland (3 5 April, LNCS 2278), and in 2003 in Colchester, UK (14 16 April, LNCS 2610). From the outset, there have always been specialized wor- hops, co-located with EuroGP, focusing on applications of evolutionary al- rithms (LNCS 1468, 1596, 1803, 2037, 2279, and 2611). This year the EvoCOP workshop on combinatorial optimization transformed itself into a conference in its own right, and the two conferences, together with the EvoWorkshops, EvoBIO, EvoIASP, EvoMUSART, EvoSTOC, EvoHOT, and EvoCOMNET, now form one of the largest events dedicated to Evolutionary Computation in Europe."

The 5th International Workshop on Learning Classi?er Systems (IWLCS2002) was held September 7-8, 2002, in Granada, Spain, during the 7th International Conference on Parallel Problem Solving from Nature (PPSN VII). We have included in this volume revised and extended versions of the papers presented at the workshop. In the ?rst paper, Browne introduces a new model of learning classi?er system, iLCS, and tests it on the Wisconsin Breast Cancer classi?cation problem. Dixon et al. present an algorithm for reducing the solutions evolved by the classi?er system XCS, so as to produce a small set of readily understandable rules. Enee and Barbaroux take a close look at Pittsburgh-style classi?er systems, focusing on the multi-agent problem known as El-farol. Holmes and Bilker investigate the effect that various types of missing data have on the classi?cation performance of learning classi?er systems. The two papers by Kovacs deal with an important theoretical issue in learning classi?er systems: the use of accuracy-based ?tness as opposed to the more traditional strength-based ?tness. In the ?rst paper, Kovacs introduces a strength-based version of XCS, called SB-XCS. The original XCS and the new SB-XCS are compared in the second paper, where - vacs discusses the different classes of solutions that XCS and SB-XCS tend to evolve.

Algorithmic learning theory is mathematics about computer programs which learn from experience. This involves considerable interaction between various mathematical disciplines including theory of computation, statistics, and c- binatorics. There is also considerable interaction with the practical, empirical ?elds of machine and statistical learning in which a principal aim is to predict, from past data about phenomena, useful features of future data from the same phenomena. The papers in this volume cover a broad range of topics of current research in the ?eld of algorithmic learning theory. We have divided the 29 technical, contributed papers in this volume into eight categories (corresponding to eight sessions) re?ecting this broad range. The categories featured are Inductive Inf- ence, Approximate Optimization Algorithms, Online Sequence Prediction, S- tistical Analysis of Unlabeled Data, PAC Learning & Boosting, Statistical - pervisedLearning, LogicBasedLearning, andQuery&ReinforcementLearning. Below we give a brief overview of the ?eld, placing each of these topics in the general context of the ?eld. Formal models of automated learning re?ect various facets of the wide range of activities that can be viewed as learning. A ?rst dichotomy is between viewing learning as an inde?nite process and viewing it as a ?nite activity with a de?ned termination. Inductive Inference models focus on inde?nite learning processes, requiring only eventual success of the learner to converge to a satisfactory conclusion.

This volume contains the papers presented at the 14th Annual Conference on Algorithmic Learning Theory (ALT 2003), which was held in Sapporo (Japan) duringOctober17-19,2003. Themainobjectiveoftheconferencewastoprovide an interdisciplinary forum for discussing the theoretical foundations of machine learning as well as their relevance to practical applications. The conference was co-locatedwiththe6thInternationalConferenceonDiscoveryScience(DS2003). The volume includes 19 technical contributions that were selected by the program committee from 37 submissions. It also contains the ALT 2003 invited talks presented by Naftali Tishby (Hebrew University, Israel) on "E?cient Data Representations that Preserve Information," by Thomas Zeugmann (University of Lub ] eck, Germany) on "Can Learning in the Limit be Done E?ciently?," and by Genshiro Kitagawa (Institute of Statistical Mathematics, Japan) on "S- nal Extraction and Knowledge Discovery Based on Statistical Modeling" (joint invited talk with DS 2003). Furthermore, this volume includes abstracts of the invitedtalksforDS2003presentedbyThomasEiter(ViennaUniversityofTe- nology, Austria) on "Abduction and the Dualization Problem" and by Akihiko Takano (National Institute of Informatics, Japan) on "Association Computation for Information Access. " The complete versions of these papers were published in the DS 2003 proceedings (Lecture Notes in Arti?cial Intelligence Vol. 2843). ALT has been awarding theE. MarkGoldAward for the most outstanding paper by a student author since 1999. This year the award was given to Sandra Zilles for her paper "Intrinsic Complexity of Uniform Learning. " This conference was the 14th in a series of annual conferences established in 1990. ContinuationoftheALTseriesissupervisedbyitssteeringcommittee, c- sisting of: Thomas Zeugmann (Univ."

The proceedings of ECML/PKDD2003 are published in two volumes: the P- ceedings of the 14th European Conference on Machine Learning (LNAI 2837) and the Proceedings of the 7th European Conference on Principles and Practice of Knowledge Discovery in Databases (LNAI 2838). The two conferences were held on September 22-26, 2003 in Cavtat, a small tourist town in the vicinity of Dubrovnik, Croatia. As machine learning and knowledge discovery are two highly related ?elds, theco-locationofbothconferencesisbene?cialforbothresearchcommunities.In Cavtat, ECML and PKDD were co-located for the third time in a row, following the successful co-location of the two European conferences in Freiburg (2001) and Helsinki (2002). The co-location of ECML2003 and PKDD2003 resulted in a joint program for the two conferences, including paper presentations, invited talks, tutorials, and workshops. Out of 332 submitted papers, 40 were accepted for publication in the ECML2003proceedings, and40wereacceptedforpublicationinthePKDD2003 proceedings. All the submitted papers were reviewed by three referees. In ad- tion to submitted papers, the conference program consisted of four invited talks, four tutorials, seven workshops, two tutorials combined with a workshop, and a discovery challenge

This volume contains papers presented at the 17th Annual Conference on Le- ning Theory (previously known as the Conference on Computational Learning Theory) held in Ban?, Canada from July 1 to 4, 2004. The technical program contained 43 papers selected from 107 submissions, 3 open problems selected from among 6 contributed, and 3 invited lectures. The invited lectures were given by Michael Kearns on 'Game Theory, Automated Trading and Social Networks', Moses Charikar on 'Algorithmic Aspects of - nite Metric Spaces', and Stephen Boyd on 'Convex Optimization, Semide?nite Programming, and Recent Applications'. These papers were not included in this volume. The Mark Fulk Award is presented annually for the best paper co-authored by a student. Thisyear theMark Fulk award wassupplemented with two further awards funded by the Machine Learning Journal and the National Information Communication Technology Centre, Australia (NICTA). We were therefore able toselectthreestudentpapersforprizes.ThestudentsselectedwereMagalieF- montforthesingle-authorpaper"ModelSelectionbyBootstrapPenalizationfor Classi?cation", Daniel Reidenbach for the single-author paper "On the Lear- bility of E-Pattern Languages over Small Alphabets", and Ran Gilad-Bachrach for the paper "Bayes and Tukey Meet at the Center Point" (co-authored with Amir Navot and Naftali Tishby).

This volume contains papers presented at the joint 16th Annual Conference on Learning Theory (COLT) and the 7th Annual Workshop on Kernel Machines, heldinWashington, DC, USA, duringAugust24 27,2003.COLT, whichrecently merged with EuroCOLT, has traditionally been a meeting place for learning theorists. We hope that COLT will bene't from the collocation with the annual workshoponkernelmachines, formerlyheldasaNIPSpostconferenceworkshop. The technical program contained 47 papers selected from 92 submissions. All 47paperswerepresentedasposters;22ofthepaperswereadditionallypresented astalks.Therewerealsotwotargetareaswithinvitedcontributions.Incompu- tional game theory, atutorialentitled LearningTopicsinGame-TheoreticDe- sionMaking wasgivenbyMichaelLittman, andaninvitedpaperon AGeneral Class of No-Regret Learning Algorithms and Game-Theoretic Equilibria was contributed by Amy Greenwald. In natural language processing, a tutorial on Machine Learning Methods in Natural Language Processing was presented by Michael Collins, followed by two invited talks, Learning from Uncertain Data by Mehryar Mohri and Learning and Parsing Stochastic Uni?cation- Based Grammars by Mark Johnson. In addition to the accepted papers and invited presentations, we solicited short open problems that were reviewed and included in the proceedings. We hope that reviewed open problems might become a new tradition for COLT. Our goal was to select simple signature problems whose solutions are likely to inspire further research. For some of the problems the authors o?ered monetary rewards. Yoav Freund acted as the open problem area chair. The open problems were presented as posters at the conference."

Computational learning theory is one of the first attempts to construct a mathematical theory of a cognitive process. It has been a field of much interest and rapid growth in recent years. This text provides a framework for studying a variety of algorithmic processes, such as those currently in use for training artificial neural networks. The authors concentrate on an approximate model for learning and gradually develop the ideas of efficiency considerations. Finally, they consider applications of the theory to artificial neural networks. An abundance of exercises and an extensive list of references round out the text. This volume provides a comprehensive review of the topic, including information drawn from logic, probability, and complexity theory. It forms a solid introduction to the theory of comptutational learning suitable for a broad spectrum of graduate students from theoretical computer science to mathematics.

For some time, all branches of the military have used a wide range of sensors to provide data for many purposes, including surveillance, reconnoitring, target detection and battle damage assessment. Many nations have also attempted to utilise these sensors for civilian applications, such as crop monitoring, agricultural disease tracking, environmental diagnostics, cartography, ocean temperature profiling, urban planning, and the characterisation of the Ozone Hole above Antarctica. The recent convergence of several important technologies has made possible new, advanced, high performance, sensor based applications relying on the near-simultaneous fusion of data from an ensemble of different types of sensors. The book examines the underlying principles of sensor operation and data fusion, the techniques and technologies that enable the process, including the operation of 'fusion engines'. Fundamental theory and the enabling technologies of data fusion are presented in a systematic and accessible manner. Applications are discussed in the areas of medicine, meteorology, BDA and targeting, transportation, cartography, the environment, agriculture, and manufacturing and process control.

This book constitutes the refereed proceedings of the Third International Conference on Intelligent Data Engineering and Automated Learning, IDEAL 2002, held in Manchester, UK in August 2002.The 89 revised papers presented were carefully reviewed and selected from more than 150 submissions. The book offers topical sections on data mining, knowledge engineering, text and document processing, internet applications, agent technology, autonomous mining, financial engineering, bioinformatics, learning systems, and pattern recognition.

This book constitutes the refereed proceedings of the Third International Workshop on Multiple Classifier Systems, MCS 2002, held in Cagliari, Italy, in June 2002.The 29 revised full papers presented together with three invited papers were carefully reviewed and selected for inclusion in the volume. The papers are organized in topical sections on bagging and boosting, ensemble learning and neural networks, design methodologies, combination strategies, analysis and performance evaluation, and applications.