This book honours the outstanding contributions of Vladimir Vapnik, a rare example of a scientist for whom the following statements hold true simultaneously: his work led to the inception of a new field of research, the theory of statistical learning and empirical inference; he has lived to see the field blossom; and he is still as active as ever. He started analyzing learning algorithms in the 1960s and he invented the first version of the generalized portrait algorithm. He later developed one of the most successful methods in machine learning, the support vector machine (SVM) - more than just an algorithm, this was a new approach to learning problems, pioneering the use of functional analysis and convex optimization in machine learning. Part I of this book contains three chapters describing and witnessing some of Vladimir Vapnik's contributions to science. In the first chapter, Leon Bottou discusses the seminal paper published in 1968 by Vapnik and Chervonenkis that lay the foundations of statistical learning theory, and the second chapter is an English-language translation of that original paper. In the third chapter, Alexey Chervonenkis presents a first-hand account of the early history of SVMs and valuable insights into the first steps in the development of the SVM in the framework of the generalised portrait method. The remaining chapters, by leading scientists in domains such as statistics, theoretical computer science, and mathematics, address substantial topics in the theory and practice of statistical learning theory, including SVMs and other kernel-based methods, boosting, PAC-Bayesian theory, online and transductive learning, loss functions, learnable function classes, notions of complexity for function classes, multitask learning, and hypothesis selection. These contributions include historical and context notes, short surveys, and comments on future research directions. This book will be of interest to researchers, engineers, and graduate students engaged with all aspects of statistical learning.

This book honours the outstanding contributions of Vladimir Vapnik, a rare example of a scientist for whom the following statements hold true simultaneously: his work led to the inception of a new field of research, the theory of statistical learning and empirical inference; he has lived to see the field blossom; and he is still as active as ever. He started analyzing learning algorithms in the 1960s and he invented the first version of the generalized portrait algorithm. He later developed one of the most successful methods in machine learning, the support vector machine (SVM) - more than just an algorithm, this was a new approach to learning problems, pioneering the use of functional analysis and convex optimization in machine learning. Part I of this book contains three chapters describing and witnessing some of Vladimir Vapnik's contributions to science. In the first chapter, Leon Bottou discusses the seminal paper published in 1968 by Vapnik and Chervonenkis that lay the foundations of statistical learning theory, and the second chapter is an English-language translation of that original paper. In the third chapter, Alexey Chervonenkis presents a first-hand account of the early history of SVMs and valuable insights into the first steps in the development of the SVM in the framework of the generalised portrait method. The remaining chapters, by leading scientists in domains such as statistics, theoretical computer science, and mathematics, address substantial topics in the theory and practice of statistical learning theory, including SVMs and other kernel-based methods, boosting, PAC-Bayesian theory, online and transductive learning, loss functions, learnable function classes, notions of complexity for function classes, multitask learning, and hypothesis selection. These contributions include historical and context notes, short surveys, and comments on future research directions. This book will be of interest to researchers, engineers, and graduate students engaged with all aspects of statistical learning.

We are delighted to present the proceedings of DAGM 2004, and wish to - press our gratitude to the many people whose e?orts made the success of the conference possible. We received 146 contributions of which we were able to - cept 22 as oral presentations and 48 as posters. Each paper received 3 reviews, upon which decisions were based. We are grateful for the dedicated work of the 38 members of the program committee and the numerous referees. The careful review process led to the exciting program which we are able to present in this volume. Among the highlights of the meeting were the talks of our four invited spe- ers, renowned experts in areas spanning learning in theory, in vision and in robotics: - William T. Freeman, Arti?cial Intelligence Laboratory, MIT: Sharing F- tures for Multi-class Object Detection - PietroPerona,Caltech:TowardsUnsupervisedLearningofObjectCategories - StefanSchaal,DepartmentofComputerScience,UniversityofSouthernC- ifornia: Real-Time Statistical Learning for Humanoid Robotics - Vladimir Vapnik, NEC Research Institute: Empirical Inference WearegratefulforeconomicsupportfromHondaResearchInstituteEurope, ABW GmbH, Transtec AG, DaimlerChrysler, and Stemmer Imaging GmbH, which enabled us to ? nance best paper prizes and a limited number of travel grants. Many thanks to our local support Sabrina Nielebock and Dagmar Maier, who dealt with the unimaginably diverse range of practical tasks involved in planning a DAGM symposium. Thanks to Richard van de Stadt for providing excellent software and support for handling the reviewing process. A special thanks goes to Jeremy Hill, who wrote and maintained the conference website.

This volume contains papers presented at the joint 16th Annual Conference on Learning Theory (COLT) and the 7th Annual Workshop on Kernel Machines, heldinWashington, DC, USA, duringAugust24 27,2003.COLT, whichrecently merged with EuroCOLT, has traditionally been a meeting place for learning theorists. We hope that COLT will bene't from the collocation with the annual workshoponkernelmachines, formerlyheldasaNIPSpostconferenceworkshop. The technical program contained 47 papers selected from 92 submissions. All 47paperswerepresentedasposters;22ofthepaperswereadditionallypresented astalks.Therewerealsotwotargetareaswithinvitedcontributions.Incompu- tional game theory, atutorialentitled LearningTopicsinGame-TheoreticDe- sionMaking wasgivenbyMichaelLittman, andaninvitedpaperon AGeneral Class of No-Regret Learning Algorithms and Game-Theoretic Equilibria was contributed by Amy Greenwald. In natural language processing, a tutorial on Machine Learning Methods in Natural Language Processing was presented by Michael Collins, followed by two invited talks, Learning from Uncertain Data by Mehryar Mohri and Learning and Parsing Stochastic Uni?cation- Based Grammars by Mark Johnson. In addition to the accepted papers and invited presentations, we solicited short open problems that were reviewed and included in the proceedings. We hope that reviewed open problems might become a new tradition for COLT. Our goal was to select simple signature problems whose solutions are likely to inspire further research. For some of the problems the authors o?ered monetary rewards. Yoav Freund acted as the open problem area chair. The open problems were presented as posters at the conference."

Numerous fascinating breakthroughs in biotechnology have generated large volumes and diverse types of high throughput data that demand the development of efficient and appropriate tools in computational statistics integrated with biological knowledge and computational algorithms. This volume collects contributed chapters from leading researchers to survey the many active research topics and promote the visibility of this research area. This volume is intended to provide an introductory and reference book for students and researchers who are interested in the recent developments of computational statistics in computational biology.

Now in its second edition, this handbook collects authoritative contributions on modern methods and tools in statistical bioinformatics with a focus on the interface between computational statistics and cutting-edge developments in computational biology. The three parts of the book cover statistical methods for single-cell analysis, network analysis, and systems biology, with contributions by leading experts addressing key topics in probabilistic and statistical modeling and the analysis of massive data sets generated by modern biotechnology. This handbook will serve as a useful reference source for students, researchers and practitioners in statistics, computer science and biological and biomedical research, who are interested in the latest developments in computational statistics as applied to computational biology.

Numerous fascinating breakthroughs in biotechnology have generated large volumes and diverse types of high throughput data that demand the development of efficient and appropriate tools in computational statistics integrated with biological knowledge and computational algorithms. This volume collects contributed chapters from leading researchers to survey the many active research topics and promote the visibility of this research area. This volume is intended to provide an introductory and reference book for students and researchers who are interested in the recent developments of computational statistics in computational biology.

A concise and self-contained introduction to causal inference, increasingly important in data science and machine learning. The mathematization of causality is a relatively recent development, and has become increasingly important in data science and machine learning. This book offers a self-contained and concise introduction to causal models and how to learn them from data. After explaining the need for causal models and discussing some of the principles underlying causal inference, the book teaches readers how to use causal models: how to compute intervention distributions, how to infer causal models from observational and interventional data, and how causal ideas could be exploited for classical machine learning problems. All of these topics are discussed first in terms of two variables and then in the more general multivariate case. The bivariate case turns out to be a particularly hard problem for causal learning because there are no conditional independences as used by classical methods for solving multivariate cases. The authors consider analyzing statistical asymmetries between cause and effect to be highly instructive, and they report on their decade of intensive research into this problem. The book is accessible to readers with a background in machine learning or statistics, and can be used in graduate courses or as a reference for researchers. The text includes code snippets that can be copied and pasted, exercises, and an appendix with a summary of the most important technical concepts.

State-of-the-art algorithms and theory in a novel domain of machine learning, prediction when the output has structure. Machine learning develops intelligent computer systems that are able to generalize from previously seen examples. A new domain of machine learning, in which the prediction must satisfy the additional constraints found in structured data, poses one of machine learning's greatest challenges: learning functional dependencies between arbitrary input and output domains. This volume presents and analyzes the state of the art in machine learning algorithms and theory in this novel field. The contributors discuss applications as diverse as machine translation, document markup, computational biology, and information extraction, among others, providing a timely overview of an exciting field. Contributors Yasemin Altun, Goekhan Bakir, Olivier Bousquet, Sumit Chopra, Corinna Cortes, Hal Daume III, Ofer Dekel, Zoubin Ghahramani, Raia Hadsell, Thomas Hofmann, Fu Jie Huang, Yann LeCun, Tobias Mann, Daniel Marcu, David McAllester, Mehryar Mohri, William Stafford Noble, Fernando Perez-Cruz, Massimiliano Pontil, Marc'Aurelio Ranzato, Juho Rousu, Craig Saunders, Bernhard Schoelkopf, Matthias W. Seeger, Shai Shalev-Shwartz, John Shawe-Taylor, Yoram Singer, Alexander J. Smola, Sandor Szedmak, Ben Taskar, Ioannis Tsochantaridis, S.V.N Vishwanathan, Jason Weston

A comprehensive introduction to Support Vector Machines and related kernel methods. In the 1990s, a new type of learning algorithm was developed, based on results from statistical learning theory: the Support Vector Machine (SVM). This gave rise to a new class of theoretically elegant learning machines that use a central concept of SVMs--kernels-for a number of learning tasks. Kernel machines provide a modular framework that can be adapted to different tasks and domains by the choice of the kernel function and the base algorithm. They are replacing neural networks in a variety of fields, including engineering, information retrieval, and bioinformatics. Learning with Kernels provides an introduction to SVMs and related kernel methods. Although the book begins with the basics, it also includes the latest research. It provides all of the concepts necessary to enable a reader equipped with some basic mathematical knowledge to enter the world of machine learning using theoretically well-founded yet easy-to-use kernel algorithms and to understand and apply the powerful algorithms that have been developed over the last few years.