This extraordinary three-volume work, written in an engaging and rigorous style by a world authority in the field, provides an accessible, comprehensive introduction to the full spectrum of mathematical and statistical techniques underpinning contemporary methods in data-driven learning and inference. This first volume, Foundations, introduces core topics in inference and learning, such as matrix theory, linear algebra, random variables, convex optimization and stochastic optimization, and prepares students for studying their practical application in later volumes. A consistent structure and pedagogy is employed throughout this volume to reinforce student understanding, with over 600 end-of-chapter problems (including solutions for instructors), 100 figures, 180 solved examples, datasets and downloadable Matlab code. Supported by sister volumes Inference and Learning, and unique in its scale and depth, this textbook sequence is ideal for early-career researchers and graduate students across many courses in signal processing, machine learning, statistical analysis, data science and inference.

Recent years have witnessed an explosion in the volume and variety of data collected in all scientific disciplines and industrial settings. Such massive data sets present a number of challenges to researchers in statistics and machine learning. This book provides a self-contained introduction to the area of high-dimensional statistics, aimed at the first-year graduate level. It includes chapters that are focused on core methodology and theory - including tail bounds, concentration inequalities, uniform laws and empirical process, and random matrices - as well as chapters devoted to in-depth exploration of particular model classes - including sparse linear models, matrix models with rank constraints, graphical models, and various types of non-parametric models. With hundreds of worked examples and exercises, this text is intended both for courses and for self-study by graduate students and researchers in statistics, machine learning, and related fields who must understand, apply, and adapt modern statistical methods suited to large-scale data.

The field of machine learning has matured to the point where many sophisticated learning approaches can be applied to practical applications. Thus it is of critical importance that researchers have the proper tools to evaluate learning approaches and understand the underlying issues. This book examines various aspects of the evaluation process with an emphasis on classification algorithms. The authors describe several techniques for classifier performance assessment, error estimation and resampling, obtaining statistical significance as well as selecting appropriate domains for evaluation. They also present a unified evaluation framework and highlight how different components of evaluation are both significantly interrelated and interdependent. The techniques presented in the book are illustrated using R and WEKA, facilitating better practical insight as well as implementation. Aimed at researchers in the theory and applications of machine learning, this book offers a solid basis for conducting performance evaluations of algorithms in practical settings.

Machine Learning and Pattern Recognition Methods in Chemistry from Multivariate and Data Driven Modeling outlines key knowledge in this area, combining critical introductory approaches with the latest advanced techniques. Beginning with an introduction of univariate and multivariate statistical analysis, the book then explores multivariate calibration and validation methods. Soft modeling in chemical data analysis, hyperspectral data analysis, and autoencoder applications in analytical chemistry are then discussed, providing useful examples of the techniques in chemistry applications. Drawing on the knowledge of a global team of researchers, this book will be a helpful guide for chemists interested in developing their skills in multivariate data and error analysis.

Machine learning is an interdisciplinary field of science and engineering that studies mathematical theories and practical applications of systems that learn. This book introduces theories, methods and applications of density ratio estimation, which is a newly emerging paradigm in the machine learning community. Various machine learning problems such as non-stationarity adaptation, outlier detection, dimensionality reduction, independent component analysis, clustering, classification and conditional density estimation can be systematically solved via the estimation of probability density ratios. The authors offer a comprehensive introduction of various density ratio estimators including methods via density estimation, moment matching, probabilistic classification, density fitting and density ratio fitting, as well as describing how these can be applied to machine learning. The book provides mathematical theories for density ratio estimation including parametric and non-parametric convergence analysis and numerical stability analysis to complete the first and definitive treatment of the entire framework of density ratio estimation in machine learning.

This book presents an integrated collection of representative approaches for scaling up machine learning and data mining methods on parallel and distributed computing platforms. Demand for parallelizing learning algorithms is highly task-specific: in some settings it is driven by the enormous dataset sizes, in others by model complexity or by real-time performance requirements. Making task-appropriate algorithm and platform choices for large-scale machine learning requires understanding the benefits, trade-offs and constraints of the available options. Solutions presented in the book cover a range of parallelization platforms from FPGAs and GPUs to multi-core systems and commodity clusters, concurrent programming frameworks including CUDA, MPI, MapReduce and DryadLINQ, and learning settings (supervised, unsupervised, semi-supervised and online learning). Extensive coverage of parallelization of boosted trees, SVMs, spectral clustering, belief propagation and other popular learning algorithms, and deep dives into several applications, make the book equally useful for researchers, students and practitioners.

In today's security-conscious society, real-world applications for authentication or identification require a highly accurate system for recognizing individual humans. The required level of performance cannot be achieved through the use of a single biometric such as face, fingerprint, ear, iris, palm, gait or speech. Fusing multiple biometrics enables the indexing of large databases, more robust performance and enhanced coverage of populations. Multiple biometrics are also naturally more robust against attacks than single biometrics. This book addresses a broad spectrum of research issues on multibiometrics for human identification, ranging from sensing modes and modalities to fusion of biometric samples and combination of algorithms. It covers publicly available multibiometrics databases, theoretical and empirical studies on sensor fusion techniques in the context of biometrics authentication, identification and performance evaluation and prediction.

The fundamental mathematical tools needed to understand machine learning include linear algebra, analytic geometry, matrix decompositions, vector calculus, optimization, probability and statistics. These topics are traditionally taught in disparate courses, making it hard for data science or computer science students, or professionals, to efficiently learn the mathematics. This self-contained textbook bridges the gap between mathematical and machine learning texts, introducing the mathematical concepts with a minimum of prerequisites. It uses these concepts to derive four central machine learning methods: linear regression, principal component analysis, Gaussian mixture models and support vector machines. For students and others with a mathematical background, these derivations provide a starting point to machine learning texts. For those learning the mathematics for the first time, the methods help build intuition and practical experience with applying mathematical concepts. Every chapter includes worked examples and exercises to test understanding. Programming tutorials are offered on the book's web site.

The field of machine learning has matured to the point where many sophisticated learning approaches can be applied to practical applications. Thus it is of critical importance that researchers have the proper tools to evaluate learning approaches and understand the underlying issues. This book examines various aspects of the evaluation process with an emphasis on classification algorithms. The authors describe several techniques for classifier performance assessment, error estimation and resampling, obtaining statistical significance as well as selecting appropriate domains for evaluation. They also present a unified evaluation framework and highlight how different components of evaluation are both significantly interrelated and interdependent. The techniques presented in the book are illustrated using R and WEKA, facilitating better practical insight as well as implementation. Aimed at researchers in the theory and applications of machine learning, this book offers a solid basis for conducting performance evaluations of algorithms in practical settings.

Correlation is a robust and general technique for pattern recognition and is used in many applications, such as automatic target recognition, biometric recognition and optical character recognition. The design, analysis and use of correlation pattern recognition algorithms requires background information, including linear systems theory, random variables and processes, matrix/vector methods, detection and estimation theory, digital signal processing and optical processing. This book provides a needed review of this diverse background material and develops the signal processing theory, the pattern recognition metrics, and the practical application know-how from basic premises. It shows both digital and optical implementations. It also contains technology presented by the team that developed it and includes case studies of significant interest, such as face and fingerprint recognition. Suitable for graduate students taking courses in pattern recognition theory, whilst reaching technical levels of interest to the professional practitioner.

String matching problems range from the relatively simple task of searching a single text for a string of characters to searching a database for approximate occurrences of a complex pattern. Recent years have witnessed a dramatic increase of interest in sophisticated string matching problems, especially in information retrieval and computational biology. This book presents a practical approach to string matching problems, focusing on the algorithms and implementations that perform best in practice. It covers searching for simple, multiple and extended strings, as well as regular expressions, and exact and approximate searching. It includes all the most significant new developments in complex pattern searching. The clear explanations, step-by-step examples, algorithm pseudocode, and implementation efficiency maps will enable researchers, professionals and students in bioinformatics, computer science, and software engineering to choose the most appropriate algorithms for their applications.

Correlation is a robust and general technique for pattern recognition and is used in many applications, such as automatic target recognition, biometric recognition and optical character recognition. The design, analysis and use of correlation pattern recognition algorithms requires background information, including linear systems theory, random variables and processes, matrix/vector methods, detection and estimation theory, digital signal processing and optical processing. This book provides a needed review of this diverse background material and develops the signal processing theory, the pattern recognition metrics, and the practical application know-how from basic premises. It shows both digital and optical implementations. It also contains state-of-the-art technology presented by the team that developed it and includes case studies of significant current interest, such as face and fingerprint recognition. Suitable for advanced undergraduate or graduate students taking courses in pattern recognition theory, whilst reaching technical levels of interest to the professional practitioner.

High-dimensional probability offers insight into the behavior of random vectors, random matrices, random subspaces, and objects used to quantify uncertainty in high dimensions. Drawing on ideas from probability, analysis, and geometry, it lends itself to applications in mathematics, statistics, theoretical computer science, signal processing, optimization, and more. It is the first to integrate theory, key tools, and modern applications of high-dimensional probability. Concentration inequalities form the core, and it covers both classical results such as Hoeffding's and Chernoff's inequalities and modern developments such as the matrix Bernstein's inequality. It then introduces the powerful methods based on stochastic processes, including such tools as Slepian's, Sudakov's, and Dudley's inequalities, as well as generic chaining and bounds based on VC dimension. A broad range of illustrations is embedded throughout, including classical and modern results for covariance estimation, clustering, networks, semidefinite programming, coding, dimension reduction, matrix completion, machine learning, compressed sensing, and sparse regression.

This book is the first of its kind to discuss error estimation with a model-based approach. From the basics of classifiers and error estimators to distributional and Bayesian theory, it covers important topics and essential issues pertaining to the scientific validity of pattern classification. Error Estimation for Pattern Recognition focuses on error estimation, which is a broad and poorly understood topic that reaches all research areas using pattern classification. It includes model-based approaches and discussions of newer error estimators such as bolstered and Bayesian estimators. This book was motivated by the application of pattern recognition to high-throughput data with limited replicates, which is a basic problem now appearing in many areas. The first two chapters cover basic issues in classification error estimation, such as definitions, test-set error estimation, and training-set error estimation. The remaining chapters in this book cover results on the performance and representation of training-set error estimators for various pattern classifiers. Additional features of the book include: - The latest results on the accuracy of error estimation - Performance analysis of re-substitution, cross-validation, and bootstrap error estimators using analytical and simulation approaches - Highly interactive computer-based exercises and end-of-chapter problems This is the first book exclusively about error estimation for pattern recognition. Ulisses M. Braga Neto is an Associate Professor in the Department of Electrical and Computer Engineering at Texas A&M University, USA. He received his PhD in Electrical and Computer Engineering from The Johns Hopkins University. Dr. Braga Neto received an NSF CAREER Award for his work on error estimation for pattern recognition with applications in genomic signal processing. He is an IEEE Senior Member. Edward R. Dougherty is a Distinguished Professor, Robert F. Kennedy '26 Chair, and Scientific Director at the Center for Bioinformatics and Genomic Systems Engineering at Texas A&M University, USA. He is a fellow of both the IEEE and SPIE, and he has received the SPIE Presidents Award. Dr. Dougherty has authored several books including Epistemology of the Cell: A Systems Perspective on Biological Knowledge and Random Processes for Image and Signal Processing (Wiley-IEEE Press).

'A must-read' New Scientist 'Fascinating' Greta Thunberg 'Enthralling' George Monbiot 'Brilliant' Philip Hoare A thrilling investigation into the pioneering world of animal communication, where big data and artificial intelligence are changing our relationship with animals forever In 2015, wildlife filmmaker Tom Mustill was whale watching when a humpback breached onto his kayak and nearly killed him. After a video clip of the event went viral, Tom found himself inundated with theories about what happened. He became obsessed with trying to find out what the whale had been thinking and sometimes wished he could just ask it. In the process of making a film about his experience, he discovered that might not be such a crazy idea. This is a story about the pioneers in a new age of discovery, whose cutting-edge developments in natural science and technology are taking us to the brink of decoding animal communication - and whales, with their giant mammalian brains and sophisticated vocalisations, offer one of the most realistic opportunities for us to do so. Using 'underwater ears,' robotic fish, big data and machine intelligence, leading scientists and tech-entrepreneurs across the world are working to turn the fantasy of Dr Dolittle into a reality, upending much of what we know about these mysterious creatures. But what would it mean if we were to make contact? And with climate change threatening ever more species with extinction, would doing so alter our approach to the natural world? Enormously original and hugely entertaining, How to Speak Whale is an unforgettable look at how close we truly are to communicating with another species - and how doing so might change our world beyond recognition.

This book highlights the latest research in pen and touch, its current use in STEM classrooms, sketching and haptics technologies. Computer and educational scientists from academia and industry presented their research at the Conference on Pen and Touch Technology on Education (CPTTE) 2017 on the advancement of digital ink technology and its applications for college and K-12 classrooms. This book is the synthesis of the presented results and the ideas generated from conference discussions. This volume contains seven parts; exploring topics like sketching forensics, teaching STEM, sketch recognition applications, creating a learning environment with sketching, teaching to sketch, and haptics. The book focuses on intelligent systems using digital ink that enable pen and touch interaction that teach and inspire students. Inspiring Students through Digital Ink is a must-read for anyone wanting to improve today's student experiences and apply innovative approaches in the classroom. Also highlighted are current and future directions in pen and touch research.

This book develops a conceptual understanding of Artificial Intelligence (AI), Deep Learning and Machine Learning in the truest sense of the word. It is an earnest endeavor to unravel what is happening at the algorithmic level, to grasp how applications are being built and to show the long adventurous road in the future. An Intuitive Exploration of Artificial Intelligence offers insightful details on how AI works and solves problems in computer vision, natural language understanding, speech understanding, reinforcement learning and synthesis of new content. From the classic problem of recognizing cats and dogs, to building autonomous vehicles, to translating text into another language, to automatically converting speech into text and back to speech, to generating neural art, to playing games, and the author's own experience in building solutions in industry, this book is about explaining how exactly the myriad applications of AI flow out of its immense potential. The book is intended to serve as a textbook for graduate and senior-level undergraduate courses in AI. Moreover, since the book provides a strong geometrical intuition about advanced mathematical foundations of AI, practitioners and researchers will equally benefit from the book.

Software design patterns are known to play a vital role in enhancing the quality of software systems while reducing development time and cost. However, the use of these design patterns has also been known to introduce problems that can significantly reduce the stability, robustness, and reusability of software. This book introduces a new process for creating software design patterns that leads to highly stable, reusable, and cost-effective software. The basis of this new process is a topology of software patterns called knowledge maps. This book provides readers with a detailed view of the art and practice of creating meaningful knowledge maps. It demonstrates how to classify software patterns within knowledge maps according to their application rationale and nature. It provides readers with a clear methodology in the form of step-by-step guidelines, heuristics, and quality factors that simplify the process of creating knowledge maps. This book is designed to allow readers to master the basics of knowledge maps from their theoretical aspects to practical application. It begins with an overview of knowledge map concepts and moves on to knowledge map goals, capabilities, stable design patterns, development scenarios, and case studies. Each chapter of the book concludes with an open research issue, review questions, exercises, and a series of projects.

The Handbook of Human Motion is a large cross-disciplinary reference work which covers the many interlinked facets of the science and technology of human motion and its measurement. Individual chapters cover fundamental principles and technological developments, the state-of-the-art and consider applications across four broad and interconnected fields; medicine, sport, forensics and animation. The huge strides in technological advancement made over the past century make it possible to measure motion with unprecedented precision, but also lead to new challenges. This work introduces the many different approaches and systems used in motion capture, including IR and ultrasound, mechanical systems and video, plus some emerging techniques. The large variety of techniques used for the study of motion science in medicine can make analysis a complicated process, but extremely effective for the treatment of the patient when well utilised. The handbook descri bes how motion capture techniques are applied in medicine, and shows how the resulting analysis can help in diagnosis and treatment. A closely related field, sports science involves a combination of in-depth medical knowledge and detailed understanding of performance and training techniques, and motion capture can play an extremely important role in linking these disciplines. The handbook considers which technologies are most appropriate in specific circumstances, how they are applied and how this can help prevent injury and improve sporting performance. The application of motion capture in forensic science and security is reviewed, with chapters dedicated to specific areas including employment law, injury analysis, criminal activity and motion/facial recognition. And in the final area of application, the book describes how novel motion capture techniques have been designed specifically to aid the creation of increasingly realistic animation within films and v ideo games, with Lord of the Rings and Avatar just two examples. Chapters will provide an overview of the bespoke motion capture techniques developed for animation, how these have influenced advances in film and game design, and the links to behavioural studies, both in humans and in robotics. Comprising a cross-referenced compendium of different techniques and applications across a broad field, the Handbook of Human Motion provides the reader with a detailed reference and simultaneously a source of inspiration for future work. The book will be of use to students, researchers, engineers and others working in any field relevant to human motion capture.

For machine intelligence applications to work successfully, machines must perform reliably under variations of data and must be able to keep up with data streams. Internet-Scale Pattern Recognition: New Techniques for Voluminous Data Sets and Data Clouds unveils computational models that address performance and scalability to achieve higher levels of reliability. It explores different ways of implementing pattern recognition using machine intelligence. Based on the authors' research from the past 10 years, the text draws on concepts from pattern recognition, parallel processing, distributed systems, and data networks. It describes fundamental research on the scalability and performance of pattern recognition, addressing issues with existing pattern recognition schemes for Internet-scale data deployment. The authors review numerous approaches and introduce possible solutions to the scalability problem. By presenting the concise body of knowledge required for reliable and scalable pattern recognition, this book shortens the learning curve and gives you valuable insight to make further innovations. It offers an extendable template for Internet-scale pattern recognition applications as well as guidance on the programming of large networks of devices.

Biometric recognition, or simply biometrics, is the science of establishing the identity of a person based on physical or behavioral attributes. It is a rapidly evolving field with applications ranging from securely accessing one's computer to gaining entry into a country. While the deployment of large-scale biometric systems in both commercial and government applications has increased the public awareness of this technology, "Introduction to Biometrics" is the first textbook to introduce the fundamentals of Biometrics to undergraduate/graduate students. The three commonly used modalities in the biometrics field, namely, fingerprint, face, and iris are covered in detail in this book. Few other modalities like hand geometry, ear, and gait are also discussed briefly along with advanced topics such as multibiometric systems and security of biometric systems. Exercises for each chapter will be available on the book website to help students gain a better understanding of the topics and obtain practical experience in designing computer programs for biometric applications. These can be found at: http://www.csee.wvu.edu/~ross/BiometricsTextBook/. Designed for undergraduate and graduate students in computer science and electrical engineering, "Introduction to Biometrics" is also suitable for researchers and biometric and computer security professionals.

A comprehensive overview of high-performance pattern recognition techniques and approaches to Computational Molecular Biology This book surveys the developments of techniques and approaches on pattern recognition related to Computational Molecular Biology. Providing a broad coverage of the field, the authors cover fundamental and technical information on these techniques and approaches, as well as discussing their related problems. The text consists of twenty nine chapters, organized into seven parts: Pattern Recognition in Sequences, Pattern Recognition in Secondary Structures, Pattern Recognition in Tertiary Structures, Pattern Recognition in Quaternary Structures, Pattern Recognition in Microarrays, Pattern Recognition in Phylogenetic Trees, and Pattern Recognition in Biological Networks. * Surveys the development of techniques and approaches on pattern recognition in biomolecular data * Discusses pattern recognition in primary, secondary, tertiary and quaternary structures, as well as microarrays, phylogenetic trees and biological networks * Includes case studies and examples to further illustrate the concepts discussed in the book Pattern Recognition in Computational Molecular Biology: Techniques and Approaches is a reference for practitioners and professional researches in Computer Science, Life Science, and Mathematics. This book also serves as a supplementary reading for graduate students and young researches interested in Computational Molecular Biology.

Stereoscopic processes are increasingly used in virtual reality and entertainment. This technology is interesting because it allows for a quick immersion of the user, especially in terms of depth perception and relief clues. However, these processes tend to cause stress on the visual system if used over a prolonged period of time, leading some to question the cause of side effects that these systems generate in their users, such as eye fatigue. This book explores the mechanisms of depth perception with and without stereoscopy and discusses the indices which are involved in the depth perception. The author describes the techniques used to capture and retransmit stereoscopic images. The causes of eyestrain related to these images are then presented along with their consequences in the long and short term. The study of the causes of eyestrain forms the basis for an improvement in these processes in the hopes of developing mechanisms for easier virtual viewing.

This groundbreaking textbook combines straightforward explanations with a wealth of practical examples to offer an innovative approach to teaching linear algebra. Requiring no prior knowledge of the subject, it covers the aspects of linear algebra - vectors, matrices, and least squares - that are needed for engineering applications, discussing examples across data science, machine learning and artificial intelligence, signal and image processing, tomography, navigation, control, and finance. The numerous practical exercises throughout allow students to test their understanding and translate their knowledge into solving real-world problems, with lecture slides, additional computational exercises in Julia and MATLAB (R), and data sets accompanying the book online. Suitable for both one-semester and one-quarter courses, as well as self-study, this self-contained text provides beginning students with the foundation they need to progress to more advanced study.

Face detection and recognition are the nonintrusive biometrics of choice in many security applications. Examples of their use include border control, driver's license issuance, law enforcement investigations, and physical access control. Face Detection and Recognition: Theory and Practice elaborates on andexplains the theory and practice of face detection and recognition systems currently in vogue. The book begins with an introduction to the state of the art, offering a general review of the available methods and an indication of future research using cognitive neurophysiology. The text then: Explores subspace methods for dimensionality reduction in face image processing, statistical methods applied to face detection, and intelligent face detection methods dominated by the use of artificial neural networks Covers face detection with colour and infrared face images, face detection in real time, face detection and recognition using set estimation theory, face recognition using evolutionary algorithms, and face recognition in frequency domain Discusses methods for the localization of face landmarks helpful in face recognition, methods of generating synthetic face images using set estimation theory, and databases of face images available for testing and training systems Features pictorial descriptions of every algorithm as well as downloadable source code (in MATLAB (R)/PYTHON) and hardware implementation strategies with code examples Demonstrates how frequency domain correlation techniques can be used supplying exhaustive test results Face Detection and Recognition: Theory and Practice provides students, researchers, and practitioners with a single source for cutting-edge information on the major approaches, algorithms, and technologies used in automated face detection and recognition.