Connectionist Speech Recognition: A Hybrid Approach describes the theory and implementation of a method to incorporate neural network approaches into state of the art continuous speech recognition systems based on hidden Markov models (HMMs) to improve their performance. In this framework, neural networks (and in particular, multilayer perceptrons or MLPs) have been restricted to well-defined subtasks of the whole system, i.e. HMM emission probability estimation and feature extraction. The book describes a successful five-year international collaboration between the authors. The lessons learned form a case study that demonstrates how hybrid systems can be developed to combine neural networks with more traditional statistical approaches. The book illustrates both the advantages and limitations of neural networks in the framework of a statistical systems. Using standard databases and comparison with some conventional approaches, it is shown that MLP probability estimation can improve recognition performance. Other approaches are discussed, though there is no such unequivocal experimental result for these methods. Connectionist Speech Recognition is of use to anyone intending to use neural networks for speech recognition or within the framework provided by an existing successful statistical approach. This includes research and development groups working in the field of speech recognition, both with standard and neural network approaches, as well as other pattern recognition and/or neural network researchers. The book is also suitable as a text for advanced courses on neural networks or speech processing.

The recent interest in artificial neural networks has motivated the publication of numerous books, including selections of research papers and textbooks presenting the most popular neural architectures and learning schemes. Artificial Neural Networks: Learning Algorithms, Performance Evaluation, and Applications presents recent developments which can have a very significant impact on neural network research, in addition to the selective review of the existing vast literature on artificial neural networks. This book can be read in different ways, depending on the background, the specialization, and the ultimate goals of the reader. A specialist will find in this book well-defined and easily reproducible algorithms, along with the performance evaluation of various neural network architectures and training schemes. Artificial Neural Networks can also help a beginner interested in the development of neural network systems to build the necessary background in an organized and comprehensive way. The presentation of the material in this book is based on the belief that the successful application of neural networks to real-world problems depends strongly on the knowledge of their learning properties and performance. Neural networks are introduced as trainable devices which have the unique ability to generalize. The pioneering work on neural networks which appeared during the past decades is presented, together with the current developments in the field, through a comprehensive and unified review of the most popular neural network architectures and learning schemes. Efficient LEarning Algorithms for Neural NEtworks (ELEANNE), which can achieve much faster convergence than existing learningalgorithms, are among the recent developments explored in this book. A new generalized criterion for the training of neural networks is presented, which leads to a variety of fast learning algorithms. Finally, Artificial Neural Networks presents the development of learning algorithms which determine the minimal architecture of multi-layered neural networks while performing their training. Artificial Neural Networks is a valuable source of information to all researchers and engineers interested in neural networks. The book may also be used as a text for an advanced course on the subject.

Learning and Generalization provides a formal mathematical theory for addressing intuitive questions of the type: * How does a machine learn a new concept on the basis of examples? * How can a neural network, after sufficient training, correctly predict the outcome of a previously unseen input? * How much training is required to achieve a specified level of accuracy in the prediction? * How can one identify the dynamical behaviour of a nonlinear control system by observing its input-output behaviour over a finite interval of time? The first edition, A Theory of Learning and Generalization, was the first book to treat the problem of machine learning in conjunction with the theory of empirical process, the latter being a well-established branch of probability theory. The treatment of both topics side-by-side leads to new insights, as well as new results in both topics. The second edition extends and improves upon this material, covering new areas including: * Support vector machines (SVM's) * Fat-shattering dimensions and applications to neural network learning * Learning with dependent samples generated by a beta-mixing process * Connections between system identification and learning theory * Probabilistic solution of 'intractable problems' in robust control and matrix theory using randomized algorithms It also contains solutions to some of the open problems posed in the first edition, while adding new open problems. This book is essential reading for control and system theorists, neural network researchers, theoretical computer scientists and probabilists The Communications and Control Engineering series reflects the major technological advances which have a great impact in the fields of communication and control. It reports on the research in industrial and academic institutions around the world to exploit the new possibilities which are becoming available

Embedded systems are usually composed of several interacting components such as custom or application specific processors, ASICs, memory blocks, and the associated communication infrastructure. The development of tools to support the design of such systems requires a further step from high-level synthesis towards a higher abstraction level. The lack of design tools accepting a system-level specification of a complete system, which may include both hardware and software components, is one of the major bottlenecks in the design of embedded systems. Thus, more and more research efforts have been spent on issues related to system-level synthesis. This book addresses the two most active research areas of design automation today: high-level synthesis and system-level synthesis. In particular, a transformational approach to synthesis from VHDL specifications is described. System Synthesis with VHDL provides a coherent view of system synthesis which includes the high-level and the system-level synthesis tasks. VHDL is used as a specification language and several issues concerning the use of VHDL for high-level and system-level synthesis are discussed. These include aspects from the compilation of VHDL into an internal design representation to the synthesis of systems specified as interacting VHDL processes. The book emphasizes the use of a transformational approach to system synthesis. A Petri net based design representation is rigorously defined and used throughout the book as a basic vehicle for illustration of transformations and other design concepts. Iterative improvement heuristics, such as tabu search, simulated annealing and genetic algorithms, are discussed and illustrated as strategies which are used to guide the optimization process in a transformation-based design environment. Advanced topics, including hardware/software partitioning, test synthesis and low power synthesis are discussed from the perspective of a transformational approach to system synthesis. System Synthesis with VHDL can be used for advanced undergraduate or graduate courses in the area of design automation and, more specifically, of high-level and system-level synthesis. At the same time the book is intended for CAD developers and researchers as well as industrial designers of digital systems who are interested in new algorithms and techniques supporting modern design tools and methodologies.

This edited volume comprises invited chapters that cover five areas of the current and the future development of intelligent systems and information sciences. Half of the chapters were presented as invited talks at the Workshop "Future Directions for Intelligent Systems and Information Sciences" held in Dunedin, New Zealand, 22-23 November 1999 after the International Conference on Neuro-Information Processing (lCONIPI ANZIISI ANNES '99) held in Perth, Australia. In order to make this volume useful for researchers and academics in the broad area of information sciences I invited prominent researchers to submit materials and present their view about future paradigms, future trends and directions. Part I contains chapters on adaptive, evolving, learning systems. These are systems that learn in a life-long, on-line mode and in a changing environment. The first chapter, written by the editor, presents briefly the paradigm of Evolving Connectionist Systems (ECOS) and some of their applications. The chapter by Sung-Bae Cho presents the paradigms of artificial life and evolutionary programming in the context of several applications (mobile robots, adaptive agents of the WWW). The following three chapters written by R.Duro, J.Santos and J.A.Becerra (chapter 3), GCoghill . (chapter 4), Y.Maeda (chapter 5) introduce new techniques for building adaptive, learning robots.

This volume presents examples of how ANNs are applied in biological sciences and related areas. Chapters focus on the analysis of intracellular sorting information, prediction of the behavior of bacterial communities, biometric authentication, studies of Tuberculosis, gene signatures in breast cancer classification, use of mass spectrometry in metabolite identification, visual navigation, and computer diagnosis. Written in the highly successful Methods in Molecular Biology series format, chapters include introductions to their respective topics, application details for both the expert and non-expert reader, and tips on troubleshooting and avoiding known pitfalls. Authoritative and practical, Artificial Neural Networks: Second Edition aids scientists in continuing to study Artificial Neural Networks (ANNs).

Among other topics, The Informational Complexity of Learning: Perspectives on Neural Networks and Generative Grammar brings together two important but very different learning problems within the same analytical framework. The first concerns the problem of learning functional mappings using neural networks, followed by learning natural language grammars in the principles and parameters tradition of Chomsky. These two learning problems are seemingly very different. Neural networks are real-valued, infinite-dimensional, continuous mappings. On the other hand, grammars are boolean-valued, finite-dimensional, discrete (symbolic) mappings. Furthermore the research communities that work in the two areas almost never overlap. The book's objective is to bridge this gap. It uses the formal techniques developed in statistical learning theory and theoretical computer science over the last decade to analyze both kinds of learning problems. By asking the same question - how much information does it take to learn? - of both problems, it highlights their similarities and differences. Specific results include model selection in neural networks, active learning, language learning and evolutionary models of language change. The Informational Complexity of Learning: Perspectives on Neural Networks and Generative Grammar is a very interdisciplinary work. Anyone interested in the interaction of computer science and cognitive science should enjoy the book. Researchers in artificial intelligence, neural networks, linguistics, theoretical computer science, and statistics will find it particularly relevant.

In recent years, spatial analysis has become an increasingly active field, as evidenced by the establishment of educational and research programs at many universities. Its popularity is due mainly to new technologies and the development of spatial data infrastructures. This book illustrates some recent developments in spatial analysis, behavioural modelling, and computational intelligence. World renown spatial analysts explain and demonstrate their new and insightful models and methods. The applications are in areas of societal interest such as the spread of infectious diseases, migration behaviour, and retail and agricultural location strategies. In addition, there is emphasis on the uses of new technologoies for the analysis of spatial data through the application of neural network concepts.

Neural Network Parallel Computing is the first book available to the professional market on neural network computing for optimization problems. This introductory book is not only for the novice reader, but for experts in a variety of areas including parallel computing, neural network computing, computer science, communications, graph theory, computer aided design for VLSI circuits, molecular biology, management science, and operations research. The goal of the book is to facilitate an understanding as to the uses of neural network models in real-world applications. Neural Network Parallel Computing presents a major breakthrough in science and a variety of engineering fields. The computational power of neural network computing is demonstrated by solving numerous problems such as N-queen, crossbar switch scheduling, four-coloring and k-colorability, graph planarization and channel routing, RNA secondary structure prediction, knight's tour, spare allocation, sorting and searching, and tiling. Neural Network Parallel Computing is an excellent reference for researchers in all areas covered by the book. Furthermore, the text may be used in a senior or graduate level course on the topic.

Both specialists and laymen will enjoy reading this book. Using a lively, non-technical style and images from everyday life, the authors present the basic principles behind computing and computers. The focus is on those aspects of computation that concern networks of numerous small computational units, whether biological neural networks or artificial electronic devices.

This monograph puts the reader in touch with a decade s worth of new developments in the field of fuzzy control specifically those of the popular Takagi Sugeno (T S) type. New techniques for stabilizing control analysis and design of arebased on multiple Lyapunov functions and linear matrix inequalities (LMIs). All the results are illustrated with numerical examples and figures and a rich bibliography is provided for further investigation.
Control saturations are taken into account within the fuzzy model. The concept of positive invariance is used to obtain sufficient conditions of asymptotic stability for the global fuzzy system with constrained control inside a subset of the state space.
The authors also consider the non-negativity of the states. This is of practical importance in many chemical, physical and biological processes that involve quantities that have intrinsically constant and non-negative sign: concentration of substances, level of liquids, etc. Results for linear systems are then extended to systems with delay. It is shown that LMI techniques can usually only handle the new constraint of non-negativity of the states when care is taken to use an adequate Lyapunov function. From these foundations, the following further problems are also treated: asymptotic stabilization of uncertain T-S fuzzy systems with time-varying delay, focusing on delay-dependent stabilization synthesis based on parallel distribution control;asymptotic stabilization of uncertain T-S fuzzy systems with multiple delays, focusing on delay-dependent stabilization synthesis based on parallel distribution control with results obtained under linear programming;design of delay-independent, observer-based, H-infinity control for T S fuzzy systems with time varying delay; andasymptotic stabilization of 2-d T S fuzzy systems.

"Advanced Takagi Sugeno Fuzzy Systems "provides researchers and graduate students interested in fuzzy control systems with further reliable means for maintaining stability and performance even when a sensor and/or actuator malfunctions."

This book presents a novel approach to neural nets and thus offers a genuine alternative to the hitherto known neuro-computers. The new edition includes a section on transformation properties of the equations of the synergetic computer and on the invariance properties of the order parameter equations. Further additions are a new section on stereopsis and recent developments in the use of pulse-coupled neural nets for pattern recognition.

Modern research in neural networks has led to powerful artificial learning systems, while recent work in the psychology of human memory has revealed much about how natural systems really learn, including the role of unconscious, implicit, memory processes. Regrettably, the two approaches typically ignore each other. This book, combining the approaches, should contribute to their mutual benefit. New empirical work is presented showing dissociations between implicit and explicit memory performance. Recently proposed explanations for such data lead to a new connectionist learning procedure: CALM (Categorizing and Learning Module), which can learn with or without supervision, and shows practical advantages over many existing procedures. Specific experiments are simulated by a network model (ELAN) composed of CALM modules. A working memory extension to the model is also discussed that could give it symbol manipulation abilities. The book will be of interest to memory psychologists and connectionists, as well as to cognitive scientists who in the past have tended to restrict themselves to symbolic models.

This book introduces readers to the fundamentals of artificial neural networks, with a special emphasis on evolutionary algorithms. At first, the book offers a literature review of several well-regarded evolutionary algorithms, including particle swarm and ant colony optimization, genetic algorithms and biogeography-based optimization. It then proposes evolutionary version of several types of neural networks such as feed forward neural networks, radial basis function networks, as well as recurrent neural networks and multi-later perceptron. Most of the challenges that have to be addressed when training artificial neural networks using evolutionary algorithms are discussed in detail. The book also demonstrates the application of the proposed algorithms for several purposes such as classification, clustering, approximation, and prediction problems. It provides a tutorial on how to design, adapt, and evaluate artificial neural networks as well, and includes source codes for most of the proposed techniques as supplementary materials.

In the last two decades the artificial neural networks have been refined and widely used by the researchers and application engineers. We have not witnessed such a large degree of evolution in any other artificial neural network as in the Adaptive Resonance Theory (ART) neural network. The ART network remains plastic, or adaptive, in response to significant events and yet remains stable in response to irrelevant events. This stability-plasticity property is a great step towards realizing intelligent machines capable of autonomous learning in real time environment.
The main aim of this book is to report a very small sample of the research on the evolution of ART neural network and its applications. Interested readers may refer literature for many more innovations in ART such as Fuzzy ART, ART2, ART2-a, ARTMAP, ARTMAP-PI, ARTMAP-DS, Gaussian ARTMAP, EXACT ART, and ART-EMAP.

This book introduces readers to the fundamentals of deep neural network architectures, with a special emphasis on memristor circuits and systems. At first, the book offers an overview of neuro-memristive systems, including memristor devices, models, and theory, as well as an introduction to deep learning neural networks such as multi-layer networks, convolution neural networks, hierarchical temporal memory, and long short term memories, and deep neuro-fuzzy networks. It then focuses on the design of these neural networks using memristor crossbar architectures in detail. The book integrates the theory with various applications of neuro-memristive circuits and systems. It provides an introductory tutorial on a range of issues in the design, evaluation techniques, and implementations of different deep neural network architectures with memristors.

Artificial neural networks possess several properties that make them particularly attractive for applications to modelling and control of complex non-linear systems. Among these properties are their universal approximation ability, their parallel network structure and the availability of on- and off-line learning methods for the interconnection weights. However, dynamic models that contain neural network architectures might be highly non-linear and difficult to analyse as a result. Artificial Neural Networks for Modelling and Control of Non-Linear Systems investigates the subject from a system theoretical point of view. However the mathematical theory that is required from the reader is limited to matrix calculus, basic analysis, differential equations and basic linear system theory. No preliminary knowledge of neural networks is explicitly required. The book presents both classical and novel network architectures and learning algorithms for modelling and control. Topics include non-linear system identification, neural optimal control, top-down model based neural control design and stability analysis of neural control systems. A major contribution of this book is to introduce NLq Theory as an extension towards modern control theory, in order to analyze and synthesize non-linear systems that contain linear together with static non-linear operators that satisfy a sector condition: neural state space control systems are an example. Moreover, it turns out that NLq Theory is unifying with respect to many problems arising in neural networks, systems and control. Examples show that complex non-linear systems can be modelled and controlled within NLq theory, including mastering chaos. The didactic flavor of this book makes it suitable for use as a text for a course on Neural Networks. In addition, researchers and designers will find many important new techniques, in particular NLq Theory, that have applications in control theory, system theory, circuit theory and Time Series Analysis.

The purpose of this book is to present an up to date account of fuzzy ideals of a semiring. The book concentrates on theoretical aspects and consists of eleven chapters including three invited chapters. Among the invited chapters, two are devoted to applications of Semirings to automata theory, and one deals with some generalizations of Semirings. This volume may serve as a useful hand book for graduate students and researchers in the areas of Mathematics and Theoretical Computer Science.

The primary purpose of this book is to present information about selected topics on the interactions and applications of fuzzy + neural. Most of the discussion centers around our own research in these areas. Fuzzy + neural can mean many things: (1) approximations between fuzzy systems and neu ral nets (Chapter 4); (2) building hybrid neural nets to equal fuzzy systems (Chapter 5); (3) using neura.l nets to solve fuzzy problems (Chapter 6); (4) approximations between fuzzy neural nets and other fuzzy systems (Chap ter 8); (5) constructing hybrid fuzzy neural nets for certain fuzzy systems (Chapters 9, 10); or (6) computing with words (Chapter 11). This book is not intend to be used primarily as a text book for a course in fuzzy + neural because we have not included problems at the end of each chapter, we have omitted most proofs (given in the references), and we have given very few references. We wanted to keep the mathematical prerequisites to a minimum so all longer, involved, proofs were omitted. Elementary dif ferential calculus is the only prerequisite needed since we do mention partial derivatives once or twice."

This volume contains the text of papers presented at the NATO Advanced Research Workshop on Emergent Computing Methods in Engineering Design, held in Nafplio, Greece, August 25-27, 1994. The workshop convened together some thirty or so researchers from Canada, France, Germany, Greece, Israel, Taiwan, The Netherlands, United Kingdom and the United States of America, to address issues related to the application of such emergent computing methods as genetic algorithms, neural networks and simulated annealing in problems of engineering design. The volume is essentially organized into three parts, with each part having some theoretical papers and other papers of a more practical nature. The frrst part, which comprises the largest number of papers, deals with genetic algorithms and evolutionary computing and presents subject matter ranging from proposed improvements to the computing methodology to specific applications in engineering design. The second part deals with neural networks and considers such topics as their application as approximation tools in design, their adaptation in control system design and theoretical issues of interpretation. The third part of the volume presents a collection of papers that examine such diverse topics as the combined use of genetic algorithms and neural networks, the application of simulated annealing techniques, problem decomposition techniques and the computer recognition and interpretation of emerging objects in engineering design.

This book develops applications of novel generalizations of fuzzy information measures in the field of pattern recognition, medical diagnosis, multi-criteria and multi-attribute decision making and suitability in linguistic variables. The focus of this presentation lies on introducing consistently strong and efficient generalizations of information and information-theoretic divergence measures in fuzzy and intuitionistic fuzzy environment covering different practical examples. The target audience comprises primarily researchers and practitioners in the involved fields but the book may also be beneficial for graduate students.

Principal Component Neural Networks Theory and Applications
Understanding the underlying principles of biological perceptual systems is of vital importance not only to neuroscientists, but, increasingly, to engineers and computer scientists who wish to develop artificial perceptual systems. In this original and groundbreaking work, the authors systematically examine the relationship between the powerful technique of Principal Component Analysis (PCA) and neural networks. Principal Component Neural Networks focuses on issues pertaining to both neural network models (i.e., network structures and algorithms) and theoretical extensions of PCA. In addition, it provides basic review material in mathematics and neurobiology. This book presents neural models originating from both the Hebbian learning rule and least squares learning rules, such as back-propagation. Its ultimate objective is to provide a synergistic exploration of the mathematical, algorithmic, application, and architectural aspects of principal component neural networks. Especially valuable to researchers and advanced students in neural network theory and signal processing, this book offers application examples from a variety of areas, including high-resolution spectral estimation, system identification, image compression, and pattern recognition.