Conventional digital computation methods have run into a se rious speed bottleneck due to their serial nature. To overcome this problem, a new computation model, called Neural Networks, has been proposed, which is based on some aspects of neurobiology and adapted to integrated circuits. The increased availability of com puting power has not only made many new applications possible but has also created the desire to perform cognitive tasks which are easily carried out by the human brain. It become obvious that new types of algorithms and/or circuits were necessary to cope with such tasks. Inspiration has been sought from the functioning of the hu man brain, which led to the artificial neural network approach. One way of looking at neural networks is to consider them to be arrays of nonlinear dynamical systems that interact with each other. This book deals with one class of locally coupled neural net works, called Cellular Neural Networks (CNNs). CNNs were intro duced in 1988 by L. O. Chua and L. Yang 27,28] as a novel class of information processing systems, which posseses some of the key fea tures of neural networks (NNs) and which has important potential applications in such areas as image processing and pattern reco gnition. Unfortunately, the highly interdisciplinary nature of the research in CNNs makes it very difficult for a newcomer to enter this important and fasciriating area of modern science."

Deep Network Design for Medical Image Computing: Principles and Applications covers a range of MIC tasks and discusses design principles of these tasks for deep learning approaches in medicine. These include skin disease classification, vertebrae identification and localization, cardiac ultrasound image segmentation, 2D/3D medical image registration for intervention, metal artifact reduction, sparse-view artifact reduction, etc. For each topic, the book provides a deep learning-based solution that takes into account the medical or biological aspect of the problem and how the solution addresses a variety of important questions surrounding architecture, the design of deep learning techniques, when to introduce adversarial learning, and more. This book will help graduate students and researchers develop a better understanding of the deep learning design principles for MIC and to apply them to their medical problems.

In recent years, considerable progress has been made in the understanding of problems of learning and generalization. In this context, intelligence basically means the ability to perform well on new data after learning a model on the basis of given data. Such problems arise in many different areas and are becoming increasingly important and crucial towards many applications such as in bioinformatics, multimedia, computer vision and signal processing, internet search and information retrieval, datamining and textmining, finance, fraud detection, measurement systems, process control and several others. Currently, the development of new technologies enables to generate massive amounts of data containing a wealth of information that remains to become explored. Often the dimensionality of the input spaces in these novel applications is huge. This can be seen in the analysis of micro-array data, for example, where expression levels of thousands of genes need to be analyzed given only a limited number of experiments. Without performing dimensionality reduction, the classical statistical paradigms show fundamental shortcomings at this point. Facing these new challenges, there is a need for new mathematical foundations and models in a way that the data can become processed in a reliable way. The subjects in this publication are very interdisciplinary and relate to problems studied in neural networks, machine learning, mathematics and statistics.

In a world of almost permanent and rapidly increasing electronic data availability, techniques of filtering, compressing, and interpreting this data to transform it into valuable and easily comprehensible information is of utmost importance. One key topic in this area is the capability to deduce future system behavior from a given data input. This book brings together for the first time the complete theory of data-based neurofuzzy modelling and the linguistic attributes of fuzzy logic in a single cohesive mathematical framework. After introducing the basic theory of data-based modelling, new concepts including extended additive and multiplicative submodels are developed and their extensions to state estimation and data fusion are derived. All these algorithms are illustrated with benchmark and real-life examples to demonstrate their efficiency. Chris Harris and his group have carried out pioneering work which has tied together the fields of neural networks and linguistic rule-based algortihms. This book is aimed at researchers and scientists in time series modeling, empirical data modeling, knowledge discovery, data mining, and data fusion.

Applications of Soft Computing have recently increased and methodological development has been strong. The book is a collection of new interesting industrial applications introduced by several research groups and industrial partners. It describes the principles and results of industrial applications of Soft Computing methods and introduces new possibilities to gain technical and economic benefits by using this methodology. The book shows how fuzzy logic and neural networks have been used in the Finnish paper and metallurgical industries putting emphasis on processes, applications and technical and economic results.

Nonlinear Modeling: Advanced Black-Box Techniques discusses methods on Neural nets and related model structures for nonlinear system identification; Enhanced multi-stream Kalman filter training for recurrent networks; The support vector method of function estimation; Parametric density estimation for the classification of acoustic feature vectors in speech recognition; Wavelet-based modeling of nonlinear systems; Nonlinear identification based on fuzzy models; Statistical learning in control and matrix theory; Nonlinear time-series analysis. It also contains the results of the K.U. Leuven time series prediction competition, held within the framework of an international workshop at the K.U. Leuven, Belgium in July 1998.

Recent years have seen many new developments in computational intelligence (CI) techniques and, consequently, this has led to an exponential increase in the number of applications in a variety of areas, including: engineering, finance, social and biomedical. In particular, CI techniques are increasingly being used in biomedical and human movement areas because of the complexity of the biological systems, as well as the limitations of the existing quantitative techniques in modelling. ""Computational Intelligence for Movement Sciences: Neural Networks and Other Emerging Techniques"" contains information regarding state-of-the-art research outcomes and cutting-edge technology from leading scientists and researchers working on various aspects of the human movement. Readers of this book will gain an insight into this field, as well as access to pertinent information, which they will be able to use for continuing research in this area.

Applications of some selected soft computing methods to acoustics and sound engineering are presented in this book. The aim of this research study is the implementation of soft computing methods to musical signal analysis and to the recognition of musical sounds and phrases. Accordingly, some methods based on such learning algorithms as neural networks, rough sets and fuzzy-logic were conceived, implemented and tested. Additionally, the above-mentioned methods were applied to the analysis and verification of subjective testing results. The last problem discussed within the framework of this book was the problem of fuzzy control of the classical pipe organ instrument.
The obtained results show that computational intelligence and soft computing may be used for solving some vital problems in both musical and architectural acoustics.

This volume is the first diverse and comprehensive treatment of algorithms and architectures for the realization of neural network systems. It presents techniques and diverse methods in numerous areas of this broad subject. The book covers major neural network systems structures for achieving effective systems, and illustrates them with examples.
This volume includes Radial Basis Function networks, the Expand-and-Truncate Learning algorithm for the synthesis of Three-Layer Threshold Networks, weight initialization, fast and efficient variants of Hamming and Hopfield neural networks, discrete time synchronous multilevel neural systems with reduced VLSI demands, probabilistic design techniques, time-based techniques, techniques for reducing physical realization requirements, and applications to finite constraint problems.
A unique and comprehensive reference for a broad array of algorithms and architectures, this book will be of use to practitioners, researchers, and students in industrial, manufacturing, electrical, and mechanical engineering, as well as in computer science and engineering.
Key Features
* Radial Basis Function networks
* The Expand-and-Truncate Learning algorithm for the synthesis of Three-Layer Threshold Networks
* Weight initialization
* Fast and efficient variants of Hamming and Hopfield neural networks
* Discrete time synchronous multilevel neural systems with reduced VLSI demands
* Probabilistic design techniques
* Time-based techniques
* Techniques for reducing physical realization requirements
* Applications to finite constraint problems
* Practical realization methods for Hebbian type associative memory systems
*Parallel self-organizing hierarchical neural network systems
* Dynamics of networks of biological neurons for utilization in computational neuroscience
Practitioners, researchers, and students in industrial, manufacturing, electrical, and mechanical engineering, as well as in computer science and engineering, will find this volume a unique and comprehensive reference to a broad array of algorithms and architectures

Based on a NATO Advanced Study Institute held in 1993, this book addresses recent advances in automatic speech recognition and speech coding. The book contains contributions by many of the most outstanding researchers from the best laboratories worldwide in the field. The contributions have been grouped into five parts: on acoustic modeling; language modeling; speech processing, analysis and synthesis; speech coding; and vector quantization and neural nets. For each of these topics, some of the best-known researchers were invited to give a lecture. In addition to these lectures, the topics were complemented with discussions and presentations of the work of those attending. Altogether, the reader is given a wide perspective on recent advances in the field and will be able to see the trends for future work.

The subject of this book is predictive modular neural networks and their ap plication to time series problems: classification, prediction and identification. The intended audience is researchers and graduate students in the fields of neural networks, computer science, statistical pattern recognition, statistics, control theory and econometrics. Biologists, neurophysiologists and medical engineers may also find this book interesting. In the last decade the neural networks community has shown intense interest in both modular methods and time series problems. Similar interest has been expressed for many years in other fields as well, most notably in statistics, control theory, econometrics etc. There is a considerable overlap (not always recognized) of ideas and methods between these fields. Modular neural networks come by many other names, for instance multiple models, local models and mixtures of experts. The basic idea is to independently develop several "subnetworks" (modules), which may perform the same or re lated tasks, and then use an "appropriate" method for combining the outputs of the subnetworks. Some of the expected advantages of this approach (when compared with the use of "lumped" or "monolithic" networks) are: superior performance, reduced development time and greater flexibility. For instance, if a module is removed from the network and replaced by a new module (which may perform the same task more efficiently), it should not be necessary to retrain the aggregate network."

The primary purpose of this book is to show that a multilayer neural network can be considered as a multistage system, and then that the learning of this class of neural networks can be treated as a special sort of the optimal control problem. In this way, the optimal control problem methodology, like dynamic programming, with modifications, can yield a new class of learning algorithms for multilayer neural networks.

Another purpose of this book is to show that the generalized net theory can be successfully used as a new description of multilayer neural networks. Several generalized net descriptions of neural networks functioning processes are considered, namely: the simulation process of networks, a system of neural networks and the learning algorithms developed in this book.

The generalized net approach to modelling of real systems may be used successfully for the description of a variety of technological and intellectual problems, it can be used not only for representing the parallel functioning of homogenous objects, but also for modelling non-homogenous systems, for example systems which consist of a different kind of subsystems.

The use of the generalized nets methodology shows a new way to describe functioning of discrete dynamic systems.

"

In this book the author presents a new approach to the study of weakly structurable dynamic systems. It differs from other approaches by considering time as a source of fuzzy uncertainty in dynamic systems. It begins with a thorough introduction, where the general research domain, the problems, and ways of their solutions are discussed. The book then progresses systematically by first covering the theoretical aspects before tackling the applications. In the application section, a software library is described, which contains discrete EFDS identification methods elaborated during fundamental research of the book.

"Extremal Fuzzy Dynamic Systems" will be of interest to theoreticians interested in modeling fuzzy processes, to researchers who use fuzzy statistics, as well as practitioners from different disciplines whose research interests include abnormal, extreme and monotone processes in nature and society. Graduate students could also find this book useful.

Frontiers of Higher Order Fuzzy Sets, provides a unified representation theorem for higher order fuzzy sets. The book elaborates on the concept of gradual elements and their integration with the higher order fuzzy sets. This book also is devoted to the introduction of new frameworks based on general T2FSs, IT2FSs, Gradual elements, Shadowed sets and rough sets. Such new frameworks will provide more capable frameworks for real applications. Applications of higher order fuzzy sets in various fields will be discussed. In particular, the properties and characteristics of the new proposed frameworks would be studied. Such frameworks that are the result of the integration of general T2FSs, IT2FSs, gradual elements, shadowed sets and rough sets will be shown to be suitable to be applied in the fields of bioinformatics, business, management, ambient intelligence, medicine, cloud computing and smart grids.

Complex-Valued Neural Networks have higher functionality, learn faster and generalize better than their real-valued counterparts. This book is devoted to the Multi-Valued Neuron (MVN) and MVN-based neural networks. It contains a comprehensive observation of MVN theory, its learning, and applications. MVN is a complex-valued neuron whose inputs and output are located on the unit circle. Its activation function is a function only of argument (phase) of the weighted sum. MVN derivative-free learning is based on the error-correction rule. A single MVN can learn those input/output mappings that are non-linearly separable in the real domain. Such classical non-linearly separable problems as XOR and Parity n are the simplest that can be learned by a single MVN. Another important advantage of MVN is a proper treatment of the phase information. These properties of MVN become even more remarkable when this neuron is used as a basic one in neural networks. The Multilayer Neural Network based on Multi-Valued Neurons (MLMVN) is an MVN-based feedforward neural network. Its backpropagation learning algorithm is derivative-free and based on the error-correction rule. It does not suffer from the local minima phenomenon. MLMVN outperforms many other machine learning techniques in terms of learning speed, network complexity and generalization capability when solving both benchmark and real-world classification and prediction problems. Another interesting application of MVN is its use as a basic neuron in multi-state associative memories. The book is addressed to those readers who develop theoretical fundamentals of neural networks and use neural networks for solving various real-world problems. It should also be very suitable for Ph.D. and graduate students pursuing their degrees in computational intelligence.

Connectionist Speech Recognition: A Hybrid Approach describes the theory and implementation of a method to incorporate neural network approaches into state of the art continuous speech recognition systems based on hidden Markov models (HMMs) to improve their performance. In this framework, neural networks (and in particular, multilayer perceptrons or MLPs) have been restricted to well-defined subtasks of the whole system, i.e. HMM emission probability estimation and feature extraction. The book describes a successful five-year international collaboration between the authors. The lessons learned form a case study that demonstrates how hybrid systems can be developed to combine neural networks with more traditional statistical approaches. The book illustrates both the advantages and limitations of neural networks in the framework of a statistical systems. Using standard databases and comparison with some conventional approaches, it is shown that MLP probability estimation can improve recognition performance. Other approaches are discussed, though there is no such unequivocal experimental result for these methods. Connectionist Speech Recognition is of use to anyone intending to use neural networks for speech recognition or within the framework provided by an existing successful statistical approach. This includes research and development groups working in the field of speech recognition, both with standard and neural network approaches, as well as other pattern recognition and/or neural network researchers. The book is also suitable as a text for advanced courses on neural networks or speech processing.

The recent interest in artificial neural networks has motivated the publication of numerous books, including selections of research papers and textbooks presenting the most popular neural architectures and learning schemes. Artificial Neural Networks: Learning Algorithms, Performance Evaluation, and Applications presents recent developments which can have a very significant impact on neural network research, in addition to the selective review of the existing vast literature on artificial neural networks. This book can be read in different ways, depending on the background, the specialization, and the ultimate goals of the reader. A specialist will find in this book well-defined and easily reproducible algorithms, along with the performance evaluation of various neural network architectures and training schemes. Artificial Neural Networks can also help a beginner interested in the development of neural network systems to build the necessary background in an organized and comprehensive way. The presentation of the material in this book is based on the belief that the successful application of neural networks to real-world problems depends strongly on the knowledge of their learning properties and performance. Neural networks are introduced as trainable devices which have the unique ability to generalize. The pioneering work on neural networks which appeared during the past decades is presented, together with the current developments in the field, through a comprehensive and unified review of the most popular neural network architectures and learning schemes. Efficient LEarning Algorithms for Neural NEtworks (ELEANNE), which can achieve much faster convergence than existing learningalgorithms, are among the recent developments explored in this book. A new generalized criterion for the training of neural networks is presented, which leads to a variety of fast learning algorithms. Finally, Artificial Neural Networks presents the development of learning algorithms which determine the minimal architecture of multi-layered neural networks while performing their training. Artificial Neural Networks is a valuable source of information to all researchers and engineers interested in neural networks. The book may also be used as a text for an advanced course on the subject.

Learning and Generalization provides a formal mathematical theory for addressing intuitive questions of the type: * How does a machine learn a new concept on the basis of examples? * How can a neural network, after sufficient training, correctly predict the outcome of a previously unseen input? * How much training is required to achieve a specified level of accuracy in the prediction? * How can one identify the dynamical behaviour of a nonlinear control system by observing its input-output behaviour over a finite interval of time? The first edition, A Theory of Learning and Generalization, was the first book to treat the problem of machine learning in conjunction with the theory of empirical process, the latter being a well-established branch of probability theory. The treatment of both topics side-by-side leads to new insights, as well as new results in both topics. The second edition extends and improves upon this material, covering new areas including: * Support vector machines (SVM's) * Fat-shattering dimensions and applications to neural network learning * Learning with dependent samples generated by a beta-mixing process * Connections between system identification and learning theory * Probabilistic solution of 'intractable problems' in robust control and matrix theory using randomized algorithms It also contains solutions to some of the open problems posed in the first edition, while adding new open problems. This book is essential reading for control and system theorists, neural network researchers, theoretical computer scientists and probabilists The Communications and Control Engineering series reflects the major technological advances which have a great impact in the fields of communication and control. It reports on the research in industrial and academic institutions around the world to exploit the new possibilities which are becoming available

This volume presents examples of how ANNs are applied in biological sciences and related areas. Chapters focus on the analysis of intracellular sorting information, prediction of the behavior of bacterial communities, biometric authentication, studies of Tuberculosis, gene signatures in breast cancer classification, use of mass spectrometry in metabolite identification, visual navigation, and computer diagnosis. Written in the highly successful Methods in Molecular Biology series format, chapters include introductions to their respective topics, application details for both the expert and non-expert reader, and tips on troubleshooting and avoiding known pitfalls. Authoritative and practical, Artificial Neural Networks: Second Edition aids scientists in continuing to study Artificial Neural Networks (ANNs).

Embedded systems are usually composed of several interacting components such as custom or application specific processors, ASICs, memory blocks, and the associated communication infrastructure. The development of tools to support the design of such systems requires a further step from high-level synthesis towards a higher abstraction level. The lack of design tools accepting a system-level specification of a complete system, which may include both hardware and software components, is one of the major bottlenecks in the design of embedded systems. Thus, more and more research efforts have been spent on issues related to system-level synthesis. This book addresses the two most active research areas of design automation today: high-level synthesis and system-level synthesis. In particular, a transformational approach to synthesis from VHDL specifications is described. System Synthesis with VHDL provides a coherent view of system synthesis which includes the high-level and the system-level synthesis tasks. VHDL is used as a specification language and several issues concerning the use of VHDL for high-level and system-level synthesis are discussed. These include aspects from the compilation of VHDL into an internal design representation to the synthesis of systems specified as interacting VHDL processes. The book emphasizes the use of a transformational approach to system synthesis. A Petri net based design representation is rigorously defined and used throughout the book as a basic vehicle for illustration of transformations and other design concepts. Iterative improvement heuristics, such as tabu search, simulated annealing and genetic algorithms, are discussed and illustrated as strategies which are used to guide the optimization process in a transformation-based design environment. Advanced topics, including hardware/software partitioning, test synthesis and low power synthesis are discussed from the perspective of a transformational approach to system synthesis. System Synthesis with VHDL can be used for advanced undergraduate or graduate courses in the area of design automation and, more specifically, of high-level and system-level synthesis. At the same time the book is intended for CAD developers and researchers as well as industrial designers of digital systems who are interested in new algorithms and techniques supporting modern design tools and methodologies.

This edited volume comprises invited chapters that cover five areas of the current and the future development of intelligent systems and information sciences. Half of the chapters were presented as invited talks at the Workshop "Future Directions for Intelligent Systems and Information Sciences" held in Dunedin, New Zealand, 22-23 November 1999 after the International Conference on Neuro-Information Processing (lCONIPI ANZIISI ANNES '99) held in Perth, Australia. In order to make this volume useful for researchers and academics in the broad area of information sciences I invited prominent researchers to submit materials and present their view about future paradigms, future trends and directions. Part I contains chapters on adaptive, evolving, learning systems. These are systems that learn in a life-long, on-line mode and in a changing environment. The first chapter, written by the editor, presents briefly the paradigm of Evolving Connectionist Systems (ECOS) and some of their applications. The chapter by Sung-Bae Cho presents the paradigms of artificial life and evolutionary programming in the context of several applications (mobile robots, adaptive agents of the WWW). The following three chapters written by R.Duro, J.Santos and J.A.Becerra (chapter 3), GCoghill . (chapter 4), Y.Maeda (chapter 5) introduce new techniques for building adaptive, learning robots.

Among other topics, The Informational Complexity of Learning: Perspectives on Neural Networks and Generative Grammar brings together two important but very different learning problems within the same analytical framework. The first concerns the problem of learning functional mappings using neural networks, followed by learning natural language grammars in the principles and parameters tradition of Chomsky. These two learning problems are seemingly very different. Neural networks are real-valued, infinite-dimensional, continuous mappings. On the other hand, grammars are boolean-valued, finite-dimensional, discrete (symbolic) mappings. Furthermore the research communities that work in the two areas almost never overlap. The book's objective is to bridge this gap. It uses the formal techniques developed in statistical learning theory and theoretical computer science over the last decade to analyze both kinds of learning problems. By asking the same question - how much information does it take to learn? - of both problems, it highlights their similarities and differences. Specific results include model selection in neural networks, active learning, language learning and evolutionary models of language change. The Informational Complexity of Learning: Perspectives on Neural Networks and Generative Grammar is a very interdisciplinary work. Anyone interested in the interaction of computer science and cognitive science should enjoy the book. Researchers in artificial intelligence, neural networks, linguistics, theoretical computer science, and statistics will find it particularly relevant.

This monograph puts the reader in touch with a decade s worth of new developments in the field of fuzzy control specifically those of the popular Takagi Sugeno (T S) type. New techniques for stabilizing control analysis and design of arebased on multiple Lyapunov functions and linear matrix inequalities (LMIs). All the results are illustrated with numerical examples and figures and a rich bibliography is provided for further investigation.
Control saturations are taken into account within the fuzzy model. The concept of positive invariance is used to obtain sufficient conditions of asymptotic stability for the global fuzzy system with constrained control inside a subset of the state space.
The authors also consider the non-negativity of the states. This is of practical importance in many chemical, physical and biological processes that involve quantities that have intrinsically constant and non-negative sign: concentration of substances, level of liquids, etc. Results for linear systems are then extended to systems with delay. It is shown that LMI techniques can usually only handle the new constraint of non-negativity of the states when care is taken to use an adequate Lyapunov function. From these foundations, the following further problems are also treated: asymptotic stabilization of uncertain T-S fuzzy systems with time-varying delay, focusing on delay-dependent stabilization synthesis based on parallel distribution control;asymptotic stabilization of uncertain T-S fuzzy systems with multiple delays, focusing on delay-dependent stabilization synthesis based on parallel distribution control with results obtained under linear programming;design of delay-independent, observer-based, H-infinity control for T S fuzzy systems with time varying delay; andasymptotic stabilization of 2-d T S fuzzy systems.

"Advanced Takagi Sugeno Fuzzy Systems "provides researchers and graduate students interested in fuzzy control systems with further reliable means for maintaining stability and performance even when a sensor and/or actuator malfunctions."

In recent years, spatial analysis has become an increasingly active field, as evidenced by the establishment of educational and research programs at many universities. Its popularity is due mainly to new technologies and the development of spatial data infrastructures. This book illustrates some recent developments in spatial analysis, behavioural modelling, and computational intelligence. World renown spatial analysts explain and demonstrate their new and insightful models and methods. The applications are in areas of societal interest such as the spread of infectious diseases, migration behaviour, and retail and agricultural location strategies. In addition, there is emphasis on the uses of new technologoies for the analysis of spatial data through the application of neural network concepts.