From household appliances to applications in robotics, engineered systems involving complex dynamics can only be as effective as the algorithms that control them. While Dynamic Programming (DP) has provided researchers with a way to optimally solve decision and control problems involving complex dynamic systems, its practical value was limited by algorithms that lacked the capacity to scale up to realistic problems. However, in recent years, dramatic developments in Reinforcement Learning (RL), the model-free counterpart of DP, changed our understanding of what is possible. Those developments led to the creation of reliable methods that can be applied even when a mathematical model of the system is unavailable, allowing researchers to solve challenging control problems in engineering, as well as in a variety of other disciplines, including economics, medicine, and artificial intelligence. Reinforcement Learning and Dynamic Programming Using Function Approximators provides a comprehensive and unparalleled exploration of the field of RL and DP. With a focus on continuous-variable problems, this seminal text details essential developments that have substantially altered the field over the past decade. In its pages, pioneering experts provide a concise introduction to classical RL and DP, followed by an extensive presentation of the state-of-the-art and novel methods in RL and DP with approximation. Combining algorithm development with theoretical guarantees, they elaborate on their work with illustrative examples and insightful comparisons. Three individual chapters are dedicated to representative algorithms from each of the major classes of techniques: value iteration, policy iteration, and policy search. The features and performance of these algorithms are highlighted in extensive experimental studies on a range of control applications. The recent development of applications involving complex systems has led to a surge of interest in RL and DP methods and the subsequent need for a quality resource on the subject. For graduate students and others new to the field, this book offers a thorough introduction to both the basics and emerging methods. And for those researchers and practitioners working in the fields of optimal and adaptive control, machine learning, artificial intelligence, and operations research, this resource offers a combination of practical algorithms, theoretical analysis, and comprehensive examples that they will be able to adapt and apply to their own work. Access the authors' website at www.dcsc.tudelft.nl/rlbook/ for additional material, including computer code used in the studies and information concerning new developments.

Fuzzy Algorithms for Control gives an overview of the research results of a number of European research groups that are active and play a leading role in the field of fuzzy modeling and control. It contains 12 chapters divided into three parts. Chapters in the first part address the position of fuzzy systems in control engineering and in the AI community. State-of-the-art surveys on fuzzy modeling and control are presented along with a critical assessment of the role of these methodologists in control engineering. The second part is concerned with several analysis and design issues in fuzzy control systems. The analytical issues addressed include the algebraic representation of fuzzy models of different types, their approximation properties, and stability analysis of fuzzy control systems. Several design aspects are addressed, including performance specification for control systems in a fuzzy decision-making framework and complexity reduction in multivariable fuzzy systems. In the third part of the book, a number of applications of fuzzy control are presented. It is shown that fuzzy control in combination with other techniques such as fuzzy data analysis is an effective approach to the control of modern processes which present many challenges for the design of control systems. One has to cope with problems such as process nonlinearity, time-varying characteristics for incomplete process knowledge. Examples of real-world industrial applications presented in this book are a blast furnace, a lime kiln and a solar plant. Other examples of challenging problems in which fuzzy logic plays an important role and which are included in this book are mobile robotics and aircraft control. The aim of this book is to address both theoretical and practical subjects in a balanced way. It will therefore be useful for readers from the academic world and also from industry who want to apply fuzzy control in practice.

Rule-based fuzzy modeling has been recognised as a powerful technique for the modeling of partly-known nonlinear systems. Fuzzy models can effectively integrate information from different sources, such as physical laws, empirical models, measurements and heuristics. Application areas of fuzzy models include prediction, decision support, system analysis, control design, etc. Fuzzy Modeling for Control addresses fuzzy modeling from the systems and control engineering points of view. It focuses on the selection of appropriate model structures, on the acquisition of dynamic fuzzy models from process measurements (fuzzy identification), and on the design of nonlinear controllers based on fuzzy models. To automatically generate fuzzy models from measurements, a comprehensive methodology is developed which employs fuzzy clustering techniques to partition the available data into subsets characterized by locally linear behaviour. The relationships between the presented identification method and linear regression are exploited, allowing for the combination of fuzzy logic techniques with standard system identification tools. Attention is paid to the trade-off between the accuracy and transparency of the obtained fuzzy models. Control design based on a fuzzy model of a nonlinear dynamic process is addressed, using the concepts of model-based predictive control and internal model control with an inverted fuzzy model. To this end, methods to exactly invert specific types of fuzzy models are presented. In the context of predictive control, branch-and-bound optimization is applied. The main features of the presented techniques are illustrated by means of simple examples. In addition, three real-world applications are described. Finally, software tools for building fuzzy models from measurements are available from the author.

The increasing complexity of our world demands new perspectives on the role of technology in decision making. Human decision making has its li- tations in terms of information-processing capacity. We need new technology to cope with the increasingly complex and information-rich nature of our modern society. This is particularly true for critical environments such as crisis management and tra?c management, where humans need to engage in close collaborations with arti?cial systems to observe and understand the situation and respond in a sensible way. We believe that close collaborations between humans and arti?cial systems will become essential and that the importance of research into Interactive Collaborative Information Systems (ICIS) is self-evident. Developments in information and communication technology have ra- cally changed our working environments. The vast amount of information available nowadays and the wirelessly networked nature of our modern so- ety open up new opportunities to handle di?cult decision-making situations such as computer-supported situation assessment and distributed decision making. To make good use of these new possibilities, we need to update our traditional views on the role and capabilities of information systems. The aim of the Interactive Collaborative Information Systems project is to develop techniques that support humans in complex information en- ronments and that facilitate distributed decision-making capabilities. ICIS emphasizes the importance of building actor-agent communities: close c- laborations between human and arti?cial actors that highlight their comp- mentary capabilities, and in which task distribution is ?exible and adaptive.

Many problems in decision making, monitoring, fault detection, and control require the knowledge of state variables and time-varying parameters that are not directly measured by sensors. In such situations, observers, or estimators, can be employed that use the measured input and output signals along with a dynamic model of the system in order to estimate the unknown states or parameters. An essential requirement in designing an observer is to guarantee the convergence of the estimates to the true values or at least to a small neighborhood around the true values. However, for nonlinear, large-scale, or time-varying systems, the design and tuning of an observer is generally complicated and involves large computational costs. This book provides a range of methods and tools to design observers for nonlinear systems represented by a special type of a dynamic nonlinear model -- the Takagi--Sugeno (TS) fuzzy model. The TS model is a convex combination of affine linear models, which facilitates its stability analysis and observer design by using effective algorithms based on Lyapunov functions and linear matrix inequalities. Takagi--Sugeno models are known to be universal approximators and, in addition, a broad class of nonlinear systems can be exactly represented as a TS system. Three particular structures of large-scale TS models are considered: cascaded systems, distributed systems, and systems affected by unknown disturbances. The reader will find in-depth theoretic analysis accompanied by illustrative examples and simulations of real-world systems. Stability analysis of TS fuzzy systems is addressed in detail. The intended audience are graduate students and researchers both from academia and industry. For newcomers to the field, the book provides a concise introduction dynamic TS fuzzy models along with two methods to construct TS models for a given nonlinear system

The increasing complexity of our world demands new perspectives on the role of technology in decision making. Human decision making has its li- tations in terms of information-processing capacity. We need new technology to cope with the increasingly complex and information-rich nature of our modern society. This is particularly true for critical environments such as crisis management and tra?c management, where humans need to engage in close collaborations with arti?cial systems to observe and understand the situation and respond in a sensible way. We believe that close collaborations between humans and arti?cial systems will become essential and that the importance of research into Interactive Collaborative Information Systems (ICIS) is self-evident. Developments in information and communication technology have ra- cally changed our working environments. The vast amount of information available nowadays and the wirelessly networked nature of our modern so- ety open up new opportunities to handle di?cult decision-making situations such as computer-supported situation assessment and distributed decision making. To make good use of these new possibilities, we need to update our traditional views on the role and capabilities of information systems. The aim of the Interactive Collaborative Information Systems project is to develop techniques that support humans in complex information en- ronments and that facilitate distributed decision-making capabilities. ICIS emphasizes the importance of building actor-agent communities: close c- laborations between human and arti?cial actors that highlight their comp- mentary capabilities, and in which task distribution is ?exible and adaptive.

Fuzzy Algorithms for Control gives an overview of the research results of a number of European research groups that are active and play a leading role in the field of fuzzy modeling and control. It contains 12 chapters divided into three parts. Chapters in the first part address the position of fuzzy systems in control engineering and in the AI community. State-of-the-art surveys on fuzzy modeling and control are presented along with a critical assessment of the role of these methodologists in control engineering. The second part is concerned with several analysis and design issues in fuzzy control systems. The analytical issues addressed include the algebraic representation of fuzzy models of different types, their approximation properties, and stability analysis of fuzzy control systems. Several design aspects are addressed, including performance specification for control systems in a fuzzy decision-making framework and complexity reduction in multivariable fuzzy systems. In the third part of the book, a number of applications of fuzzy control are presented. It is shown that fuzzy control in combination with other techniques such as fuzzy data analysis is an effective approach to the control of modern processes which present many challenges for the design of control systems. One has to cope with problems such as process nonlinearity, time-varying characteristics for incomplete process knowledge. Examples of real-world industrial applications presented in this book are a blast furnace, a lime kiln and a solar plant. Other examples of challenging problems in which fuzzy logic plays an important role and which are included in this book are mobile robotics and aircraft control. The aim of this book is to address both theoretical and practical subjects in a balanced way. It will therefore be useful for readers from the academic world and also from industry who want to apply fuzzy control in practice.

Rule-based fuzzy modeling has been recognised as a powerful technique for the modeling of partly-known nonlinear systems. Fuzzy models can effectively integrate information from different sources, such as physical laws, empirical models, measurements and heuristics. Application areas of fuzzy models include prediction, decision support, system analysis, control design, etc. Fuzzy Modeling for Control addresses fuzzy modeling from the systems and control engineering points of view. It focuses on the selection of appropriate model structures, on the acquisition of dynamic fuzzy models from process measurements (fuzzy identification), and on the design of nonlinear controllers based on fuzzy models. To automatically generate fuzzy models from measurements, a comprehensive methodology is developed which employs fuzzy clustering techniques to partition the available data into subsets characterized by locally linear behaviour. The relationships between the presented identification method and linear regression are exploited, allowing for the combination of fuzzy logic techniques with standard system identification tools. Attention is paid to the trade-off between the accuracy and transparency of the obtained fuzzy models. Control design based on a fuzzy model of a nonlinear dynamic process is addressed, using the concepts of model-based predictive control and internal model control with an inverted fuzzy model. To this end, methods to exactly invert specific types of fuzzy models are presented. In the context of predictive control, branch-and-bound optimization is applied. The main features of the presented techniques are illustrated by means of simple examples. In addition, three real-world applications are described. Finally, software tools for building fuzzy models from measurements are available from the author.