From household appliances to applications in robotics, engineered systems involving complex dynamics can only be as effective as the algorithms that control them. While Dynamic Programming (DP) has provided researchers with a way to optimally solve decision and control problems involving complex dynamic systems, its practical value was limited by algorithms that lacked the capacity to scale up to realistic problems. However, in recent years, dramatic developments in Reinforcement Learning (RL), the model-free counterpart of DP, changed our understanding of what is possible. Those developments led to the creation of reliable methods that can be applied even when a mathematical model of the system is unavailable, allowing researchers to solve challenging control problems in engineering, as well as in a variety of other disciplines, including economics, medicine, and artificial intelligence. Reinforcement Learning and Dynamic Programming Using Function Approximators provides a comprehensive and unparalleled exploration of the field of RL and DP. With a focus on continuous-variable problems, this seminal text details essential developments that have substantially altered the field over the past decade. In its pages, pioneering experts provide a concise introduction to classical RL and DP, followed by an extensive presentation of the state-of-the-art and novel methods in RL and DP with approximation. Combining algorithm development with theoretical guarantees, they elaborate on their work with illustrative examples and insightful comparisons. Three individual chapters are dedicated to representative algorithms from each of the major classes of techniques: value iteration, policy iteration, and policy search. The features and performance of these algorithms are highlighted in extensive experimental studies on a range of control applications. The recent development of applications involving complex systems has led to a surge of interest in RL and DP methods and the subsequent need for a quality resource on the subject. For graduate students and others new to the field, this book offers a thorough introduction to both the basics and emerging methods. And for those researchers and practitioners working in the fields of optimal and adaptive control, machine learning, artificial intelligence, and operations research, this resource offers a combination of practical algorithms, theoretical analysis, and comprehensive examples that they will be able to adapt and apply to their own work. Access the authors' website at www.dcsc.tudelft.nl/rlbook/ for additional material, including computer code used in the studies and information concerning new developments.

Transient Stability of Power Systems is a monograph devoted to a hybrid-direct temporal method called SIME (for Single Machine Equivalent). SIME processes temporal information about the multimachine system dynamics to assess and control any type of transient instabilities under any type and model of power systems. Two approaches may be distinguished depending upon the source of information used: 'Preventative SIME' which relies on a time-domain program to simulate anticipated contingencies, and 'Emergency SIME' which uses real-time measurements. Preventative SIME mainly comprises two techniques: contingency filtering, ranking, and assessment; and (simultaneous) stabilization of harmful contingencies. The resulting preventative transient stability assessment and control (TSA&C) software can be used in all application contexts of transient stability studies. In a control center, for instance, its computational performances enable it to cope with very stringent requirements of real-time operation. Besides, interfacing SIME with an OPF algorithm allows combining transient stability constraints with specifics of the liberalized electricity market.Emergency SIME is a novel closed-loop control technique which contains the transient instabilities caused by contingencies' actual occurrence. It relies on real-time measurements to predict (the size of) instability and, accordingly, to design and trigger control actions able to impede system loss of synchronism. Emergency SIME is particularly suitable for protecting important generation sites and can complement preventative SIME. Both approaches rely on the same principles and basic software which yields a comprehensive and unified approach to TSA&C. The design of near optimal control techniques is a major asset of this software. This book provides extensive illustrations on a variety of power systems ranging from a simple 3-machine test system to real-world power systems comprising up to 627 generators and 4112 busses. Transient Stability of Power Systems will be especially helpful to researchers, utility engineers, and software designers and developers who are developing various types of transient stability software packages.

Transient Stability of Power Systems is a monograph devoted to a hybrid-direct temporal method called SIME (for Single Machine Equivalent). SIME processes temporal information about the multimachine system dynamics to assess and control any type of transient instabilities under any type and model of power systems. Two approaches may be distinguished depending upon the source of information used: Preventative SIME' which relies on a time-domain program to simulate anticipated contingencies, and Emergency SIME' which uses real-time measurements. Preventative SIME mainly comprises two techniques: contingency filtering, ranking, and assessment; and (simultaneous) stabilization of harmful contingencies. The resulting preventative transient stability assessment and control (TSA&C) software can be used in all application contexts of transient stability studies. In a control center, for instance, its computational performances enable it to cope with very stringent requirements of real-time operation. Besides, interfacing SIME with an OPF algorithm allows combining transient stability constraints with specifics of the liberalized electricity market. Emergency SIME is a novel closed-loop control technique which contains the transient instabilities caused by contingencies' actual occurrence. It relies on real-time measurements to predict (the size of) instability and, accordingly, to design and trigger control actions able to impede system loss of synchronism. Emergency SIME is particularly suitable for protecting important generation sites and can complement preventative SIME. Both approaches rely on the same principles and basic software which yields a comprehensive and unified approach toTSA&C. The design of near optimal control techniques is a major asset of this software. This book provides extensive illustrations on a variety of power systems ranging from a simple 3-machine test system to real-world power systems comprising up to 627 generators and 4112 busses. Transient Stability of Power Systems will be especially helpful to researchers, utility engineers, and software designers and developers who are developing various types of transient stability software packages.