A high school student can create deep Q-learning code to control her robot, without any understanding of the meaning of 'deep' or 'Q', or why the code sometimes fails. This book is designed to explain the science behind reinforcement learning and optimal control in a way that is accessible to students with a background in calculus and matrix algebra. A unique focus is algorithm design to obtain the fastest possible speed of convergence for learning algorithms, along with insight into why reinforcement learning sometimes fails. Advanced stochastic process theory is avoided at the start by substituting random exploration with more intuitive deterministic probing for learning. Once these ideas are understood, it is not difficult to master techniques rooted in stochastic control. These topics are covered in the second part of the book, starting with Markov chain theory and ending with a fresh look at actor-critic methods for reinforcement learning.

This volume consists of selected essays by participants of the workshop Control at Large Scales: Energy Markets and Responsive Grids held at the Institute for Mathematics and its Applications, Minneapolis, Minnesota, U.S.A. from May 9-13, 2016. The workshop brought together a diverse group of experts to discuss current and future challenges in energy markets and controls, along with potential solutions. The volume includes chapters on significant challenges in the design of markets and incentives, integration of renewable energy and energy storage, risk management and resilience, and distributed and multi-scale optimization and control. Contributors include leading experts from academia and industry in power systems and markets as well as control science and engineering. This volume will be of use to experts and newcomers interested in all aspects of the challenges facing the creation of a more sustainable electricity infrastructure, in areas such as distributed and stochastic optimization and control, stability theory, economics, policy, and financial mathematics, as well as in all aspects of power system operation.

Meyn & Tweedie is back The bible on Markov chains in general state spaces has been brought up to date to reflect developments in the field since 1996 - many of them sparked by publication of the first edition. The pursuit of more efficient simulation algorithms for complex Markovian models, or algorithms for computation of optimal policies for controlled Markov models, has opened new directions for research on Markov chains. As a result, new applications have emerged across a wide range of topics including optimisation, statistics, and economics. New commentary and an epilogue by Sean Meyn summarise recent developments and references have been fully updated. This second edition reflects the same discipline and style that marked out the original and helped it to become a classic: proofs are rigorous and concise, the range of applications is broad and knowledgeable, and key ideas are accessible to practitioners with limited mathematical background.

Power grids, flexible manufacturing, cellular communications: interconnectedness has consequences. This remarkable book gives the tools and philosophy you need to build network models detailed enough to capture essential dynamics but simple enough to expose the structure of effective control solutions and to clarify analysis. Core chapters assume only exposure to stochastic processes and linear algebra at the undergraduate level; later chapters are for advanced graduate students and researchers/practitioners. This gradual development bridges classical theory with the state-of-the-art. The workload model that is the basis of traditional analysis of the single queue becomes a foundation for workload relaxations used in the treatment of complex networks. Lyapunov functions and dynamic programming equations lead to the celebrated MaxWeight policy along with many generalizations. Other topics include methods for synthesizing hedging and safety stocks, stability theory for networks, and techniques for accelerated simulation. Examples and figures throughout make ideas concrete. Solutions to end-of-chapter exercises available on a companion website.

This volume consists of selected essays by participants of the workshop Control at Large Scales: Energy Markets and Responsive Grids held at the Institute for Mathematics and its Applications, Minneapolis, Minnesota, U.S.A. from May 9-13, 2016. The workshop brought together a diverse group of experts to discuss current and future challenges in energy markets and controls, along with potential solutions. The volume includes chapters on significant challenges in the design of markets and incentives, integration of renewable energy and energy storage, risk management and resilience, and distributed and multi-scale optimization and control. Contributors include leading experts from academia and industry in power systems and markets as well as control science and engineering. This volume will be of use to experts and newcomers interested in all aspects of the challenges facing the creation of a more sustainable electricity infrastructure, in areas such as distributed and stochastic optimization and control, stability theory, economics, policy, and financial mathematics, as well as in all aspects of power system operation.