The book presents a thorough development of the modern theory of stochastic approximation or recursive stochastic algorithms for both constrained and unconstrained problems. There is a complete development of both probability one and weak convergence methods for very general noise processes. The proofs of convergence use the ODE method, the most powerful to date, with which the asymptotic behavior is characterized by the limit behavior of a mean ODE. The assumptions and proof methods are designed to cover the needs of recent applications. The development proceeds from simple to complex problems, allowing the underlying ideas to be more easily understood. Rate of convergence, iterate averaging, high-dimensional problems, stability-ODE methods, two time scale, asynchronous and decentralized algorithms, general correlated and state-dependent noise, perturbed test function methods, and large devitations methods, are covered. Many motivational examples from learning theory, ergodic cost problems for discrete event systems, wireless communications, adaptive control, signal processing, and elsewhere, illustrate the application of the theory. This second edition is a thorough revision, although the main features and the structure remain unchanged. It contains many additional applications and results, and more detailed discussion. Harold J. Kushner is a University Professor and Professor of Applied Mathematics at Brown University. He has written numerous books and articles on virtually all aspects of stochastic systems theory, and has received various awards including the IEEE Control Systems Field Award.

This volume contains papers based on invited talks given at the 2005 IMA Summer Workshop on Wireless Communications, held at the Institute for Mathematics and Its Applications, University of Minnesota, June 22 - July 1, 2005. It presents some of the highlights of the workshop, and collects papers covering a broad spectrum of important and pressing issues in wireless communications.

A team of launched and coordinated Unmanned aerial vehicles (UAVs), requires advanced technologies in sensing, communication, computing, and control to improve their intelligence and robustness towards autonomous operations. To enhance reliability, robustness, and mission capability of a team of UAVs, a system-oriented and holistic approach is desirable in which all components and subsystems are considered in terms of their roles and impact on the entire system. This volume aims to summarize the recent progress, identify challenges and opportunities, and develop new methodologies and systems on coordinated UAV control. A group of experts working in this area have contributed to this volume in several related aspects of autonomous control of networked UAVs. Their papers introduce new control methodologies, algorithms, and systems that address several important issues in developing intelligent, autonomous or semi-autonomous, networked systems for the next generation of UAVs. The papers share a common focus on improved coordination of the members of the networked system to accomplish a common mission, to achieve heightened capability in system reconfiguration to compensate for lost members or connections, and to enhance robustness against terrain complications and attacks.

This book encompasses the study of hybrid switching di usion processes and their applications. The word \hybrid" signi es the coexistence of c- tinuous dynamics and discrete events, which is one of the distinct features of the processes under consideration. Much of the book is concerned with the interactions of the continuous dynamics and the discrete events. Our motivations for studying such processes originate from emerging and - isting applications in wireless communications, signal processing, queueing networks, production planning, biological systems, ecosystems, nancial engineering, and modeling, analysis, and control and optimization of lar- scale systems, under the in uence of random environments. Displaying mixture distributions, switching di usions may be described by the associated operators or by systems of stochastic di erential eq- tions together with the probability transition laws of the switching actions. We either have Markov-modulated switching di usions or processes with continuous state-dependent switching. The latter turns out to be much more challenging to deal with. Viewing the hybrid di usions as a number of di usions joined together by the switching process, they may be se- ingly not much di erent from their di usion counterpart. Nevertheless, the underlying problems become more di cult to handle, especially when the switching processes depend on continuous states. The di culty is due to the interaction of the discrete and continuous processes and the tangled and hybrid information pattern.

To honor Rafail Z. Khasminskii, on his seventy-fifth birthday, for his contributions to stochastic processes and nonparametric estimation theory an IMA participating institution conference entitled "Conference on Asymptotic Analysis in Stochastic Processes, Nonparametric Estimation, and Related Problems" was held. This volume commemorates this special event. Dedicated to Professor Khasminskii, it consists of nine papers on various topics in probability and statistics.

This book presents a thorough development of the modern theory of stochastic approximation or recursive stochastic algorithms for both constrained and unconstrained problems. This second edition is a thorough revision, although the main features and structure remain unchanged. It contains many additional applications and results as well as more detailed discussion.

This edited volume contains 16 research articles. It presents recent and pressing issues in stochastic processes, control theory, differential games, optimization, and their applications in finance, manufacturing, queueing networks, and climate control. One of the salient features is that the book is highly multi-disciplinary. The book is dedicated to Professor Suresh Sethi on the occasion of his 60th birthday, in view of his distinguished career.

This book focuses on two-time-scale Markov chains in discrete time. Our motivation stems from existing and emerging applications in optimization and control of complex systems in manufacturing, wireless communication, and ?nancial engineering. Much of our e?ort in this book is devoted to designing system models arising from various applications, analyzing them via analytic and probabilistic techniques, and developing feasible compu- tionalschemes. Ourmainconcernistoreducetheinherentsystemcompl- ity. Although each of the applications has its own distinct characteristics, all of them are closely related through the modeling of uncertainty due to jump or switching random processes. Oneofthesalientfeaturesofthisbookistheuseofmulti-timescalesin Markovprocessesandtheirapplications. Intuitively,notallpartsorcom- nents of a large-scale system evolve at the same rate. Some of them change rapidly and others vary slowly. The di?erent rates of variations allow us to reduce complexity via decomposition and aggregation. It would be ideal if we could divide a large system into its smallest irreducible subsystems completely separable from one another and treat each subsystem indep- dently. However, this is often infeasible in reality due to various physical constraints and other considerations. Thus, we have to deal with situations in which the systems are only nearly decomposable in the sense that there are weak links among the irreducible subsystems, which dictate the oc- sional regime changes of the system. An e?ective way to treat such near decomposability is time-scale separation. That is, we set up the systems as if there were two time scales, fast vs. slow. xii Preface Followingthetime-scaleseparation,weusesingularperturbationmeth- ology to treat the underlying systems.

This book encompasses the study of hybrid switching di usion processes and their applications. The word \hybrid" signi es the coexistence of c- tinuous dynamics and discrete events, which is one of the distinct features of the processes under consideration. Much of the book is concerned with the interactions of the continuous dynamics and the discrete events. Our motivations for studying such processes originate from emerging and - isting applications in wireless communications, signal processing, queueing networks, production planning, biological systems, ecosystems, nancial engineering, and modeling, analysis, and control and optimization of lar- scale systems, under the in uence of random environments. Displaying mixture distributions, switching di usions may be described by the associated operators or by systems of stochastic di erential eq- tions together with the probability transition laws of the switching actions. We either have Markov-modulated switching di usions or processes with continuous state-dependent switching. The latter turns out to be much more challenging to deal with. Viewing the hybrid di usions as a number of di usions joined together by the switching process, they may be se- ingly not much di erent from their di usion counterpart. Nevertheless, the underlying problems become more di cult to handle, especially when the switching processes depend on continuous states. The di culty is due to the interaction of the discrete and continuous processes and the tangled and hybrid information pattern.

This volume contains papers based on invited talks given at the 2005 IMA Summer Workshop on Wireless Communications, held at the Institute for Mathematics and Its Applications, University of Minnesota, June 22-July 1, 2005. The workshop provided a great opportunity to facilitate the communications between academia and the industry, and to bridge the mathematical sciences, engineering, information theory, and communication communities. The emphasis were on design and analysis of computationally efficient algorithms to better understand the behavior and to control the wireless telecommunication networks. As an achieve, this volume presents some of the highlights of the workshop, and collects papers covering a broad spectrum of important and pressing issues in wireless communications. All papers have been reviewed. One of the book's distinct features is highly multi-disciplinary. This book is useful for researchers and advanced graduate students working in communication networks, information theory, signal processing, and applied probability and stochastic processes, among others.

This edited volume contains sixteen research articles and presents recent and pressing issues in stochastic processes, control theory, differential games, optimization, and their applications in finance, manufacturing, queueing networks, and climate control. One of the salient features is that the book is highly multi-disciplinary. It assembles experts from the fields of operations research, control theory and optimization, stochastic analysis, and financial engineering to review and substantially update the recent progress in these fields. Another distinct characteristic of the book is that all papers are motivated by applications in which optimization, control, and stochastics are inseparable. The book will be a timely addition to the literature and will be of interest to people working in the aforementioned fields.

Most importantly, this volume is dedicated to Professor Suresh Sethi on the occasion of his 60th birthday. In view of his fundamental contributions, his distinguished career, his substantial achievements, his influence on the areas of control theory and applications, operations research, and management science, and his dedication to the scientific community, a number of leading experts in the fields of optimization, control, and operation management, have contributed to this volume in honor of him.

In view of Professor Wendell Fleming's many fundamental contributions, his profound influence on the mathematical and systems theory communi ties, his service to the profession, and his dedication to mathematics, we have invited a number of leading experts in the fields of control, optimiza tion, and stochastic systems to contribute to this volume in his honor on the occasion of his 70th birthday. These papers focus on various aspects of stochastic analysis, control theory and optimization, and applications. They include authoritative expositions and surveys as well as research papers on recent and important issues. The papers are grouped according to the following four major themes: (1) large deviations, risk sensitive and Hoc control, (2) partial differential equations and viscosity solutions, (3) stochastic control, filtering and parameter esti mation, and (4) mathematical finance and other applications. We express our deep gratitude to all of the authors for their invaluable contributions, and to the referees for their careful and timely reviews. We thank Harold Kushner for having graciously agreed to undertake the task of writing the foreword. Particular thanks go to H. Thomas Banks for his help, advice and suggestions during the entire preparation process, as well as for the generous support of the Center for Research in Scientific Computation. The assistance from the Birkhauser professional staff is also greatly appreciated."

In a competitive world, research in manufacturing systems plays an important role in creating, updating and improving the technologies and management practices of the economy. This volume presents some of the most recent results in stochastic manufacturing systems. Experts from the fields of applied mathematics, engineering and management sciences review and substantially update the recent advances in the control and optimization of manufacturing systems. Recent Advances in Control and Optimization of Manufacturing Systems consists of eight chapters divided into three parts which focus on Optimal Production Planning, Scheduling and Improvability and Approximate Optimality and Robustness. This book is intended for researchers and practitioners in the fields of systems theory, control and optimization, and operation management as well as in applied probability and stochastic processes.

This volume is a collection of research works to honor the late Professor Mark H.A. Davis, whose pioneering work in the areas of Stochastic Processes, Filtering, and Stochastic Optimization spans more than five decades. Invited authors include his dissertation advisor, past collaborators, colleagues, mentees, and graduate students of Professor Davis, as well as scholars who have worked in the above areas. Their contributions may expand upon topics in piecewise deterministic processes, pathwise stochastic calculus, martingale methods in stochastic optimization, filtering, mean-field games, time-inconsistency, as well as impulse, singular, risk-sensitive and robust stochastic control.

This volume collects papers, based on invited talks given at the IMA workshop in Modeling, Stochastic Control, Optimization, and Related Applications, held at the Institute for Mathematics and Its Applications, University of Minnesota, during May and June, 2018. There were four week-long workshops during the conference. They are (1) stochastic control, computation methods, and applications, (2) queueing theory and networked systems, (3) ecological and biological applications, and (4) finance and economics applications. For broader impacts, researchers from different fields covering both theoretically oriented and application intensive areas were invited to participate in the conference. It brought together researchers from multi-disciplinary communities in applied mathematics, applied probability, engineering, biology, ecology, and networked science, to review, and substantially update most recent progress. As an archive, this volume presents some of the highlights of the workshops, and collect papers covering a broad range of topics.

This volume collects papers, based on invited talks given at the IMA workshop in Modeling, Stochastic Control, Optimization, and Related Applications, held at the Institute for Mathematics and Its Applications, University of Minnesota, during May and June, 2018. There were four week-long workshops during the conference. They are (1) stochastic control, computation methods, and applications, (2) queueing theory and networked systems, (3) ecological and biological applications, and (4) finance and economics applications. For broader impacts, researchers from different fields covering both theoretically oriented and application intensive areas were invited to participate in the conference. It brought together researchers from multi-disciplinary communities in applied mathematics, applied probability, engineering, biology, ecology, and networked science, to review, and substantially update most recent progress. As an archive, this volume presents some of the highlights of the workshops, and collect papers covering a broad range of topics.

In view of Professor Wendell Fleming's many fundamental contributions, his profound influence on the mathematical and systems theory communi ties, his service to the profession, and his dedication to mathematics, we have invited a number of leading experts in the fields of control, optimiza tion, and stochastic systems to contribute to this volume in his honor on the occasion of his 70th birthday. These papers focus on various aspects of stochastic analysis, control theory and optimization, and applications. They include authoritative expositions and surveys as well as research papers on recent and important issues. The papers are grouped according to the following four major themes: (1) large deviations, risk sensitive and Hoc control, (2) partial differential equations and viscosity solutions, (3) stochastic control, filtering and parameter esti mation, and (4) mathematical finance and other applications. We express our deep gratitude to all of the authors for their invaluable contributions, and to the referees for their careful and timely reviews. We thank Harold Kushner for having graciously agreed to undertake the task of writing the foreword. Particular thanks go to H. Thomas Banks for his help, advice and suggestions during the entire preparation process, as well as for the generous support of the Center for Research in Scientific Computation. The assistance from the Birkhauser professional staff is also greatly appreciated."

This book presents recently developed methodologies that utilize quantized information in system identification and explores their potential in extending control capabilities for systems with limited sensor information or networked systems. The results of these methodologies can be applied to signal processing and control design of communication and computer networks, sensor networks, mobile agents, coordinated data fusion, remote sensing, telemedicine, and other fields in which noise-corrupted quantized data need to be processed.

System Identification with Quantized Observations is an excellent resource for graduate students, systems theorists, control engineers, applied mathematicians, as well as practitioners who use identification algorithms in their work.