The book presents a thorough development of the modern theory of stochastic approximation or recursive stochastic algorithms for both constrained and unconstrained problems. There is a complete development of both probability one and weak convergence methods for very general noise processes. The proofs of convergence use the ODE method, the most powerful to date, with which the asymptotic behavior is characterized by the limit behavior of a mean ODE. The assumptions and proof methods are designed to cover the needs of recent applications. The development proceeds from simple to complex problems, allowing the underlying ideas to be more easily understood. Rate of convergence, iterate averaging, high-dimensional problems, stability-ODE methods, two time scale, asynchronous and decentralized algorithms, general correlated and state-dependent noise, perturbed test function methods, and large devitations methods, are covered. Many motivational examples from learning theory, ergodic cost problems for discrete event systems, wireless communications, adaptive control, signal processing, and elsewhere, illustrate the application of the theory. This second edition is a thorough revision, although the main features and the structure remain unchanged. It contains many additional applications and results, and more detailed discussion. Harold J. Kushner is a University Professor and Professor of Applied Mathematics at Brown University. He has written numerous books and articles on virtually all aspects of stochastic systems theory, and has received various awards including the IEEE Control Systems Field Award.

This volume contains papers based on invited talks given at the 2005 IMA Summer Workshop on Wireless Communications, held at the Institute for Mathematics and Its Applications, University of Minnesota, June 22 - July 1, 2005. It presents some of the highlights of the workshop, and collects papers covering a broad spectrum of important and pressing issues in wireless communications.

This book encompasses the study of hybrid switching di usion processes and their applications. The word \hybrid" signi es the coexistence of c- tinuous dynamics and discrete events, which is one of the distinct features of the processes under consideration. Much of the book is concerned with the interactions of the continuous dynamics and the discrete events. Our motivations for studying such processes originate from emerging and - isting applications in wireless communications, signal processing, queueing networks, production planning, biological systems, ecosystems, nancial engineering, and modeling, analysis, and control and optimization of lar- scale systems, under the in uence of random environments. Displaying mixture distributions, switching di usions may be described by the associated operators or by systems of stochastic di erential eq- tions together with the probability transition laws of the switching actions. We either have Markov-modulated switching di usions or processes with continuous state-dependent switching. The latter turns out to be much more challenging to deal with. Viewing the hybrid di usions as a number of di usions joined together by the switching process, they may be se- ingly not much di erent from their di usion counterpart. Nevertheless, the underlying problems become more di cult to handle, especially when the switching processes depend on continuous states. The di culty is due to the interaction of the discrete and continuous processes and the tangled and hybrid information pattern.

To honor Rafail Z. Khasminskii, on his seventy-fifth birthday, for his contributions to stochastic processes and nonparametric estimation theory an IMA participating institution conference entitled "Conference on Asymptotic Analysis in Stochastic Processes, Nonparametric Estimation, and Related Problems" was held. This volume commemorates this special event. Dedicated to Professor Khasminskii, it consists of nine papers on various topics in probability and statistics.

This book presents a thorough development of the modern theory of stochastic approximation or recursive stochastic algorithms for both constrained and unconstrained problems. This second edition is a thorough revision, although the main features and structure remain unchanged. It contains many additional applications and results as well as more detailed discussion.

This edited volume contains 16 research articles. It presents recent and pressing issues in stochastic processes, control theory, differential games, optimization, and their applications in finance, manufacturing, queueing networks, and climate control. One of the salient features is that the book is highly multi-disciplinary. The book is dedicated to Professor Suresh Sethi on the occasion of his 60th birthday, in view of his distinguished career.

This book focuses on two-time-scale Markov chains in discrete time. Our motivation stems from existing and emerging applications in optimization and control of complex systems in manufacturing, wireless communication, and ?nancial engineering. Much of our e?ort in this book is devoted to designing system models arising from various applications, analyzing them via analytic and probabilistic techniques, and developing feasible compu- tionalschemes. Ourmainconcernistoreducetheinherentsystemcompl- ity. Although each of the applications has its own distinct characteristics, all of them are closely related through the modeling of uncertainty due to jump or switching random processes. Oneofthesalientfeaturesofthisbookistheuseofmulti-timescalesin Markovprocessesandtheirapplications. Intuitively,notallpartsorcom- nents of a large-scale system evolve at the same rate. Some of them change rapidly and others vary slowly. The di?erent rates of variations allow us to reduce complexity via decomposition and aggregation. It would be ideal if we could divide a large system into its smallest irreducible subsystems completely separable from one another and treat each subsystem indep- dently. However, this is often infeasible in reality due to various physical constraints and other considerations. Thus, we have to deal with situations in which the systems are only nearly decomposable in the sense that there are weak links among the irreducible subsystems, which dictate the oc- sional regime changes of the system. An e?ective way to treat such near decomposability is time-scale separation. That is, we set up the systems as if there were two time scales, fast vs. slow. xii Preface Followingthetime-scaleseparation,weusesingularperturbationmeth- ology to treat the underlying systems.

This book encompasses the study of hybrid switching di usion processes and their applications. The word \hybrid" signi es the coexistence of c- tinuous dynamics and discrete events, which is one of the distinct features of the processes under consideration. Much of the book is concerned with the interactions of the continuous dynamics and the discrete events. Our motivations for studying such processes originate from emerging and - isting applications in wireless communications, signal processing, queueing networks, production planning, biological systems, ecosystems, nancial engineering, and modeling, analysis, and control and optimization of lar- scale systems, under the in uence of random environments. Displaying mixture distributions, switching di usions may be described by the associated operators or by systems of stochastic di erential eq- tions together with the probability transition laws of the switching actions. We either have Markov-modulated switching di usions or processes with continuous state-dependent switching. The latter turns out to be much more challenging to deal with. Viewing the hybrid di usions as a number of di usions joined together by the switching process, they may be se- ingly not much di erent from their di usion counterpart. Nevertheless, the underlying problems become more di cult to handle, especially when the switching processes depend on continuous states. The di culty is due to the interaction of the discrete and continuous processes and the tangled and hybrid information pattern.

This volume contains papers based on invited talks given at the 2005 IMA Summer Workshop on Wireless Communications, held at the Institute for Mathematics and Its Applications, University of Minnesota, June 22-July 1, 2005. The workshop provided a great opportunity to facilitate the communications between academia and the industry, and to bridge the mathematical sciences, engineering, information theory, and communication communities. The emphasis were on design and analysis of computationally efficient algorithms to better understand the behavior and to control the wireless telecommunication networks. As an achieve, this volume presents some of the highlights of the workshop, and collects papers covering a broad spectrum of important and pressing issues in wireless communications. All papers have been reviewed. One of the book's distinct features is highly multi-disciplinary. This book is useful for researchers and advanced graduate students working in communication networks, information theory, signal processing, and applied probability and stochastic processes, among others.

This edited volume contains sixteen research articles and presents recent and pressing issues in stochastic processes, control theory, differential games, optimization, and their applications in finance, manufacturing, queueing networks, and climate control. One of the salient features is that the book is highly multi-disciplinary. It assembles experts from the fields of operations research, control theory and optimization, stochastic analysis, and financial engineering to review and substantially update the recent progress in these fields. Another distinct characteristic of the book is that all papers are motivated by applications in which optimization, control, and stochastics are inseparable. The book will be a timely addition to the literature and will be of interest to people working in the aforementioned fields.

Most importantly, this volume is dedicated to Professor Suresh Sethi on the occasion of his 60th birthday. In view of his fundamental contributions, his distinguished career, his substantial achievements, his influence on the areas of control theory and applications, operations research, and management science, and his dedication to the scientific community, a number of leading experts in the fields of optimization, control, and operation management, have contributed to this volume in honor of him.

In view of Professor Wendell Fleming's many fundamental contributions, his profound influence on the mathematical and systems theory communi ties, his service to the profession, and his dedication to mathematics, we have invited a number of leading experts in the fields of control, optimiza tion, and stochastic systems to contribute to this volume in his honor on the occasion of his 70th birthday. These papers focus on various aspects of stochastic analysis, control theory and optimization, and applications. They include authoritative expositions and surveys as well as research papers on recent and important issues. The papers are grouped according to the following four major themes: (1) large deviations, risk sensitive and Hoc control, (2) partial differential equations and viscosity solutions, (3) stochastic control, filtering and parameter esti mation, and (4) mathematical finance and other applications. We express our deep gratitude to all of the authors for their invaluable contributions, and to the referees for their careful and timely reviews. We thank Harold Kushner for having graciously agreed to undertake the task of writing the foreword. Particular thanks go to H. Thomas Banks for his help, advice and suggestions during the entire preparation process, as well as for the generous support of the Center for Research in Scientific Computation. The assistance from the Birkhauser professional staff is also greatly appreciated."

In view of Professor Wendell Fleming's many fundamental contributions, his profound influence on the mathematical and systems theory communi ties, his service to the profession, and his dedication to mathematics, we have invited a number of leading experts in the fields of control, optimiza tion, and stochastic systems to contribute to this volume in his honor on the occasion of his 70th birthday. These papers focus on various aspects of stochastic analysis, control theory and optimization, and applications. They include authoritative expositions and surveys as well as research papers on recent and important issues. The papers are grouped according to the following four major themes: (1) large deviations, risk sensitive and Hoc control, (2) partial differential equations and viscosity solutions, (3) stochastic control, filtering and parameter esti mation, and (4) mathematical finance and other applications. We express our deep gratitude to all of the authors for their invaluable contributions, and to the referees for their careful and timely reviews. We thank Harold Kushner for having graciously agreed to undertake the task of writing the foreword. Particular thanks go to H. Thomas Banks for his help, advice and suggestions during the entire preparation process, as well as for the generous support of the Center for Research in Scientific Computation. The assistance from the Birkhauser professional staff is also greatly appreciated."

This book presents recently developed methodologies that utilize quantized information in system identification and explores their potential in extending control capabilities for systems with limited sensor information or networked systems. The results of these methodologies can be applied to signal processing and control design of communication and computer networks, sensor networks, mobile agents, coordinated data fusion, remote sensing, telemedicine, and other fields in which noise-corrupted quantized data need to be processed.

System Identification with Quantized Observations is an excellent resource for graduate students, systems theorists, control engineers, applied mathematicians, as well as practitioners who use identification algorithms in their work.