Reinforcement learning (RL) is one of the foundational pillars of artificial intelligence and machine learning. An important consideration in any optimization or control problem is the notion of risk, but its incorporation into RL has been a fairly recent development. This monograph surveys research on risk-sensitive RL that uses policy gradient search. The authors survey some of the recent work in this area specifically where policy gradient search is the solution approach. In the first risk-sensitive RL setting, they cover popular risk measures based on variance, conditional value at-risk and chance constraints, and present a template for policy gradient-based risk-sensitive RL algorithms using a Lagrangian formulation. For the setting where risk is incorporated directly into the objective function, they consider an exponential utility formulation, cumulative prospect theory, and coherent risk measures. Written for novices and experts alike the authors have made the text completely self-contained but also organized in a manner that allows expert readers to skip background chapters. This is a complete guide for students and researchers working on this aspect of machine learning.

The "Handbook of Simulation Optimization" presents an overview of the state of the art of simulation optimization, providing a survey of the most well-established approaches for optimizing stochastic simulation models and a sampling of recent research advances in theory and methodology. Leading contributors cover such topics as discrete optimization via simulation, ranking and selection, efficient simulation budget allocation, random search methods, response surface methodology, stochastic gradient estimation, stochastic approximation, sample average approximation, stochastic constraints, variance reduction techniques, model-based stochastic search methods and Markov decision processes.

This single volume should serve as a reference for those already in the field and as a means for those new to the field for understanding and applying the main approaches. The intended audience includes researchers, practitioners and graduate students in the business/engineering fields of operations research, management science, operations management and stochastic control, as well as in economics/finance and computer science.

Conditional Monte Carlo: Gradient Estimation and Optimization Applications deals with various gradient estimation techniques of perturbation analysis based on the use of conditional expectation. The primary setting is discrete-event stochastic simulation. This book presents applications to queueing and inventory, and to other diverse areas such as financial derivatives, pricing and statistical quality control. To researchers already in the area, this book offers a unified perspective and adequately summarizes the state of the art. To researchers new to the area, this book offers a more systematic and accessible means of understanding the techniques without having to scour through the immense literature and learn a new set of notation with each paper. To practitioners, this book provides a number of diverse application areas that makes the intuition accessible without having to fully commit to understanding all the theoretical niceties. In sum, the objectives of this monograph are two-fold: to bring together many of the interesting developments in perturbation analysis based on conditioning under a more unified framework, and to illustrate the diversity of applications to which these techniques can be applied. Conditional Monte Carlo: Gradient Estimation and Optimization Applications is suitable as a secondary text for graduate level courses on stochastic simulations, and as a reference for researchers and practitioners in industry.

This self-contained volume brings together a collection of chapters by some of the most distinguished researchers and practitioners in the fields of mathematical finance and financial engineering. Presenting state-of-the-art developments in theory and practice, the Festschrift is dedicated to Dilip B. Madan on the occasion of his 60th birthday. Specific topics covered include: * Theory and application of the Variance-Gamma process * L?vy process driven fixed-income and credit-risk models, including CDO pricing * Numerical PDE and Monte Carlo methods * Asset pricing and derivatives valuation and hedging * It? formulas for fractional Brownian motion * Martingale characterization of asset price bubbles * Utility valuation for credit derivatives and portfolio management Advances in Mathematical Finance is a valuable resource for graduate students, researchers, and practitioners in mathematical finance and financial engineering. Contributors: H. Albrecher, D. C. Brody, P. Carr, E. Eberlein, R. J. Elliott, M. C. Fu, H. Geman, M. Heidari, A. Hirsa, L. P. Hughston, R. A. Jarrow, X. Jin, W. Kluge, S. A. Ladoucette, A. Macrina, D. B. Madan, F. Milne, M. Musiela, P. Protter, W. Schoutens, E. Seneta, K. Shimbo, R. Sircar, J. van der Hoek, M.Yor, T. Zariphopoulou

The book is an edited volume from leading research scholars in the field of Operations Research, focusing on future perspectives in OR. Each of the contributors offers their perspective looking forward to the further development of the field. The theme will provide pivotal interest in the book because of prominence of the contributors and Saul Gass' position as one of the founders of OR and his involvement in writing about the history of OR. The history of operations research is of considerable interest and this book takes a pivotal perspective of OR's history by examining current trends and the future of the field.

Markov decision process (MDP) models are widely used for modeling sequential decision-making problems that arise in engineering, economics, computer science, and the social sciences. Many real-world problems modeled by MDPs have huge state and/or action spaces, giving an opening to the curse of dimensionality and so making practical solution of the resulting models intractable. In other cases, the system of interest is too complex to allow explicit specification of some of the MDP model parameters, but simulation samples are readily available (e.g., for random transitions and costs). For these settings, various sampling and population-based algorithms have been developed to overcome the difficulties of computing an optimal solution in terms of a policy and/or value function. Specific approaches include adaptive sampling, evolutionary policy iteration, evolutionary random policy search, and model reference adaptive search.
This substantially enlarged new edition reflects the latest developments in novel algorithms and their underpinning theories, and presents an updated account of the topics that have emerged since the publication of the first edition. Includes:
innovative material on MDPs, both in constrained settings and with uncertain transition properties;
game-theoretic method for solving MDPs;
theories for developing roll-out based algorithms; and
details of approximation stochastic annealing, a population-based on-line simulation-based algorithm.
The self-contained approach of this book will appeal not only to researchers in MDPs, stochastic modeling, and control, and simulation but will be a valuable source of tuition and reference for students of control and operations research.

The Handbook of Simulation Optimization presents an overview of the state of the art of simulation optimization, providing a survey of the most well-established approaches for optimizing stochastic simulation models and a sampling of recent research advances in theory and methodology. Leading contributors cover such topics as discrete optimization via simulation, ranking and selection, efficient simulation budget allocation, random search methods, response surface methodology, stochastic gradient estimation, stochastic approximation, sample average approximation, stochastic constraints, variance reduction techniques, model-based stochastic search methods and Markov decision processes. This single volume should serve as a reference for those already in the field and as a means for those new to the field for understanding and applying the main approaches. The intended audience includes researchers, practitioners and graduate students in the business/engineering fields of operations research, management science, operations management and stochastic control, as well as in economics/finance and computer science.

Markov decision process (MDP) models are widely used for modeling sequential decision-making problems that arise in engineering, economics, computer science, and the social sciences. Many real-world problems modeled by MDPs have huge state and/or action spaces, giving an opening to the curse of dimensionality and so making practical solution of the resulting models intractable. In other cases, the system of interest is too complex to allow explicit specification of some of the MDP model parameters, but simulation samples are readily available (e.g., for random transitions and costs). For these settings, various sampling and population-based algorithms have been developed to overcome the difficulties of computing an optimal solution in terms of a policy and/or value function. Specific approaches include adaptive sampling, evolutionary policy iteration, evolutionary random policy search, and model reference adaptive search. This substantially enlarged new edition reflects the latest developments in novel algorithms and their underpinning theories, and presents an updated account of the topics that have emerged since the publication of the first edition. Includes: innovative material on MDPs, both in constrained settings and with uncertain transition properties; game-theoretic method for solving MDPs; theories for developing roll-out based algorithms; and details of approximation stochastic annealing, a population-based on-line simulation-based algorithm. The self-contained approach of this book will appeal not only to researchers in MDPs, stochastic modeling, and control, and simulation but will be a valuable source of tuition and reference for students of control and operations research.

Conditional Monte Carlo: Gradient Estimation and Optimization Applications deals with various gradient estimation techniques of perturbation analysis based on the use of conditional expectation. The primary setting is discrete-event stochastic simulation. This book presents applications to queueing and inventory, and to other diverse areas such as financial derivatives, pricing and statistical quality control. To researchers already in the area, this book offers a unified perspective and adequately summarizes the state of the art. To researchers new to the area, this book offers a more systematic and accessible means of understanding the techniques without having to scour through the immense literature and learn a new set of notation with each paper. To practitioners, this book provides a number of diverse application areas that makes the intuition accessible without having to fully commit to understanding all the theoretical niceties. In sum, the objectives of this monograph are two-fold: to bring together many of the interesting developments in perturbation analysis based on conditioning under a more unified framework, and to illustrate the diversity of applications to which these techniques can be applied. Conditional Monte Carlo: Gradient Estimation and Optimization Applications is suitable as a secondary text for graduate level courses on stochastic simulations, and as a reference for researchers and practitioners in industry.

A Symposium was held on February 25, 2006 in honor of the 80th birthday of Saul I. Gass and his major contributions to the field of operations research over 50 years. This volume includes articles from each of the Symposium speakers plus 16 other articles from friends, colleagues, and former students. Each contributor offers a forward-looking perspective on the future development of the field.

Markov decision process (MDP) models are widely used for modeling sequential decision-making problems that arise in engineering, economics, computer science, and the social sciences. This book brings the state-of-the-art research together for the first time. It provides practical modeling methods for many real-world problems with high dimensionality or complexity which have not hitherto been treatable with Markov decision processes.

The goal of the "Encyclopedia of Operations Research and Management Science" is to provide decision makers and problem solvers in business, industry, government, and academia a comprehensive overview of the wide range of ideas, methodologies, and synergistic forces that combine to form the preeminent decision-aiding fields of operations research and management science (OR/MS). The impact of OR/MS on the quality-of life and economic well-being of everyone is a story. The "Encyclopedia of Operations Research and Management Science "is the prologue to that story.

The editors, working with the "Encyclopedia's" Editorial Advisory Board, surveyed and divided (OR/MS into specific topics that collectively encompass the foundations, applications, and emerging elements of this ever-changing field. We also wanted to establish the close associations that OR/MS has maintained with other scientific endeavors, with special emphasis on its symbiotic relationships to computer science, information processing, and mathematics. Based on our broad view of OR/MS, we enlisted a distinguished international group of academics and practitioners to contribute articles on subjects for which they are renowned. We commissioned over 300 major expository articles and complemented them by numerous descriptions, discussions, definitions, and abbreviations. The connections between topics are highlighted by an entry's final "See" statement, as appropriate. Each article provides a background or history of the topic, describes relevant applications, overviews present and future trends, and lists seminal and current references. To allow for variety in exposition, the authors were instructed to present their material from their research and applied perspectives. In particular, the authors were allowed to use whatever mathematical notation they felt was "standard" for their topics. Of significant importance is that each contributed article has been authored by an leading authority on that particular topic.

The "Encyclopedia's "intended audience is technically diverse and wide; it includes anyone concerned with the science, techniques, and ideas of how one makes decisions. As this audience encompasses many professions, educational backgrounds and skills, we were attentive to the form, format, and scope of the articles. Thus, the articles are designed to serve as initial sources of information for all such readers, with special emphasis on the needs of students.