In a stochastic network, such as those in computer/telecommunications and manufacturing, discrete units move among a network of stations where they are processed or served. Randomness may occur in the servicing and routing of units, and there may be queueing for services. This book describes several basic stochastic network processes, beginning with Jackson networks and ending with spatial queueing systems in which units, such as cellular phones, move in a space or region where they are served. The focus is on network processes that have tractable (closed-form) expressions for the equilibrium probability distribution of the numbers of units at the stations. These distributions yield network performance parameters such as expectations of throughputs, delays, costs, and travel times. The book is intended for graduate students and researchers in engineering, science and mathematics interested in the basics of stochastic networks that have been developed over the last twenty years. Assuming a graduate course in stochastic processes without measure theory, the emphasis is on multi-dimensional Markov processes. There is also some self-contained material on point processes involving real analysis. The book also contains rather complete introductions to reversible Markov processes, Palm probabilities for stationary systems, Little laws for queueing systems and space-time Poisson processes. This material is used in describing reversible networks, waiting times at stations, travel times and space-time flows in networks. Richard Serfozo received the Ph.D. degree in Industrial Engineering and Management Sciences at Northwestern University in 1969 and is currently Professor of Industrial and Systems Engineering at Georgia Institute of Technology. Prior to that he held positions in the Boeing Company, Syracuse University, and Bell Laboratories. He has held

Safety critical and high-integrity systems, such as industrial plants and economic systems can be subject to abrupt changes - for instance due to component or interconnection failure, and sudden environment changes etc.

Combining probability and operator theory, Discrete-Time Markov Jump Linear Systems provides a unified and rigorous treatment of recent results for the control theory of discrete jump linear systems, which are used in these areas of application.

The book is designed for experts in linear systems with Markov jump parameters, but is also of interest for specialists in stochastic control since it presents stochastic control problems for which an explicit solution is possible - making the book suitable for course use.

From the reviews:

"This text is very well written...it may prove valuable to those who work in the area, are at home with its mathematics, and are interested in stability of linear systems, optimal control, and filtering." Journal of the American Statistical Association, December 2005

This book introduces data-driven remaining useful life prognosis techniques, and shows how to utilize the condition monitoring data to predict the remaining useful life of stochastic degrading systems and to schedule maintenance and logistics plans. It is also the first book that describes the basic data-driven remaining useful life prognosis theory systematically and in detail. The emphasis of the book is on the stochastic models, methods and applications employed in remaining useful life prognosis. It includes a wealth of degradation monitoring experiment data, practical prognosis methods for remaining useful life in various cases, and a series of applications incorporated into prognostic information in decision-making, such as maintenance-related decisions and ordering spare parts. It also highlights the latest advances in data-driven remaining useful life prognosis techniques, especially in the contexts of adaptive prognosis for linear stochastic degrading systems, nonlinear degradation modeling based prognosis, residual storage life prognosis, and prognostic information-based decision-making.

This book presents the first part of a planned two-volume series devoted to a systematic exposition of some recent developments in the theory of discrete-time Markov control processes (MCPs). Interest is mainly confined to MCPs with Borel state and control (or action) spaces, and possibly unbounded costs and noncompact control constraint sets. MCPs are a class of stochastic control problems, also known as Markov decision processes, controlled Markov processes, or stochastic dynamic pro grams; sometimes, particularly when the state space is a countable set, they are also called Markov decision (or controlled Markov) chains. Regardless of the name used, MCPs appear in many fields, for example, engineering, economics, operations research, statistics, renewable and nonrenewable re source management, (control of) epidemics, etc. However, most of the lit erature (say, at least 90%) is concentrated on MCPs for which (a) the state space is a countable set, and/or (b) the costs-per-stage are bounded, and/or (c) the control constraint sets are compact. But curiously enough, the most widely used control model in engineering and economics--namely the LQ (Linear system/Quadratic cost) model-satisfies none of these conditions. Moreover, when dealing with "partially observable" systems) a standard approach is to transform them into equivalent "completely observable" sys tems in a larger state space (in fact, a space of probability measures), which is uncountable even if the original state process is finite-valued."

This collection of selected reprints presents as broad a selection as possible, emphasizing formal and numerical aspects of Stochastic Quantization. It reviews and explains the most important concepts placing selected reprints and crucial papers into perspective and compact form.

This collection of selected reprints presents as broad a selection as possible, emphasizing formal and numerical aspects of Stochastic Quantization. It reviews and explains the most important concepts placing selected reprints and crucial papers into perspective and compact form.

A Levy process is a continuous-time analogue of a random walk, and as such, is at the cradle of modern theories of stochastic processes. Martingales, Markov processes, and diffusions are extensions and generalizations of these processes. In the past, representatives of the Levy class were considered most useful for applications to either Brownian motion or the Poisson process. Nowadays the need for modeling jumps, bursts, extremes and other irregular behavior of phenomena in nature and society has led to a renaissance of the theory of general Levy processes. Researchers and practitioners in fields as diverse as physics, meteorology, statistics, insurance, and finance have rediscovered the simplicity of Levy processes and their enormous flexibility in modeling tails, dependence and path behavior.

This volume, with an excellent introductory preface, describes the state-of-the-art of this rapidly evolving subject with special emphasis on the non-Brownian world. Leading experts present surveys of recent developments, or focus on some most promising applications. Despite its special character, every topic is aimed at the non- specialist, keen on learning about the new exciting face of a rather aged class of processes. An extensive bibliography at the end of each article makes this an invaluable comprehensive reference text. For the researcher and graduate student, every article contains open problems and points out directions for futurearch.

The accessible nature of the work makes this an ideal introductory text for graduate seminars in applied probability, stochastic processes, physics, finance, and telecommunications, and a unique guide to the world of Levy processes.

"

Nonlinear modelling has become increasingly important and widely used in economics. This valuable book brings together recent advances in the area including contributions covering cross-sectional studies of income distribution and discrete choice models, time series models of exchange rate dynamics and jump processes, and artificial neural network and genetic algorithm models of financial markets. Attention is given to the development of theoretical models as well as estimation and testing methods with a wide range of applications in micro and macroeconomics, labour and finance. The book provides valuable introductory material that is accessible to students and scholars interested in this exciting research area, as well as presenting the results of new and original research. Nonlinear Economic Models provides a sequel to Chaos and Nonlinear Models in Economics by the same editors.

The last decade has seen a remarkable development of the "Marginal and Moment Problems" as a research area in Probability and Statistics. Its attractiveness stemmed from its lasting ability to provide a researcher with difficult theoretical problems that have direct consequences for appli cations outside of mathematics. The relevant research aims centered mainly along the following lines that very frequently met each other to provide sur prizing and useful results: -To construct a probability distribution (to prove its existence, at least) with a given support and with some additional inner stochastic property defined typically either by moments or by marginal distributions. -To study the geometrical and topological structure of the set of prob ability distributions generated by such a property mostly with the aim to propose a procedure that would result in a stochastic model with some optimal properties within the set of probability distributions. These research aims characterize also, though only very generally, the scientific program of the 1996 conference "Distributions with given marginals and moment problems" held at the beginning of September in Prague, Czech Republic, to perpetuate the tradition and achievements of the closely related 1990 Roma symposium "On Frechet Classes" 1 and 1993 Seattle" AMS Summer Conference on Marginal Problem.""

In many areas of finance and stochastics, significant advances have been made since this field of research was opened by Black, Scholes and Merton in 1973. Advances in Finance and Stochastics contains a collection of original articles by a number of highly distinguished authors on research topics that are currently in the focus of interest of both academics and practitioners. The topics span risk management, portfolio theory and multi-asset derivatives, market imperfections, interest-rate modelling and exotic options.

The past two decades have seen a great deal of research into the stochastic modelling of production, manufacturing, and inventory systems for the purpose of improving their performance. This book provides a graduate-level introduction to these techniques covering exact, approximate, and numerical techniques. The author has aimed to strike a balance between theoretical issues and the practical aspects of modelling manufacturing systems. It is based on graduate courses given to operations research and industrial engineering students and includes numerous examples and exercises.

This book presents the latest findings on stochastic dynamic programming models and on solving optimal control problems in networks. It includes the authors' new findings on determining the optimal solution of discrete optimal control problems in networks and on solving game variants of Markov decision problems in the context of computational networks. First, the book studies the finite state space of Markov processes and reviews the existing methods and algorithms for determining the main characteristics in Markov chains, before proposing new approaches based on dynamic programming and combinatorial methods. Chapter two is dedicated to infinite horizon stochastic discrete optimal control models and Markov decision problems with average and expected total discounted optimization criteria, while Chapter three develops a special game-theoretical approach to Markov decision processes and stochastic discrete optimal control problems. In closing, the book's final chapter is devoted to finite horizon stochastic control problems and Markov decision processes. The algorithms developed represent a valuable contribution to the important field of computational network theory.

Markov decision processes (MDPs), also called stochastic dynamic programming, were first studied in the 1960s. MDPs can be used to model and solve dynamic decision-making problems that are multiperiod and in stochastic circumstances. There are three basic branches in MDPs: discrete-time MDPs, continuous-time MDPs and semi-Markov decision processes. Starting from these three branches, many generalized MDPs models have been applied to various practical problems. These models include partially observable MDPs, adaptive MDPs, MDPs in stochastic environments, and MDPs with multiple objectives, constraints or imprecise parameters. MDPs have been applied in many areas, such as communications, signal processing, artificial intelligence, stochastic scheduling and manufacturing systems, discrete event systems, management and economies. This book examines MDPs and their applications in the optimal control of discrete event systems (DESs), optimal replacement, and optimal allocations in sequential online auctions.The book presents three main topics: a new methodology for MDPs with discounted total reward criterion; transformation of continuous-time MDPs and semi-Markov decision processes into a discrete-time MDPs model, thereby simplifying the application of MDPs; application of MDPs in stochastic environments, which greatly extends the area where MDPs can be applied. Each topic is used to study optimal control problems or other types of problems.

This IMA Volume in Mathematics and its Applications CLASSICAL AND MODERN BRANCHING PROCESSES is based on the proceedings with the same title and was an integral part of the 1993-94 IMA program on "Emerging Applications of Probability." We would like to thank Krishna B. Athreya and Peter J agers for their hard work in organizing this meeting and in editing the proceedings. We also take this opportunity to thank the National Science Foundation, the Army Research Office, and the National Security Agency, whose financial support made this workshop possible. A vner Friedman Robert Gulliver v PREFACE The IMA workshop on Classical and Modern Branching Processes was held during June 13-171994 as part of the IMA year on Emerging Appli cations of Probability. The organizers of the year long program identified branching processes as one of the active areas in which a workshop should be held. Krish na B. Athreya and Peter Jagers were asked to organize this. The topics covered by the workshop could broadly be divided into the following areas: 1. Tree structures and branching processes; 2. Branching random walks; 3. Measure valued branching processes; 4. Branching with dependence; 5. Large deviations in branching processes; 6. Classical branching processes."

The book is devoted to the new trends in random evolutions and their various applications to stochastic evolutionary sytems (SES). Such new developments as the analogue of Dynkin's formulae, boundary value problems, stochastic stability and optimal control of random evolutions, stochastic evolutionary equations driven by martingale measures are considered. The book also contains such new trends in applied probability as stochastic models of financial and insurance mathematics in an incomplete market. In the famous classical financial mathematics Black-Scholes model of a (B, S) market for securities prices, which is used for the description of the evolution of bonds and stocks prices and also for their derivatives, such as options, futures, forward contracts, etc., it is supposed that the dynamic of bonds and stocks prices are set by a linear differential and linear stochastic differential equations, respectively, with interest rate, appreciation rate and volatility such that they are predictable processes. Also, in the Arrow-Debreu economy, the securities prices which support a Radner dynamic equilibrium are a combination of an Ito process and a random point process, with the all coefficients and jumps being predictable processes."

In 1978 Edwin T. Jaynes and Myron Tribus initiated a series of workshops to exchange ideas and recent developments in technical aspects and applications of Bayesian probability theory. The first workshop was held at the University of Wyoming in 1981 organized by C.R. Smith and W.T. Grandy. Due to its success, the workshop was held annually during the last 18 years. Over the years, the emphasis of the workshop shifted gradually from fundamental concepts of Bayesian probability theory to increasingly realistic and challenging applications. The 18th international workshop on Maximum Entropy and Bayesian Methods was held in Garching / Munich (Germany) (27-31. July 1998). Opening lectures by G. Larry Bretthorst and by Myron Tribus were dedicated to one of th the pioneers of Bayesian probability theory who died on the 30 of April 1998: Edwin Thompson Jaynes. Jaynes revealed and advocated the correct meaning of 'probability' as the state of knowledge rather than a physical property. This inter pretation allowed him to unravel longstanding mysteries and paradoxes. Bayesian probability theory, "the logic of science" - as E.T. Jaynes called it - provides the framework to make the best possible scientific inference given all available exper imental and theoretical information. We gratefully acknowledge the efforts of Tribus and Bretthorst in commemorating the outstanding contributions of E.T. Jaynes to the development of probability theory."

Queues and stochastic networks are analyzed in this book with purely probabilistic methods. The purpose of these lectures is to show that general results from Markov processes, martingales or ergodic theory can be used directly to study the corresponding stochastic processes. Recent developments have shown that, instead of having ad-hoc methods, a better understanding of fundamental results on stochastic processes is crucial to study the complex behavior of stochastic networks. In this book, various aspects of these stochastic models are investigated in depth in an elementary way: Existence of equilibrium, characterization of stationary regimes, transient behaviors (rare events, hitting times) and critical regimes, etc. A simple presentation of stationary point processes and Palm measures is given. Scaling methods and functional limit theorems are a major theme of this book. In particular, a complete chapter is devoted to fluid limits of Markov processes.

Schr dinger Equations and Diffusion Theory addresses the question "What is the Schr dinger equation?" in terms of diffusion processes, and shows that the Schr dinger equation and diffusion equations in duality are equivalent. In turn, Schr dinger's conjecture of 1931 is solved. The theory of diffusion processes for the Schr dinger equation tell us that we must go further into the theory of systems of (infinitely) many interacting quantum (diffusion) particles.
The method of relative entropy and the theory of transformations enable us to construct severely singular diffusion processes which appear to be equivalent to Schr dinger equations.
The theory of large deviations and the propagation of chaos of interacting diffusion particles reveal the statistical mechanical nature of the Schr dinger equation, namely, quantum mechanics.
The text is practically self-contained and requires only an elementary knowledge of probability theory at the graduate level.

This book considers recent developments in numerical error estimation and adaptive discretization for finite element and finite volume methods with particular attention given to discretization methods used frequently in computational fluid dynamics. The volume consists of six detailed articles by leading specialists covering a range of topics including a posteriori error estimation of functionals, one- and two-sided error bounds, error indicators for ad aptivity, and nd geometrical aspects of adaptive mesh refinement. This book should be of interest to readers actively working in the field as well as readers seeking a comprehensive introduction to the topic.

Bifurcation theory and catastrophe theory are two well-known areas within the field of dynamical systems. Both are studies of smooth systems, focusing on properties that seem to be manifestly non-smooth. Bifurcation theory is concerned with the sudden changes that occur in a system when one or more parameters are varied. Examples of such are familiar to students of differential equations, from phase portraits. Understanding the bifurcations of the differential equations that describe real physical systems provides important information about the behavior of the systems. Catastrophe theory became quite famous during the 1970's, mostly because of the sensation caused by the usually less than rigorous applications of its principal ideas to "hot topics," such as the characterization of personalities and the difference between a "genius" and a "maniac." Catastrophe theory is accurately described as singularity theory and its (genuine) applications. The authors of this book, previously published as Volume 5 of the Encyclopaedia, have given a masterly exposition of these two theories, with penetrating insight.

Control theory has applications to a number of areas in engineering and communication theory. This introductory text on the subject is fairly self-contained and aimed primarily at advanced mathematics and engineering students in various disciplines.

The topics covered include realization problems, linear-quadratic optimal control, stability theory, stochastic modeling and recursive estimation algorithms in communications and control, and distributed system modeling. These topics have a wide range of applicability, and provide background for further study in the control and communications areas.

In the early chapters the basics of linear control systems as well as the fundamentals of stochastic control are presented in a unique way so that the methods generalize to a useful class of distributed parameter and nonlinear system models. The control of distributed parameter systems (systems governed by PDEs) is based on the framework of linear quadratic Gaussian optimization problems.

The approach here utilizes methods based on Wiener-Hopf integral equations. Additionally, the important notion of state space modeling of distributed systems is examined. Basic results due to Gohberg and Krein on convolution are given and many results are illustrated with some examples that carry throughout the text. The standard linear regulator problem is studied in both the continuous and discrete time cases, followed by a discussion of the (dual) filtering problems. Later chapters treat the stationary regulator and filtering problems with a Wiener-Hopf approach. This leads to spectral factorization problems and useful iterative algorithms that follow naturally from the methods employed. Theinterplay between time and frequency domain approaches is emphasized.

Whether costs are to be reduced, profits to be maximized, or scarce resources to be used wisely, optimization methods are available to guide decision making. In online optimization the main issue is incomplete data, and the scientific challenge: How well can an online algorithm perform? Can one guarantee solution quality, even without knowing all data in advance? In real-time optimization there is an additional requirement, decisions have to be computed very fast in relation to the time frame of the instance we consider. Online and real-time optimization problems occur in all branches of optimization. These areas have developed their own techniques but they are addressing the same issues: quality, stability, and robustness of the solutions. To fertilize this emerging topic of optimization theory and to foster cooperation between the different branches of optimization, the Deutsche Forschungsgemeinschaft (DFG) has supported a Priority Programme "Online Optimization of Large Systems".

This book presents a selection of papers presented to the Second Inter national Symposium on Semi-Markov Models: Theory and Applications held in Compiegne (France) in December 1998. This international meeting had the same aim as the first one held in Brussels in 1984: to make, fourteen years later, the state of the art in the field of semi-Markov processes and their applications, bring together researchers in this field and also to stimulate fruitful discussions. The set of the subjects of the papers presented in Compiegne has a lot of similarities with the preceding Symposium; this shows that the main fields of semi-Markov processes are now well established particularly for basic applications in Reliability and Maintenance, Biomedicine, Queue ing, Control processes and production. A growing field is the one of insurance and finance but this is not really a surprising fact as the problem of pricing derivative products represents now a crucial problem in economics and finance. For example, stochastic models can be applied to financial and insur ance models as we have to evaluate the uncertainty of the future market behavior in order, firstly, to propose different measures for important risks such as the interest risk, the risk of default or the risk of catas trophe and secondly, to describe how to act in order to optimize the situation in time. Recently, the concept of VaR (Value at Risk) was "discovered" in portfolio theory enlarging so the fundamental model of Markowitz."

High dimensional probability, in the sense that encompasses the topics rep resented in this volume, began about thirty years ago with research in two related areas: limit theorems for sums of independent Banach space valued random vectors and general Gaussian processes. An important feature in these past research studies has been the fact that they highlighted the es sential probabilistic nature of the problems considered. In part, this was because, by working on a general Banach space, one had to discard the extra, and often extraneous, structure imposed by random variables taking values in a Euclidean space, or by processes being indexed by sets in R or Rd. Doing this led to striking advances, particularly in Gaussian process theory. It also led to the creation or introduction of powerful new tools, such as randomization, decoupling, moment and exponential inequalities, chaining, isoperimetry and concentration of measure, which apply to areas well beyond those for which they were created. The general theory of em pirical processes, with its vast applications in statistics, the study of local times of Markov processes, certain problems in harmonic analysis, and the general theory of stochastic processes are just several of the broad areas in which Gaussian process techniques and techniques from probability in Banach spaces have made a substantial impact. Parallel to this work on probability in Banach spaces, classical proba bility and empirical process theory were enriched by the development of powerful results in strong approximations."