This book studies the large deviations for empirical measures and vector-valued additive functionals of Markov chains with general state space. Under suitable recurrence conditions, the ergodic theorem for additive functionals of a Markov chain asserts the almost sure convergence of the averages of a real or vector-valued function of the chain to the mean of the function with respect to the invariant distribution. In the case of empirical measures, the ergodic theorem states the almost sure convergence in a suitable sense to the invariant distribution. The large deviation theorems provide precise asymptotic estimates at logarithmic level of the probabilities of deviating from the preponderant behavior asserted by the ergodic theorems.

The prolonged boom in the US and European stock markets has led to increased interest in the mathematics of security markets, most notably in the theory of stochastic integration. This text gives a rigorous development of the theory of stochastic integration as it applies to the valuation of derivative securities. It includes all the tools necessary for readers to understand how the stochastic integral is constructed with respect to a general continuous martingale.

The author develops the stochastic calculus from first principles, but at a relaxed pace that includes proofs that are detailed, but streamlined to applications to finance. The treatment requires minimal prerequisites-a basic knowledge of measure theoretic probability and Hilbert space theory-and devotes an entire chapter to application in finances, including the Black Scholes market, pricing contingent claims, the general market model, pricing of random payoffs, and interest rate derivatives.

Continuous Stochastic Calculus with Application to Finance is your first opportunity to explore stochastic integration at a reasonable and practical mathematical level. It offers a treatment well balanced between aesthetic appeal, degree of generality, depth, and ease of reading.

This book presents the probabilistic methods around Hardy martingales for an audience interested in their applications to complex, harmonic, and functional analysis. Building on work of Bourgain, Garling, Jones, Maurey, Pisier, and Varopoulos, it discusses in detail those martingale spaces that reflect characteristic qualities of complex analytic functions. Its particular themes are holomorphic random variables on Wiener space, and Hardy martingales on the infinite torus product, and numerous deep applications to the geometry and classification of complex Banach spaces, e.g., the SL estimates for Doob's projection operator, the embedding of L1 into L1/H1, the isomorphic classification theorem for the polydisk algebras, or the real variables characterization of Banach spaces with the analytic Radon Nikodym property. Due to the inclusion of key background material on stochastic analysis and Banach space theory, it's suitable for a wide spectrum of researchers and graduate students working in classical and functional analysis.

Compound renewal processes (CRPs) are among the most ubiquitous models arising in applications of probability. At the same time, they are a natural generalization of random walks, the most well-studied classical objects in probability theory. This monograph, written for researchers and graduate students, presents the general asymptotic theory and generalizes many well-known results concerning random walks. The book contains the key limit theorems for CRPs, functional limit theorems, integro-local limit theorems, large and moderately large deviation principles for CRPs in the state space and in the space of trajectories, including large deviation principles in boundary crossing problems for CRPs, with an explicit form of the rate functionals, and an extension of the invariance principle for CRPs to the domain of moderately large and small deviations. Applications establish the key limit laws for Markov additive processes, including limit theorems in the domains of normal and large deviations.

Statistical inference carries great significance in model building from both the theoretical and the applications points of view. Its applications to engineering and economic systems, financial economics, and the biological and medical sciences have made statistical inference for stochastic processes a well-recognized and important branch of statistics and probability.
The class of semimartingales includes a large class of stochastic processes, including diffusion type processes, point processes, and diffusion type processes with jumps, widely used for stochastic modeling. Until now, however, researchers have had no single reference that collected the research conducted on the asymptotic theory for semimartingales.

Semimartingales and their Statistical Inference, fills this need by presenting a comprehensive discussion of the asymptotic theory of semimartingales at a level needed for researchers working in the area of statistical inference for stochastic processes. The author brings together into one volume the state-of-the-art in the inferential aspect for such processes. The topics discussed include:

· Asymptotic likelihood theory
· Quasi-likelihood
· Likelihood and efficiency
· Inference for counting processes
· Inference for semimartingale regression models

The author addresses a number of stochastic modeling applications from engineering, economic systems, financial economics, and medical sciences. He also includes some of the new and challenging statistical and probabilistic problems facing today's active researchers working in the area of inference for stochastic processes.

This book provides a unified approach for the study of constrained Markov decision processes with a finite state space and unbounded costs. Unlike the single controller case considered in many other books, the author considers a single controller with several objectives, such as minimizing delays and loss, probabilities, and maximization of throughputs. It is desirable to design a controller that minimizes one cost objective, subject to inequality constraints on other cost objectives. This framework describes dynamic decision problems arising frequently in many engineering fields. A thorough overview of these applications is presented in the introduction.
The book is then divided into three sections that build upon each other.
The first part explains the theory for the finite state space. The author characterizes the set of achievable expected occupation measures as well as performance vectors, and identifies simple classes of policies among which optimal policies exist. This allows the reduction of the original dynamic into a linear program. A Lagranian approach is then used to derive the dual linear program using dynamic programming techniques.
In the second part, these results are extended to the infinite state space and action spaces. The author provides two frameworks: the case where costs are bounded below and the contracting framework.
The third part builds upon the results of the first two parts and examines asymptotical results of the convergence of both the value and the policies in the time horizon and in the discount factor. Finally, several state truncation algorithms that enable the approximation of the solution of the original control problem via finite linear programs are given.

Optimal Control and Optimization of Stochastic Supply Chain Systems examines its subject the context of the presence of a variety of uncertainties. Numerous examples with intuitive illustrations and tables are provided, to demonstrate the structural characteristics of the optimal control policies in various stochastic supply chains and to show how to make use of these characteristics to construct easy-to-operate sub-optimal policies. In Part I, a general introduction to stochastic supply chain systems is provided. Analytical models for various stochastic supply chain systems are formulated and analysed in Part II. In Part III the structural knowledge of the optimal control policies obtained in Part II is utilized to construct easy-to-operate sub-optimal control policies for various stochastic supply chain systems accordingly. Finally, Part IV discusses the optimisation of threshold-type control policies and their robustness. A key feature of the book is its tying together of the complex analytical models produced by the requirements of operational practice, and the simple solutions needed for implementation. The analytical models and theoretical analysis propounded in this monograph will be of benefit to academic researchers and graduate students looking at logistics and supply chain management from standpoints in operations research or industrial, manufacturing, or control engineering. The practical tools and solutions and the qualitative insights into the ideas underlying functional supply chain systems will be of similar use to readers from more industrially-based backgrounds.

The seminar on Stochastic Analysis and Mathematical Physics started in 1984 at the Catholic University of Chile in Santiago and has been an on going research activity. Since 1995, the group has organized international workshops as a way of promoting a broader dialogue among experts in the areas of classical and quantum stochastic analysis, mathematical physics and physics. This volume, consisting primarily of contributions to the Third Inter national Workshop on Stochastic Analysis and Mathematical Physics (in Spanish ANESTOC), held in Santiago, Chile, in October 1998, focuses on an analysis of quantum dynamics and related problems in probability the ory. Various articles investigate quantum dynamical semigroups and new results on q-deformed oscillator algebras, while others examine the appli cation of classical stochastic processes in quantum modeling. As in previous workshops, the topic of quantum flows and semigroups occupied an important place. In her paper, R. Carbone uses a spectral type analysis to obtain exponential rates of convergence towards the equilibrium of a quantum dynamical semigroup in the GBP2 sense. The method is illus trated with a quantum extension of a classical birth and death process. Quantum extensions of classical Markov processes lead to subtle problems of domains. This is in particular illustrated by F. Fagnola, who presents a pathological example of a semigroup for which the largest * -subalgebra (of the von Neumann algebra of bounded linear operators of GBP2 (lR+, IC)), con tained in the domain of its infinitesimal generator, is not a-weakly dense.

Presenting statistical and stochastic methods for the analysis and design of technological systems in engineering and applied areas, this work documents developments in statistical modelling, identification, estimation and signal processing. The book covers such topics as subspace methods, stochastic realization, state space modelling, and identification and parameter estimation.

The book addresses the system performance with a focus on the network-enhanced complexities and developing the engineering-oriented design framework of controllers and filters with potential applications in system sciences, control engineering and signal processing areas. Therefore, it provides a unified treatment on the analysis and synthesis for discrete-time stochastic systems with guarantee of certain performances against network-enhanced complexities with applications in sensor networks and mobile robotics. Such a result will be of great importance in the development of novel control and filtering theories including industrial impact. Key Features Provides original methodologies and emerging concepts to deal with latest issues in the control and filtering with an emphasis on a variety of network-enhanced complexities Gives results of stochastic control and filtering distributed control and filtering, and security control of complex networked systems Captures the essence of performance analysis and synthesis for stochastic control and filtering Concepts and performance indexes proposed reflect the requirements of engineering practice Methodologies developed in this book include backward recursive Riccati difference equation approach and the discrete-time version of input-to-state stability in probability

Geographical Models with Mathematica provides a fairly comprehensive overview of the types of models necessary for the development of new geographical knowledge, including stochastic models, models for data analysis, for geostatistics, for networks, for dynamic systems, for cellular automata and for multi-agent systems, all discussed in their theoretical context. The author then provides over 65 programs, written in the Mathematica language, that formalize these models. Case studies are provided to help the reader apply these programs to their own studies.

This book deals with certain important problems in Classical and Quantum Information Theory Quantum Information Theory, A Selection of Matrix Inequalities Stochastic Filtering Theory Applied to Electromagnetic Fields and Strings Wigner-distributions in Quantum Mechanics Quantization of Classical Field Theories Statistical Signal Processing Quantum Field Theory, Quantum Statistics, Gravity, Stochastic Fields and Information Problems in Information Theory It will be very helpful for students of Undergraduate and Postgraduate Courses in Electronics, Communication and Signal Processing. Print edition not for sale in South Asia (India, Sri Lanka, Nepal, Bangladesh, Pakistan or Bhutan).

In pioneering work in the 1950s, S. Karlin and J. McGregor showed that probabilistic aspects of certain Markov processes can be studied by analyzing orthogonal eigenfunctions of associated operators. In the decades since, many authors have extended and deepened this surprising connection between orthogonal polynomials and stochastic processes. This book gives a comprehensive analysis of the spectral representation of the most important one-dimensional Markov processes, namely discrete-time birth-death chains, birth-death processes and diffusion processes. It brings together the main results from the extensive literature on the topic with detailed examples and applications. Also featuring an introduction to the basic theory of orthogonal polynomials and a selection of exercises at the end of each chapter, it is suitable for graduate students with a solid background in stochastic processes as well as researchers in orthogonal polynomials and special functions who want to learn about applications of their work to probability.

Applications of queueing network models have multiplied in the last generation, including scheduling of large manufacturing systems, control of patient flow in health systems, load balancing in cloud computing, and matching in ride sharing. These problems are too large and complex for exact solution, but their scale allows approximation. This book is the first comprehensive treatment of fluid scaling, diffusion scaling, and many-server scaling in a single text presented at a level suitable for graduate students. Fluid scaling is used to verify stability, in particular treating max weight policies, and to study optimal control of transient queueing networks. Diffusion scaling is used to control systems in balanced heavy traffic, by solving for optimal scheduling, admission control, and routing in Brownian networks. Many-server scaling is studied in the quality and efficiency driven Halfin-Whitt regime and applied to load balancing in the supermarket model and to bipartite matching in ride-sharing applications.

Applications of queueing network models have multiplied in the last generation, including scheduling of large manufacturing systems, control of patient flow in health systems, load balancing in cloud computing, and matching in ride sharing. These problems are too large and complex for exact solution, but their scale allows approximation. This book is the first comprehensive treatment of fluid scaling, diffusion scaling, and many-server scaling in a single text presented at a level suitable for graduate students. Fluid scaling is used to verify stability, in particular treating max weight policies, and to study optimal control of transient queueing networks. Diffusion scaling is used to control systems in balanced heavy traffic, by solving for optimal scheduling, admission control, and routing in Brownian networks. Many-server scaling is studied in the quality and efficiency driven Halfin-Whitt regime and applied to load balancing in the supermarket model and to bipartite matching in ride-sharing applications.

This book is intended as a text covering the central concepts and techniques of Competitive Markov Decision Processes. It is an attempt to present a rig orous treatment that combines two significant research topics: Stochastic Games and Markov Decision Processes, which have been studied exten sively, and at times quite independently, by mathematicians, operations researchers, engineers, and economists. Since Markov decision processes can be viewed as a special noncompeti tive case of stochastic games, we introduce the new terminology Competi tive Markov Decision Processes that emphasizes the importance of the link between these two topics and of the properties of the underlying Markov processes. The book is designed to be used either in a classroom or for self-study by a mathematically mature reader. In the Introduction (Chapter 1) we outline a number of advanced undergraduate and graduate courses for which this book could usefully serve as a text. A characteristic feature of competitive Markov decision processes - and one that inspired our long-standing interest - is that they can serve as an "orchestra" containing the "instruments" of much of modern applied (and at times even pure) mathematics. They constitute a topic where the instruments of linear algebra, applied probability, mathematical program ming, analysis, and even algebraic geometry can be "played" sometimes solo and sometimes in harmony to produce either beautifully simple or equally beautiful, but baroque, melodies, that is, theorems."

The main subject of this introductory book is simple random walk on the integer lattice, with special attention to the two-dimensional case. This fascinating mathematical object is the point of departure for an intuitive and richly illustrated tour of related topics at the active edge of research. It starts with three different proofs of the recurrence of the two-dimensional walk, via direct combinatorial arguments, electrical networks, and Lyapunov functions. After reviewing some relevant potential-theoretic tools, the reader is guided toward the relatively new topic of random interlacements - which can be viewed as a 'canonical soup' of nearest-neighbour loops through infinity - again with emphasis on two dimensions. On the way, readers will visit conditioned simple random walks - which are the 'noodles' in the soup - and also discover how Poisson processes of infinite objects are constructed and review the recently introduced method of soft local times. Each chapter ends with many exercises, making it suitable for courses and independent study.

The main subject of this introductory book is simple random walk on the integer lattice, with special attention to the two-dimensional case. This fascinating mathematical object is the point of departure for an intuitive and richly illustrated tour of related topics at the active edge of research. It starts with three different proofs of the recurrence of the two-dimensional walk, via direct combinatorial arguments, electrical networks, and Lyapunov functions. After reviewing some relevant potential-theoretic tools, the reader is guided toward the relatively new topic of random interlacements - which can be viewed as a 'canonical soup' of nearest-neighbour loops through infinity - again with emphasis on two dimensions. On the way, readers will visit conditioned simple random walks - which are the 'noodles' in the soup - and also discover how Poisson processes of infinite objects are constructed and review the recently introduced method of soft local times. Each chapter ends with many exercises, making it suitable for courses and independent study.

This book aims to position itself between the level of elementary probability texts and advanced works on stochastic processes. The pre-requisites to consult this book are a course on elementary probability theory and statistics, and a course on advanced calculus. In this book numberous examples have been given, based on theories discussed and a large number of problems along with their answers have also been provided. This revised edition further updates the materials and references and some new chapters have been introduced. The text has been designed particularly for advanced undergraduate, postgraduate and research level courses in applied maths, statistics, operations research, computer science, different branches of engineering, telecommunications, business and management, economics and life sciences.

This unique two-volume set presents the subjects of stochastic processes, information theory, and Lie groups in a unified setting, thereby building bridges between fields that are rarely studied by the same individuals. Unlike the many excellent formal treatments available for each of these subjects individually, the emphasis in both of these volumes is on the use of stochastic, geometric, and group-theoretic concepts in the modeling of physical phenomena.

Stochastic Models, Information Theory, and Lie Groups will be of interest to advanced undergraduate and graduate students, researchers, and practitioners working in applied mathematics, the physical sciences, and engineering. Extensive exercises and motivating examples make the work suitable as a textbook for use in courses that emphasize applied stochastic processes or differential geometry.

Probability theory has diverse applications in a plethora of fields, including physics, engineering, computer science, chemistry, biology and economics. This book will familiarize students with various applications of probability theory, stochastic modeling and random processes, using examples from all these disciplines and more. The reader learns via case studies and begins to recognize the sort of problems that are best tackled probabilistically. The emphasis is on conceptual understanding, the development of intuition and gaining insight, keeping technicalities to a minimum. Nevertheless, a glimpse into the depth of the topics is provided, preparing students for more specialized texts while assuming only an undergraduate-level background in mathematics. The wide range of areas covered - never before discussed together in a unified fashion - includes Markov processes and random walks, Langevin and Fokker-Planck equations, noise, generalized central limit theorem and extreme values statistics, random matrix theory and percolation theory.

This book provides an essential introduction to Stochastic Programming, especially intended for graduate students. The book begins by exploring a linear programming problem with random parameters, representing a decision problem under uncertainty. Several models for this problem are presented, including the main ones used in Stochastic Programming: recourse models and chance constraint models. The book not only discusses the theoretical properties of these models and algorithms for solving them, but also explains the intrinsic differences between the models. In the book's closing section, several case studies are presented, helping students apply the theory covered to practical problems. The book is based on lecture notes developed for an Econometrics and Operations Research course for master students at the University of Groningen, the Netherlands - the longest-standing Stochastic Programming course worldwide.

Elementary treatments of Markov chains, especially those devoted to discrete-time and finite state-space theory, leave the impression that everything is smooth and easy to understand. This exposition of the works of Kolmogorov, Feller, Chung, Kato, and other mathematical luminaries, which focuses on time-continuous chains but is not so far from being elementary itself, reminds us again that the impression is false: an infinite, but denumerable, state-space is where the fun begins. If you have not heard of Blackwell's example (in which all states are instantaneous), do not understand what the minimal process is, or do not know what happens after explosion, dive right in. But beware lest you are enchanted: 'There are more spells than your commonplace magicians ever dreamed of.'

Stochastic phenomena play a central role in various scientific disciplines and underpin applications in popular industrial sectors. The purpose of the book is to introduce the reader to advanced concepts in the analysis of stochastic models starting from a detailed, intuitive and yet rigorous presentation of basic concepts. A special emphasis will be placed on problem solving and numerical implementations, with detailed solutions to all of the results and source code in the C programming language provided. The book will also cover recent specialised techniques for popular problems, providing a valuable reference for advanced readers with an active interest in the field.

The relatively young theory of structured dependence between stochastic processes has many real-life applications in areas including finance, insurance, seismology, neuroscience, and genetics. With this monograph, the first to be devoted to the modeling of structured dependence between random processes, the authors not only meet the demand for a solid theoretical account but also develop a stochastic processes counterpart of the classical copula theory that exists for finite-dimensional random variables. Presenting both the technical aspects and the applications of the theory, this is a valuable reference for researchers and practitioners in the field, as well as for graduate students in pure and applied mathematics programs. Numerous theoretical examples are included, alongside examples of both current and potential applications, aimed at helping those who need to model structured dependence between dynamic random phenomena.