Meyn & Tweedie is back The bible on Markov chains in general state spaces has been brought up to date to reflect developments in the field since 1996 - many of them sparked by publication of the first edition. The pursuit of more efficient simulation algorithms for complex Markovian models, or algorithms for computation of optimal policies for controlled Markov models, has opened new directions for research on Markov chains. As a result, new applications have emerged across a wide range of topics including optimisation, statistics, and economics. New commentary and an epilogue by Sean Meyn summarise recent developments and references have been fully updated. This second edition reflects the same discipline and style that marked out the original and helped it to become a classic: proofs are rigorous and concise, the range of applications is broad and knowledgeable, and key ideas are accessible to practitioners with limited mathematical background.

Over the last thirty years there has been extensive use of continuous time econometric methods in macroeconomic modelling. This monograph presents the first continuous time macroeconometric model of the United Kingdom incorporating stochastic trends. Its development represents a major step forward in continuous time macroeconomic modelling. The book describes the new model in detail and, like earlier models, it is designed in such a way as to permit a rigorous mathematical analysis of its steady-state and stability properties, thus providing a valuable check on the capacity of the model to generate plausible long-run behaviour. The model is estimated using newly developed exact Gaussian estimation methods for continuous time econometric models incorporating unobservable stochastic trends. The book also includes discussion of the application of the model to dynamic analysis and forecasting.

This book was first published in 2006. Written by two of the foremost researchers in the field, this book studies the local times of Markov processes by employing isomorphism theorems that relate them to certain associated Gaussian processes. It builds to this material through self-contained but harmonized 'mini-courses' on the relevant ingredients, which assume only knowledge of measure-theoretic probability. The streamlined selection of topics creates an easy entrance for students and experts in related fields. The book starts by developing the fundamentals of Markov process theory and then of Gaussian process theory, including sample path properties. It then proceeds to more advanced results, bringing the reader to the heart of contemporary research. It presents the remarkable isomorphism theorems of Dynkin and Eisenbaum and then shows how they can be applied to obtain new properties of Markov processes by using well-established techniques in Gaussian process theory. This original, readable book will appeal to both researchers and advanced graduate students.

The book provides a general introduction to the theory of large deviations and a wide overview of the metastable behaviour of stochastic dynamics. With only minimal prerequisites, the book covers all the main results and brings the reader to the most recent developments. Particular emphasis is given to the fundamental Freidlin-Wentzell results on small random perturbations of dynamical systems. Metastability is first described on physical grounds, following which more rigorous approaches to its description are developed. Many relevant examples are considered from the point of view of the so-called pathwise approach. The first part of the book develops the relevant tools including the theory of large deviations which are then used to provide a physically relevant dynamical description of metastability. Written to be accessible to graduate students, this book provides an excellent route into contemporary research.

Twenty-five articles have been selected from the first 14 volumes of the "SA(c)minaire de ProbabilitA(c)s," all out of print, for their historical and/or mathematical interest. Among the many articles devoted to Martingale theory in the early volumes of the SA(c)minaire, we have chosen to reprint those that are particularly significant from a historical point of view, as well as those that can still be useful today. They are reprinted here verbatim, with a short retrospective comment, for the benefit of researchers in the theory of stochastic processes, in mathematical finance, or in history of mathematics.

This third edition is a revised, updated, and greatly expanded version of previous edition of 2001. The 1300+ exercises contained within are not merely drill problems, but have been chosen to illustrate the concepts, illuminate the subject, and both inform and entertain the reader. A broad range of subjects is covered, including elementary aspects of probability and random variables, sampling, generating functions, Markov chains, convergence, stationary processes, renewals, queues, martingales, diffusions, Levy processes, stability and self-similarity, time changes, and stochastic calculus including option pricing via the Black-Scholes model of mathematical finance. The text is intended to serve students as a companion for elementary, intermediate, and advanced courses in probability, random processes and operations research. It will also be useful for anyone needing a source for large numbers of problems and questions in these fields. In particular, this book acts as a companion to the authors' volume, Probability and Random Processes, fourth edition (OUP 2020).

This important work addresses problems in financial mathematics of pricing and hedging derivative securities in an environment of uncertain and changing market volatility. These problems are important to investors from large trading institutions to pension funds. The authors present mathematical and statistical tools that exploit the volatile nature of the market. The mathematics is introduced through examples and illustrated with simulations and the modeling approach that is described is validated and tested on market data. The material is suitable for a one-semester course for graduate students with some exposure to methods of stochastic modeling and arbitrage pricing theory in finance. The volume is easily accessible to derivatives practitioners in the financial engineering industry.

The second volume concentrates on stochastic integrals, stochastic differential equations, excursion theory and the general theory of processes. These subjects are made accessible in the many concrete examples that illustrate techniques of calculation, and in the treatment of all topics from the ground up, starting from simple cases. Many of the examples and proofs are new; some important calculational techniques appear for the first time in this book.

Now available in paperback, this celebrated book remains a key systematic guide to a large part of the modern theory of Probability. The authors not only present the subject of Brownian motion as a dry part of mathematical analysis, but convey its real meaning and fascination. The opening, heuristic chapter does just this, and it is followed by a comprehensive and self-contained account of the foundations of theory of stochastic processes. Chapter 3 is a lively presentation of the theory of Markov processes. Together with its companion volume, this book equips graduate students for research into a subject of great intrinsic interest and wide applications.

All the papers in the volume are original research papers, discussing fundamental properties of stochastic processes. The topics under study (martingales, filtrations, path properties, etc.) represent an important part of the current research performed in 1996-97 by various groups of probabilists in France and abroad.

Nearly one hundred years ago Jacques Hadamard used infinite sequences of symbols to analyze the distribution of geodesics on certain surfaces. That was the beginning of symbolic dynamics. In the 1930's and 40's Arnold Hedlund and Marston Morse again used infinite sequences to investigate geodesics on surfaces of negative curvature. They coined the term symbolic dynamics and began to study sequence spaces with the shift transformation as dynamical systems. In the 1940's Claude Shannon used sequence spaces to describe infor mation channels. Since that time symbolic dynamics has been used in ergodic theory, topological dynamics, hyperbolic dynamics, information theory and complex dynamics. Symbolic dynamical systems with a finite memory are stud ied in this book. They are the topological Markov shifts. Each can be defined by transition rules and the rules can be summarized by a transition matrix. The study naturally divides into two parts. The first part is about topological Markov shifts where the alphabet is finite. The second part is concerned with topological Markov shifts whose alphabet is count ably infinite. The techniques used in the two cases are quite different. When the alphabet is finite most of the methods are combinatorial or algebraic. When the alphabet is infinite the methods are much more analytic. This book grew from notes for a graduate course taught at Wesleyan Uni versity in the fall of 1994 and is intended as a graduate text and as a reference book for mathematicians working in related fields."

The different papers contained in this volume are all research papers. The main directions of research which are being developed are: quantum probability, semimartingales and stochastic calculus.

Probability comes of age with this, the first dictionary of probability and its applications in English, which supplies a guide to the concepts and vocabulary of this rapidly expanding field. Besides the basic theory of probability and random processes, applications covered here include financial and insurance mathematics, operations research (including queueing, reliability, and inventories), decision and game theory, optimization, time series, networks, and communication theory, as well as classic problems and paradoxes. The dictionary is reliable, stable, concise, and cohesive. Each entry provides a rigorous definition, a sketch of the context, and a reference pointing the reader to the wider literature. Judicious use of figures makes complex concepts easier to follow without oversimplifying. As the only dictionary on the market, this will be a guiding reference for all those working in, or learning, probability together with its applications.

A cornerstone of applied probability, Markov chains can be used to help model how plants grow, chemicals react, and atoms diffuse--and applications are increasingly being found in such areas as engineering, computer science, economics, and education. To apply the techniques to real problems, however, it is necessary to understand how Markov chains can be solved numerically. In this book, the first to offer a systematic and detailed treatment of the numerical solution of Markov chains, William Stewart provides scientists on many levels with the power to put this theory to use in the actual world, where it has applications in areas as diverse as engineering, economics, and education. His efforts make for essential reading in a rapidly growing field.

Here Stewart explores all aspects of numerically computing solutions of Markov chains, especially when the state is huge. He provides extensive background to both discrete-time and continuous-time Markov chains and examines many different numerical computing methods--direct, single-and multi-vector iterative, and projection methods. More specifically, he considers recursive methods often used when the structure of the Markov chain is upper Hessenberg, iterative aggregation/disaggregation methods that are particularly appropriate when it is NCD (nearly completely decomposable), and reduced schemes for cases in which the chain is periodic. There are chapters on methods for computing transient solutions, on stochastic automata networks, and, finally, on currently available software. Throughout Stewart draws on numerous examples and comparisons among the methods he so thoroughly explains.

Kiyosi Ito's greatest contribution to probability theory may be his introduction of stochastic differential equations to explain the Kolmogorov-Feller theory of Markov processes. Starting with the geometric ideas that guided him, this book gives an account of Ito's program.

The modern theory of Markov processes was initiated by A. N. Kolmogorov. However, Kolmogorov's approach was too analytic to reveal the probabilistic foundations on which it rests. In particular, it hides the central role played by the simplest Markov processes: those with independent, identically distributed increments. To remedy this defect, Ito interpreted Kolmogorov's famous forward equation as an equation that describes the integral curve of a vector field on the space of probability measures. Thus, in order to show how Ito's thinking leads to his theory of stochastic integral equations, Stroock begins with an account of integral curves on the space of probability measures and then arrives at stochastic integral equations when he moves to a pathspace setting. In the first half of the book, everything is done in the context of general independent increment processes and without explicit use of Ito's stochastic integral calculus. In the second half, the author provides a systematic development of Ito's theory of stochastic integration: first for Brownian motion and then for continuous martingales. The final chapter presents Stratonovich's variation on Ito's theme and ends with an application to the characterization of the paths on which a diffusion is supported.

The book should be accessible to readers who have mastered the essentials of modern probability theory and should provide such readers with a reasonably thorough introduction to continuous-time, stochastic processes."

Most branches of science involving random fluctuations can be approached by Stochastic Calculus. These include, but are not limited to, signal processing, noise filtering, stochastic control, optimal stopping, electrical circuits, financial markets, molecular chemistry, population dynamics, etc. All these applications assume a strong mathematical background, which in general takes a long time to develop. Stochastic Calculus is not an easy to grasp theory, and in general, requires acquaintance with the probability, analysis and measure theory.The goal of this book is to present Stochastic Calculus at an introductory level and not at its maximum mathematical detail. The author's goal was to capture as much as possible the spirit of elementary deterministic Calculus, at which students have been already exposed. This assumes a presentation that mimics similar properties of deterministic Calculus, which facilitates understanding of more complicated topics of Stochastic Calculus.The second edition contains several new features that improved the first edition both qualitatively and quantitatively. First, two more chapters have been added, Chapter 12 and Chapter 13, dealing with applications of stochastic processes in Electrochemistry and global optimization methods.This edition contains also a final chapter material containing fully solved review problems and provides solutions, or at least valuable hints, to all proposed problems. The present edition contains a total of about 250 exercises.This edition has also improved presentation from the first edition in several chapters, including new material.

This textbook introduces the theory of stochastic processes, that is, randomness which proceeds in time. Using concrete examples like repeated gambling and jumping frogs, it presents fundamental mathematical results through simple, clear, logical theorems and examples. It covers in detail such essential material as Markov chain recurrence criteria, the Markov chain convergence theorem, and optional stopping theorems for martingales. The final chapter provides a brief introduction to Brownian motion, Markov processes in continuous time and space, Poisson processes, and renewal theory.Interspersed throughout are applications to such topics as gambler's ruin probabilities, random walks on graphs, sequence waiting times, branching processes, stock option pricing, and Markov Chain Monte Carlo (MCMC) algorithms.The focus is always on making the theory as well-motivated and accessible as possible, to allow students and readers to learn this fascinating subject as easily and painlessly as possible.

This book provides an overview of global sensitivity analysis methods and algorithms, including their theoretical basis and mathematical properties. The authors use a practical point of view and real case studies as well as numerous examples, and applications of the different approaches are illustrated throughout using R code to explain their usage and usefulness in practice. Basics and Trends in Sensitivity Analysis: Theory and Practice in R covers a lot of material, including theoretical aspects of Sobol' indices as well as sampling-based formulas, spectral methods, and metamodel-based approaches for estimation purposes; screening techniques devoted to identifying influential and noninfluential inputs; variance-based measures when model inputs are statistically dependent (and several other approaches that go beyond variance-based sensitivity measures); and a case study in R related to a COVID-19 epidemic model where the full workflow of sensitivity analysis combining several techniques is presented. This book is intended for engineers, researchers, and undergraduate students who use complex numerical models and have an interest in sensitivity analysis techniques and is appropriate for anyone with a solid mathematical background in basic statistical and probability theories who develops and uses numerical models in all scientific and engineering domains.