By reducing mathematical detail and focusing on real-world applications, this book provides engineers with an easy-to-understand overview of stochastic modeling. An entire chapter is included on how to set up the problem, and then another complete chapter presents examples of applications before doing any math. A previously unpublished computational method for solving equations related to Markov processes is added. The book shows how to add costs or revenues to the basic probability structures without much additional effort. In addition, numerous examples are included that show how the theory can be used. Engineers will also find explanations on how to formulate word problems into the models that the math worked on.

Highlighting modern computational methods, Applied Stochastic Modelling, Second Edition provides students with the practical experience of scientific computing in applied statistics through a range of interesting real-world applications. It also successfully revises standard probability and statistical theory. Along with an updated bibliography and improved figures, this edition offers numerous updates throughout. New to the Second Edition An extended discussion on Bayesian methods A large number of new exercises A new appendix on computational methods The book covers both contemporary and classical aspects of statistics, including survival analysis, Kernel density estimation, Markov chain Monte Carlo, hypothesis testing, regression, bootstrap, and generalised linear models. Although the book can be used without reference to computational programs, the author provides the option of using powerful computational tools for stochastic modelling. All of the data sets and MATLAB (R) and R programs found in the text as well as lecture slides and other ancillary material are available for download at www.crcpress.com Continuing in the bestselling tradition of its predecessor, this textbook remains an excellent resource for teaching students how to fit stochastic models to data.

This book presents state-of-the-art solution methods and applications of stochastic optimal control. It is a collection of extended papers discussed at the traditional Liverpool workshop on controlled stochastic processes with participants from both the east and the west. New problems are formulated, and progresses of ongoing research are reported. Topics covered in this book include theoretical results and numerical methods for Markov and semi-Markov decision processes, optimal stopping of Markov processes, stochastic games, problems with partial information, optimal filtering, robust control, Q-learning, and self-organizing algorithms. Real-life case studies and applications, e.g., queueing systems, forest management, control of water resources, marketing science, and healthcare, are presented. Scientific researchers and postgraduate students interested in stochastic optimal control,- as well as practitioners will find this book appealing and a valuable reference.

The book contains two contributions about the work of Emmanuele DiBenedetto and a selection of original papers. The authors are some of the main experts in Harnack's inequalities and nonlinear operators. These papers are part of the contributions presented during the conference to celebrate the 70th birthday of Prof. Emmanuele DiBenedetto, which was held at "Il Palazzone" in Cortona from June 18th to 24th, 2017. The papers are focused on current research topics regarding the qualitative properties of solutions, connections with calculus of variations, Harnack inequality and regularity theory. Some papers are also related to various applications. Many of the authors have shared with Prof. DiBenedetto an intense scientific and personal collaboration, while many others have taken inspiration from and further developed his field of research. The topics of the conference are certainly of great interest for the international mathematical community.

The book provides a systemic treatment of time-dependent strong Markov processes with values in a Polish space. It describes its generators and the link with stochastic differential equations in infinite dimensions. In a unifying way, where the square gradient operator is employed, new results for backward stochastic differential equations and long-time behavior are discussed in depth. The book also establishes a link between propagators or evolution families with the Feller property and time-inhomogeneous Markov processes. This mathematical material finds its applications in several branches of the scientific world, among which are mathematical physics, hedging models in financial mathematics, and population models.

This book is aimed at researchers, graduate students and engineers who would like to be initiated to Piecewise Deterministic Markov Processes (PDMPs). A PDMP models a deterministic mechanism modified by jumps that occur at random times. The fields of applications are numerous : insurance and risk, biology, communication networks, dependability, supply management, etc. Indeed, the PDMPs studied so far are in fact deterministic functions of CSMPs (Completed Semi-Markov Processes), i.e. semi-Markov processes completed to become Markov processes. This remark leads to considerably broaden the definition of PDMPs and allows their properties to be deduced from those of CSMPs, which are easier to grasp. Stability is studied within a very general framework. In the other chapters, the results become more accurate as the assumptions become more precise. Generalized Chapman-Kolmogorov equations lead to numerical schemes. The last chapter is an opening on processes for which the deterministic flow of the PDMP is replaced with a Markov process. Marked point processes play a key role throughout this book.

This textbook, now in its fourth edition, offers a rigorous and self-contained introduction to the theory of continuous-time stochastic processes, stochastic integrals, and stochastic differential equations. Expertly balancing theory and applications, it features concrete examples of modeling real-world problems from biology, medicine, finance, and insurance using stochastic methods. No previous knowledge of stochastic processes is required. Unlike other books on stochastic methods that specialize in a specific field of applications, this volume examines the ways in which similar stochastic methods can be applied across different fields. Beginning with the fundamentals of probability, the authors go on to introduce the theory of stochastic processes, the Ito Integral, and stochastic differential equations. The following chapters then explore stability, stationarity, and ergodicity. The second half of the book is dedicated to applications to a variety of fields, including finance, biology, and medicine. Some highlights of this fourth edition include a more rigorous introduction to Gaussian white noise, additional material on the stability of stochastic semigroups used in models of population dynamics and epidemic systems, and the expansion of methods of analysis of one-dimensional stochastic differential equations. An Introduction to Continuous-Time Stochastic Processes, Fourth Edition is intended for graduate students taking an introductory course on stochastic processes, applied probability, stochastic calculus, mathematical finance, or mathematical biology. Prerequisites include knowledge of calculus and some analysis; exposure to probability would be helpful but not required since the necessary fundamentals of measure and integration are provided. Researchers and practitioners in mathematical finance, biomathematics, biotechnology, and engineering will also find this volume to be of interest, particularly the applications explored in the second half of the book.

This monograph presents the geoscientific context arising in decorrelative gravitational exploration to determine the mass density distribution inside the Earth. First, an insight into the current state of research is given by reducing gravimetry to mathematically accessible, and thus calculable, decorrelated models. In this way, the various unresolved questions and problems of gravimetry are made available to a broad scientific audience and the exploration industry. New theoretical developments will be given, and innovative ways of modeling geologic layers and faults by mollifier regularization techniques are shown. This book is dedicated to surface as well as volume geology with potential data primarily of terrestrial origin. For deep geology, the geomathematical decorrelation methods are to be designed in such a way that depth information (e.g., in boreholes) may be canonically entered. Bridging several different geo-disciplines, this book leads in a cycle from the potential measurements made by geoengineers, to the cleansing of data by geophysicists and geoengineers, to the subsequent theory and model formation, computer-based implementation, and numerical calculation and simulations made by geomathematicians, to interpretation by geologists, and, if necessary, back. It therefore spans the spectrum from geoengineering, especially geodesy, via geophysics to geomathematics and geology, and back. Using the German Saarland area for methodological tests, important new fields of application are opened, particularly for regions with mining-related cavities or dense development in today's geo-exploration.

The aim of this book is to promote interaction between engineering, finance and insurance, as these three domains have many models and methods of solution in common for solving real-life problems. The authors point out the strict inter-relations that exist among the diffusion models used in engineering, finance and insurance. In each of the three fields, the basic diffusion models are presented and their strong similarities are discussed. Analytical, numerical and Monte Carlo simulation methods are explained with a view to applying them to obtain the solutions to the different problems presented in the book. Advanced topics such as nonlinear problems, Levy processes and semi-Markov models in interactions with the diffusion models are discussed, as well as possible future interactions among engineering, finance and insurance.

Contents

1. Diffusion Phenomena and Models.2. Probabilistic Models of Diffusion Processes.3. Solving Partial Differential Equations of Second Order.4. Problems in Finance.5. Basic PDE in Finance.6. Exotic and American Options Pricing Theory.7. Hitting Times for Diffusion Processes and Stochastic Models in Insurance.8. Numerical Methods.9. Advanced Topics in Engineering: Nonlinear Models.10. Levy Processes.11. Advanced Topics in Insurance: Copula Models and VaR Techniques.12. Advanced Topics in Finance: Semi-Markov Models.13. Monte Carlo Semi-Markov Simulation Methods.

About the Authors

Jacques Janssen is now Honorary Professor at the Solvay Business School (ULB) in Brussels, Belgium, having previously taught at EURIA (Euro-Institut d'Actuariat, University of West Brittany, Brest, France) and Telecom-Bretagne (Brest, France) as well as being a director of Jacan Insurance and Finance Services, a consultancy and training company.Oronzio Manca is Professor of thermal sciences at Seconda Universita degli Studi di Napoli in Italy. He is currently Associate Editor of ASME Journal of Heat Transfer and Journal of Porous Media and a member of the editorial advisory boards for The Open Thermodynamics Journal, Advances in Mechanical Engineering, The Open Fuels & Energy Science Journal.Raimondo Manca is Professor of mathematical methods applied to economics, finance and actuarial science at University of Rome "La Sapienza" in Italy. He is associate editor for the journal Methodology and Computing in Applied Probability. His main research interests are multidimensional linear algebra, computational probability, application of stochastic processes to economics, finance and insurance and simulation models.

This collection of contributions originates from the well-established conference series "Fractal Geometry and Stochastics" which brings together researchers from different fields using concepts and methods from fractal geometry. Carefully selected papers from keynote and invited speakers are included, both discussing exciting new trends and results and giving a gentle introduction to some recent developments. The topics covered include Assouad dimensions and their connection to analysis, multifractal properties of functions and measures, renewal theorems in dynamics, dimensions and topology of random discrete structures, self-similar trees, p-hyperbolicity, phase transitions from continuous to discrete scale invariance, scaling limits of stochastic processes, stemi-stable distributions and fractional differential equations, and diffusion limited aggregation. Representing a rich source of ideas and a good starting point for more advanced topics in fractal geometry, the volume will appeal to both established experts and newcomers.

This book deals with certain important problems in Classical and Quantum Information Theory Quantum Information Theory, A Selection of Matrix Inequalities Stochastic Filtering Theory Applied to Electromagnetic Fields and Strings Wigner-distributions in Quantum Mechanics Quantization of Classical Field Theories Statistical Signal Processing Quantum Field Theory, Quantum Statistics, Gravity, Stochastic Fields and Information Problems in Information Theory It will be very helpful for students of Undergraduate and Postgraduate Courses in Electronics, Communication and Signal Processing. Print edition not for sale in South Asia (India, Sri Lanka, Nepal, Bangladesh, Pakistan or Bhutan).

This book is a collection of selected papers presented at the International Conference on Semigroups and Applications, held at the Cochin University of Science and Technology, India, from December 9-12, 2019. This book discusses the recent developments in semigroups theory, category theory and the applications of these in various areas of research, including structure theory of semigroups, lattices, rings and partial algebras. This book presents chapters on ordering orders and quotient rings, block groups and Hall's relations, quotients of the Booleanization of inverse semigroup, Markov chains through semigroup graph expansions, polycyclic inverse monoids and Thompson group, balanced category and bundle category. This book will be of much value to researchers working in areas of semigroup and operator theory.

Products of Random Variables explores the theory of products of random variables through from distributions and limit theorems, to characterizations, to applications in physics, order statistics, and number theory. It uses entirely probabilistic arguments in actualizing the potential of the asymptotic theory of products of independent random variables and obtaining results with dependent variables using a new Bonferroni-type argument. Systematically and comprehensively tracks the progression of research completed in the area over the last twenty years. Well-indexed and well-referenced, Products of Random Variables -Clarifies foundational concepts such as symmetric and limiting distributions of products -Examines various limit theorems, from logarithmically Poisson distributions to triangular arrays -Explores characterization theorems, detailing normal, Cauchy, and bivariate distributions -Describes models of interactive particles -Elucidates dual systems of interactive particles, dual systems of increasing size, and random walks -Covers the Kubilius-Turan inequality and distributions for multiplicative functions -Probes sequences of prime divisors and prime numbers -Discusses Markov chains, Hilbert spaces, and quotients of random variables -Presents income growth models and numerous other applied models tapping products of random variables Authored by eminent scholars in the field, this volume is an important research reference for applied mathematicians, statisticians, physicists, and graduate students in these disciplines.

This book provides, as simply as possible, sound foundations for an in-depth understanding of reliability engineering with regard to qualitative analysis, modelling, and probabilistic calculations of safety and production systems. Drawing on the authors' extensive experience within the field of reliability engineering, it addresses and discusses a variety of topics, including: * Background and overview of safety and dependability studies; * Explanation and critical analysis of definitions related to core concepts; * Risk identification through qualitative approaches (preliminary hazard analysis, HAZOP, FMECA, etc.); * Modelling of industrial systems through static (fault tree, reliability block diagram), sequential (cause-consequence diagrams, event trees, LOPA, bowtie), and dynamic (Markov graphs, Petri nets) approaches; * Probabilistic calculations through state-of-the-art analytical or Monte Carlo simulation techniques; * Analysis, modelling, and calculations of common cause failure and uncertainties; * Linkages and combinations between the various modelling and calculation approaches; * Reliability data collection and standardization. The book features illustrations, explanations, examples, and exercises to help readers gain a detailed understanding of the topic and implement it into their own work. Further, it analyses the production availability of production systems and the functional safety of safety systems (SIL calculations), showcasing specific applications of the general theory discussed. Given its scope, this book is a valuable resource for engineers, software designers, standard developers, professors, and students.

The book presents, for the first time, a detailed analysis of harmonizable processes and fields (in the weak sense) that contain the corresponding stationary theory as a subclass. It also gives the structural and some key applications in detail. These include Levy's Brownian motion, a probabilistic proof of the longstanding Riemann's hypothesis, random fields indexed by LCA and hypergroups, extensions to bistochastic operators, Cramer-Karhunen classes, as well as bistochastic operators with some statistical applications.The material is accessible to graduate students in probability and statistics as well as to engineers in theoretical applications. There are numerous extensions and applications pointed out in the book that will inspire readers to delve deeper.

The prolonged boom in the US and European stock markets has led to increased interest in the mathematics of security markets, most notably in the theory of stochastic integration. This text gives a rigorous development of the theory of stochastic integration as it applies to the valuation of derivative securities. It includes all the tools necessary for readers to understand how the stochastic integral is constructed with respect to a general continuous martingale.

The author develops the stochastic calculus from first principles, but at a relaxed pace that includes proofs that are detailed, but streamlined to applications to finance. The treatment requires minimal prerequisites-a basic knowledge of measure theoretic probability and Hilbert space theory-and devotes an entire chapter to application in finances, including the Black Scholes market, pricing contingent claims, the general market model, pricing of random payoffs, and interest rate derivatives.

Continuous Stochastic Calculus with Application to Finance is your first opportunity to explore stochastic integration at a reasonable and practical mathematical level. It offers a treatment well balanced between aesthetic appeal, degree of generality, depth, and ease of reading.

This book presents the probabilistic methods around Hardy martingales for an audience interested in their applications to complex, harmonic, and functional analysis. Building on work of Bourgain, Garling, Jones, Maurey, Pisier, and Varopoulos, it discusses in detail those martingale spaces that reflect characteristic qualities of complex analytic functions. Its particular themes are holomorphic random variables on Wiener space, and Hardy martingales on the infinite torus product, and numerous deep applications to the geometry and classification of complex Banach spaces, e.g., the SL estimates for Doob's projection operator, the embedding of L1 into L1/H1, the isomorphic classification theorem for the polydisk algebras, or the real variables characterization of Banach spaces with the analytic Radon Nikodym property. Due to the inclusion of key background material on stochastic analysis and Banach space theory, it's suitable for a wide spectrum of researchers and graduate students working in classical and functional analysis.

A high school student can create deep Q-learning code to control her robot, without any understanding of the meaning of 'deep' or 'Q', or why the code sometimes fails. This book is designed to explain the science behind reinforcement learning and optimal control in a way that is accessible to students with a background in calculus and matrix algebra. A unique focus is algorithm design to obtain the fastest possible speed of convergence for learning algorithms, along with insight into why reinforcement learning sometimes fails. Advanced stochastic process theory is avoided at the start by substituting random exploration with more intuitive deterministic probing for learning. Once these ideas are understood, it is not difficult to master techniques rooted in stochastic control. These topics are covered in the second part of the book, starting with Markov chain theory and ending with a fresh look at actor-critic methods for reinforcement learning.

Compound renewal processes (CRPs) are among the most ubiquitous models arising in applications of probability. At the same time, they are a natural generalization of random walks, the most well-studied classical objects in probability theory. This monograph, written for researchers and graduate students, presents the general asymptotic theory and generalizes many well-known results concerning random walks. The book contains the key limit theorems for CRPs, functional limit theorems, integro-local limit theorems, large and moderately large deviation principles for CRPs in the state space and in the space of trajectories, including large deviation principles in boundary crossing problems for CRPs, with an explicit form of the rate functionals, and an extension of the invariance principle for CRPs to the domain of moderately large and small deviations. Applications establish the key limit laws for Markov additive processes, including limit theorems in the domains of normal and large deviations.

Optimal Control and Optimization of Stochastic Supply Chain Systems examines its subject the context of the presence of a variety of uncertainties. Numerous examples with intuitive illustrations and tables are provided, to demonstrate the structural characteristics of the optimal control policies in various stochastic supply chains and to show how to make use of these characteristics to construct easy-to-operate sub-optimal policies. In Part I, a general introduction to stochastic supply chain systems is provided. Analytical models for various stochastic supply chain systems are formulated and analysed in Part II. In Part III the structural knowledge of the optimal control policies obtained in Part II is utilized to construct easy-to-operate sub-optimal control policies for various stochastic supply chain systems accordingly. Finally, Part IV discusses the optimisation of threshold-type control policies and their robustness. A key feature of the book is its tying together of the complex analytical models produced by the requirements of operational practice, and the simple solutions needed for implementation. The analytical models and theoretical analysis propounded in this monograph will be of benefit to academic researchers and graduate students looking at logistics and supply chain management from standpoints in operations research or industrial, manufacturing, or control engineering. The practical tools and solutions and the qualitative insights into the ideas underlying functional supply chain systems will be of similar use to readers from more industrially-based backgrounds.

This book discusses recent developments in dynamic reliability in multi-state systems (MSS), addressing such important issues as reliability and availability analysis of aging MSS, the impact of initial conditions on MSS reliability and availability, changing importance of components over time in MSS with aging components, and the determination of age-replacement policies. It also describes modifications of traditional methods, such as Markov processes with rewards, as well as a modern mathematical method based on the extended universal generating function technique, the Lz-transform, presenting various successful applications and demonstrating their use in real-world problems. This book provides theoretical insights, information on practical applications, and real-world case studies that are of interest to engineers and industrial managers as well as researchers. It also serves as a textbook or supporting text for graduate and postgraduate courses in industrial, electrical, and mechanical engineering.

Statistical inference carries great significance in model building from both the theoretical and the applications points of view. Its applications to engineering and economic systems, financial economics, and the biological and medical sciences have made statistical inference for stochastic processes a well-recognized and important branch of statistics and probability.
The class of semimartingales includes a large class of stochastic processes, including diffusion type processes, point processes, and diffusion type processes with jumps, widely used for stochastic modeling. Until now, however, researchers have had no single reference that collected the research conducted on the asymptotic theory for semimartingales.

Semimartingales and their Statistical Inference, fills this need by presenting a comprehensive discussion of the asymptotic theory of semimartingales at a level needed for researchers working in the area of statistical inference for stochastic processes. The author brings together into one volume the state-of-the-art in the inferential aspect for such processes. The topics discussed include:

· Asymptotic likelihood theory
· Quasi-likelihood
· Likelihood and efficiency
· Inference for counting processes
· Inference for semimartingale regression models

The author addresses a number of stochastic modeling applications from engineering, economic systems, financial economics, and medical sciences. He also includes some of the new and challenging statistical and probabilistic problems facing today's active researchers working in the area of inference for stochastic processes.

This book provides a unified approach for the study of constrained Markov decision processes with a finite state space and unbounded costs. Unlike the single controller case considered in many other books, the author considers a single controller with several objectives, such as minimizing delays and loss, probabilities, and maximization of throughputs. It is desirable to design a controller that minimizes one cost objective, subject to inequality constraints on other cost objectives. This framework describes dynamic decision problems arising frequently in many engineering fields. A thorough overview of these applications is presented in the introduction.
The book is then divided into three sections that build upon each other.
The first part explains the theory for the finite state space. The author characterizes the set of achievable expected occupation measures as well as performance vectors, and identifies simple classes of policies among which optimal policies exist. This allows the reduction of the original dynamic into a linear program. A Lagranian approach is then used to derive the dual linear program using dynamic programming techniques.
In the second part, these results are extended to the infinite state space and action spaces. The author provides two frameworks: the case where costs are bounded below and the contracting framework.
The third part builds upon the results of the first two parts and examines asymptotical results of the convergence of both the value and the policies in the time horizon and in the discount factor. Finally, several state truncation algorithms that enable the approximation of the solution of the original control problem via finite linear programs are given.

The main goal of this book is to systematically address the mathematical methods that are applied in the study of synchronization of infinite-dimensional evolutionary dissipative or partially dissipative systems. It bases its unique monograph presentation on both general and abstract models and covers several important classes of coupled nonlinear deterministic and stochastic PDEs which generate infinite-dimensional dissipative systems. This text, which adapts readily to advanced graduate coursework in dissipative dynamics, requires some background knowledge in evolutionary equations and introductory functional analysis as well as a basic understanding of PDEs and the theory of random processes. Suitable for researchers in synchronization theory, the book is also relevant to physicists and engineers interested in both the mathematical background and the methods for the asymptotic analysis of coupled infinite-dimensional dissipative systems that arise in continuum mechanics.

This book is dedicated to the systematization and development of models, methods, and algorithms for queuing systems with correlated arrivals. After first setting up the basic tools needed for the study of queuing theory, the authors concentrate on complicated systems: multi-server systems with phase type distribution of service time or single-server queues with arbitrary distribution of service time or semi-Markovian service. They pay special attention to practically important retrial queues, tandem queues, and queues with unreliable servers. Mathematical models of networks and queuing systems are widely used for the study and optimization of various technical, physical, economic, industrial, and administrative systems, and this book will be valuable for researchers, graduate students, and practitioners in these domains.