This book presents classical Markov Decision Processes (MDP) for real-life applications and optimization. MDP allows users to develop and formally support approximate and simple decision rules, and this book showcases state-of-the-art applications in which MDP was key to the solution approach. The book is divided into six parts. Part 1 is devoted to the state-of-the-art theoretical foundation of MDP, including approximate methods such as policy improvement, successive approximation and infinite state spaces as well as an instructive chapter on Approximate Dynamic Programming. It then continues with five parts of specific and non-exhaustive application areas. Part 2 covers MDP healthcare applications, which includes different screening procedures, appointment scheduling, ambulance scheduling and blood management. Part 3 explores MDP modeling within transportation. This ranges from public to private transportation, from airports and traffic lights to car parking or charging your electric car . Part 4 contains three chapters that illustrates the structure of approximate policies for production or manufacturing structures. In Part 5, communications is highlighted as an important application area for MDP. It includes Gittins indices, down-to-earth call centers and wireless sensor networks. Finally Part 6 is dedicated to financial modeling, offering an instructive review to account for financial portfolios and derivatives under proportional transactional costs. The MDP applications in this book illustrate a variety of both standard and non-standard aspects of MDP modeling and its practical use. This book should appeal to readers for practitioning, academic research and educational purposes, with a background in, among others, operations research, mathematics, computer science, and industrial engineering.

This book presents in-depth coverage of laboratory experiments, theories, modeling techniques, and practices for the analysis and design of rock slopes in complex geological settings. It addresses new concepts in connection with the kinematical element method, discontinuity kinematical element method, integrated karst cave stochastic model-limit equilibrium method, improved strength reduction method, and fracture mechanics method, taking into account the relevant geological features. The book is chiefly intended as a reference guide for geotechnical engineering and engineering geology professionals, and as a textbook for related graduate courses.

2020 Taylor & Francis Award Winner for Outstanding New Textbook! Featuring recent advances in the field, this new textbook presents probability and statistics, and their applications in stochastic processes. This book presents key information for understanding the essential aspects of basic probability theory and concepts of reliability as an application. The purpose of this book is to provide an option in this field that combines these areas in one book, balances both theory and practical applications, and also keeps the practitioners in mind. Features Includes numerous examples using current technologies with applications in various fields of study Offers many practical applications of probability in queueing models, all of which are related to the appropriate stochastic processes (continuous time such as waiting time, and fuzzy and discrete time like the classic Gambler's Ruin Problem) Presents different current topics like probability distributions used in real-world applications of statistics such as climate control and pollution Different types of computer software such as MATLAB (R), Minitab, MS Excel, and R as options for illustration, programing and calculation purposes and data analysis Covers reliability and its application in network queues

Since the parameters in dynamical systems of biological interest are inherently positive and bounded, bounded noises are a natural way to model the realistic stochastic fluctuations of a biological system that are caused by its interaction with the external world. Bounded Noises in Physics, Biology, and Engineering is the first contributed volume devoted to the modeling of bounded noises in theoretical and applied statistical mechanics, quantitative biology, and mathematical physics. It gives an overview of the current state-of-the-art and is intended to stimulate further research. The volume is organized in four parts. The first part presents the main kinds of bounded noises and their applications in theoretical physics. The theory of bounded stochastic processes is intimately linked to its applications to mathematical and statistical physics, and it would be difficult and unnatural to separate the theory from its physical applications. The second is devoted to framing bounded noises in the theory of random dynamical systems and random bifurcations, while the third is devoted to applications of bounded stochastic processes in biology, one of the major areas of potential applications of this subject. The final part concerns the application of bounded stochastic processes in mechanical and structural engineering, the area where the renewed interest for non-Gaussian bounded noises started. Pure mathematicians working on stochastic calculus will find here a rich source of problems that are challenging from the point of view of contemporary nonlinear analysis. Bounded Noises in Physics, Biology, and Engineering is intended for scientists working on stochastic processes with an interest in both fundamental issues and applications. It will appeal to a broad range of applied mathematicians, mathematical biologists, physicists, engineers, and researchers in other fields interested in complexity theory. It is accessible to anyone with a working knowledge of stochastic modeling, from advanced undergraduates to senior researchers.

It is well-known that modern stochastic calculus has been exhaustively developed under usual conditions. Despite such a well-developed theory, there is evidence to suggest that these very convenient technical conditions cannot necessarily be fulfilled in real-world applications. Optional Processes: Theory and Applications seeks to delve into the existing theory, new developments and applications of optional processes on "unusual" probability spaces. The development of stochastic calculus of optional processes marks the beginning of a new and more general form of stochastic analysis. This book aims to provide an accessible, comprehensive and up-to-date exposition of optional processes and their numerous properties. Furthermore, the book presents not only current theory of optional processes, but it also contains a spectrum of applications to stochastic differential equations, filtering theory and mathematical finance. Features Suitable for graduate students and researchers in mathematical finance, actuarial science, applied mathematics and related areas Compiles almost all essential results on the calculus of optional processes in unusual probability spaces Contains many advanced analytical results for stochastic differential equations and statistics pertaining to the calculus of optional processes Develops new methods in finance based on optional processes such as a new portfolio theory, defaultable claim pricing mechanism, etc.

Focusing on the importance of the application of statistical techniques, this book covers the design of experiments and stochastic modeling in textile engineering. Textile Engineering: Statistical Techniques, Design of Experiments and Stochastic Modeling focuses on the analysis and interpretation of textile data for improving the quality of textile processes and products using various statistical techniques. FEATURES Explores probability, random variables, probability distribution, estimation, significance test, ANOVA, acceptance sampling, control chart, regression and correlation, design of experiments and stochastic modeling pertaining to textiles Presents step-by-step mathematical derivations Includes MATLAB (R) codes for solving various numerical problems Consists of case studies, practical examples and homework problems in each chapter This book is aimed at graduate students, researchers and professionals in textile engineering, textile clothing, textile management and industrial engineering. This book is equally useful for learners and practitioners in other scientific and technological domains.

Networked control systems are increasingly ubiquitous today, with applications ranging from vehicle communication and adaptive power grids to space exploration and economics. The optimal design of such systems presents major challenges, requiring tools from various disciplines within applied mathematics such as decentralized control, stochastic control, information theory, and quantization.

A thorough, self-contained book, "Stochastic Networked Control Systems: Stabilization and Optimization under Information Constraints" aims to connect these diverse disciplines with precision and rigor, while conveying design guidelines to controller architects. Unique in the literature, it lays a comprehensive theoretical foundation for the study of networked control systems, and introduces an array of concrete tools for work in the field. Salient features included:

. Characterization, comparison and optimal design of information structures in static and dynamic teams. Operational, structural and topological properties of information structures in optimal decision making, with a systematic program for generating optimal encoding and control policies. The notion of signaling, and its utilization in stabilization and optimization of decentralized control systems.

. Presentation of mathematical methods for stochastic stability of networked control systems using random-time, state-dependent drift conditions and martingale methods.

. Characterization and study of information channels leading to various forms of stochastic stability such as stationarity, ergodicity, and quadratic stability; and connections with information and quantization theories. Analysis of various classes of centralized and decentralized control systems.

. Jointly optimal design of encoding and control policies over various information channels and under general optimization criteria, including a detailed coverage of linear-quadratic-Gaussian models.

. Decentralized agreement and dynamic optimization under information constraints.

This monograph is geared toward a broad audience of academic and industrial researchers interested in control theory, information theory, optimization, economics, and applied mathematics. It could likewise serve as a supplemental graduate text. The reader is expected to have some familiarity with linear systems, stochastic processes, and Markov chains, but the necessary background can also be acquired in part through the four appendices included at the end.

. Characterization, comparison and optimal design of information structures in static and dynamic teams. Operational, structural and topological properties of information structures in optimal decision making, with a systematic program for generating optimal encoding and control policies. The notion of signaling, and its utilization in stabilization and optimization of decentralized control systems.

. Presentation of mathematical methods for stochastic stability of networked control systems using random-time, state-dependent drift conditions and martingale methods.

. Characterization and study of information channels leading to various forms of stochastic stability such as stationarity, ergodicity, and quadratic stability; and connections with information and quantization theories. Analysis of various classes of centralized and decentralized control systems.

. Jointly optimal design of encoding and control policies over various information channels and under general optimization criteria, including a detailed coverage of linear-quadratic-Gaussian models.

. Decentralized agreement and dynamic optimization under information constraints.

This monograph is geared toward a broad audience of academic and industrial researchers interested in control theory, information theory, optimization, economics, and applied mathematics. It could likewise serve as a supplemental graduate text. The reader is expected to have some familiarity with linear systems, stochastic processes, and Markov chains, but the necessary background can also be acquired in part through the four appendices included at the end.

'Et moi, ..., si j'avait su comment en revenir, One service mathematics has rendered the je n'y serais point aile.' human race. It has put common sense back Jules Verne where it belongs, on the topmost shelf next to the dusty canister labelled 'discarded n- sense'. The series is divergent; therefore we may be able to do something with it. Eric T. Bell O. Heaviside Mathematics is a tool for thought. A highly necessary tool in a world where both feedback and non linearities abound. Similarly, all kinds of parts of mathematics serve as tools for other parts and for other sciences. Applying a simple rewriting rule to the quote on the right above one finds such statements as: 'One service topology has rendered mathematical physics ... '; 'One service logic has rendered com puter science .. .'; 'One service category theory has rendered mathematics .. .'. All arguably true. And all statements obtainable this way form part of the raison d'etre of this series."

Completely revised and greatly expanded, the new edition of this text takes readers who have been exposed to only basic courses in analysis through the modern general theory of random processes and stochastic integrals as used by systems theorists, electronic engineers and, more recently, those working in quantitative and mathematical finance. Building upon the original release of this title, this text will be of great interest to research mathematicians and graduate students working in those fields, as well as quants in the finance industry. New features of this edition include: End of chapter exercises; New chapters on basic measure theory and Backward SDEs; Reworked proofs, examples and explanatory material; Increased focus on motivating the mathematics; Extensive topical index. "Such a self-contained and complete exposition of stochastic calculus and applications fills an existing gap in the literature. The book can be recommended for first-year graduate studies. It will be useful for all who intend to work with stochastic calculus as well as with its applications."-Zentralblatt (from review of the First Edition)

Game theory involves multi-person decision making and differential dynamic game theory has been widely applied to n-person decision making problems, which are stimulated by a vast number of applications. This book addresses the gap to discuss general stochastic n-person noncooperative and cooperative game theory with wide applications to control systems, signal processing systems, communication systems, managements, financial systems, and biological systems. H8 game strategy, n-person cooperative and noncooperative game strategy are discussed for linear and nonlinear stochastic systems along with some computational algorithms developed to efficiently solve these game strategies.

Is the first volume devoted entirely to stochastic inverse problems. Includes survey articles which makes it self-contained. Aimed at a diverse audience, including applied mathematicians, engineers, economists, and professionals from academia. Includes the most recent developments on the subject, which so far have only been available in the research literature.

At the dawn of the 21st century, computational stochastic dynamics is an emerging research frontier. This book focuses on advanced computational methods and software tools which can highly assist in tackling complex problems in stochastic dynamic/seismic analysis and design of structures. The book is primarily intended for researchers and post-graduate students in the fields of computational mechanics and stochastic structural dynamics. Nevertheless, practice engineers as well could benefit from it as most code provisions tend to incorporate probabilistic concepts in the analysis and design of structures. The book addresses mathematical and numerical issues in stochastic structural dynamics and connects them to real-world applications. It consists of 16 chapters dealing with recent advances in a wide range of related topics (dynamic response variability and reliability of stochastic systems, risk assessment, stochastic simulation of earthquake ground motions, efficient solvers for the analysis of stochastic systems, dynamic stability, stochastic modelling of heterogeneous materials). Numerical examples demonstrating the significance of the proposed methods are presented in each chapter.

Most branches of science involving random fluctuations can be approached by Stochastic Calculus. These include, but are not limited to, signal processing, noise filtering, stochastic control, optimal stopping, electrical circuits, financial markets, molecular chemistry, population dynamics, etc. All these applications assume a strong mathematical background, which in general takes a long time to develop. Stochastic Calculus is not an easy to grasp theory, and in general, requires acquaintance with the probability, analysis and measure theory.The goal of this book is to present Stochastic Calculus at an introductory level and not at its maximum mathematical detail. The author's goal was to capture as much as possible the spirit of elementary deterministic Calculus, at which students have been already exposed. This assumes a presentation that mimics similar properties of deterministic Calculus, which facilitates understanding of more complicated topics of Stochastic Calculus.The second edition contains several new features that improved the first edition both qualitatively and quantitatively. First, two more chapters have been added, Chapter 12 and Chapter 13, dealing with applications of stochastic processes in Electrochemistry and global optimization methods.This edition contains also a final chapter material containing fully solved review problems and provides solutions, or at least valuable hints, to all proposed problems. The present edition contains a total of about 250 exercises.This edition has also improved presentation from the first edition in several chapters, including new material.

This book will cover heuristic optimization techniques and applications in engineering problems. The book will be divided into three sections that will provide coverage of the techniques, which can be employed by engineers, researchers, and manufacturing industries, to improve their productivity with the sole motive of socio-economic development. This will be the first book in the category of heuristic techniques with relevance to engineering problems and achieving optimal solutions. Features Explains the concept of optimization and the relevance of using heuristic techniques for optimal solutions in engineering problems Illustrates the various heuristics techniques Describes evolutionary heuristic techniques like genetic algorithm and particle swarm optimization Contains natural based techniques like ant colony optimization, bee algorithm, firefly optimization, and cuckoo search Offers sample problems and their optimization, using various heuristic techniques

This book presents state-of-the-art solution methods and applications of stochastic optimal control. It is a collection of extended papers discussed at the traditional Liverpool workshop on controlled stochastic processes with participants from both the east and the west. New problems are formulated, and progresses of ongoing research are reported. Topics covered in this book include theoretical results and numerical methods for Markov and semi-Markov decision processes, optimal stopping of Markov processes, stochastic games, problems with partial information, optimal filtering, robust control, Q-learning, and self-organizing algorithms. Real-life case studies and applications, e.g., queueing systems, forest management, control of water resources, marketing science, and healthcare, are presented. Scientific researchers and postgraduate students interested in stochastic optimal control,- as well as practitioners will find this book appealing and a valuable reference.

This book contains both expository articles and original research in the areas of function theory and operator theory. The contributions include extended versions of some of the lectures by invited speakers at the conference in honor of the memory of Serguei Shimorin at the Mittag-Leffler Institute in the summer of 2018. The book is intended for all researchers in the fields of function theory, operator theory and complex analysis in one or several variables. The expository articles reflecting the current status of several well-established and very dynamical areas of research will be accessible and useful to advanced graduate students and young researchers in pure and applied mathematics, and also to engineers and physicists using complex analysis methods in their investigations.

The general theory of stochastic processes and the more specialized theory of Markov processes evolved enormously in the second half of the last century. In parallel, the theory of controlled Markov chains (or Markov decision processes) was being pioneered by control engineers and operations researchers. Researchers in Markov processes and controlled Markov chains have been, for a long time, aware of the synergies between these two subject areas. However, this may be the first volume dedicated to highlighting these synergies and, almost certainly, it is the first volume that emphasizes the contributions of the vibrant and growing Chinese school of probability. The chapters that appear in this book reflect both the maturity and the vitality of modern day Markov processes and controlled Markov chains. They also will provide an opportunity to trace the connections that have emerged between the work done by members of the Chinese school of probability and the work done by the European, US, Central and South American and Asian scholars.

This is a development of the book entitled Multidimensional Second Order Stochastic Processes. It provides a research expository treatment of infinite-dimensional stationary and nonstationary stochastic processes or time series, based on Hilbert and Banach space-valued second order random variables. Stochastic measures and scalar or operator bimeasures are fully discussed to develop integral representations of various classes of nonstationary processes such as harmonizable, V-bounded, Cramer and Karhunen classes as well as the stationary class. A new type of the Radon-Nikodym derivative of a Banach space-valued measure is introduced, together with Schauder basic measures, to study uniformly bounded linearly stationary processes.Emphasis is on the use of functional analysis and harmonic analysis as well as probability theory. Applications are made from the probabilistic and statistical points of view to prediction problems, Kalman filter, sampling theorems and strong laws of large numbers. Generalizations are made to consider Banach space-valued stochastic processes to include processes of pth order for p 1. Readers may find that the covariance kernel is always emphasized and reveals another aspect of stochastic processes.This book is intended not only for probabilists and statisticians, but also for functional analysts and communication engineers.

Probability has been an important part of mathematics for more than three centuries. Moreover, its importance has grown in recent decades, since the computing power now widely available has allowed probabilistic and stochastic techniques to attack problems such as speech and image processing, geophysical exploration, radar, sonar, etc. -- all of which are covered here. The book contains three exceptionally clear expositions on wavelets, frames and their applications. A further extremely active current research area, well covered here, is the relation between probability and partial differential equations, including probabilistic representations of solutions to elliptic and parabolic PDEs. New approaches, such as the PDE method for large deviation problems, and stochastic optimal control and filtering theory, are beginning to yield their secrets. Another topic dealt with is the application of probabilistic techniques to mathematical analysis. Finally, there are clear explanations of normal numbers and dynamic systems, and the influence of probability on our daily lives.

Markov Random Flights is the first systematic presentation of the theory of Markov random flights in the Euclidean spaces of different dimensions. Markov random flights is a stochastic dynamic system subject to the control of an external Poisson process and represented by the stochastic motion of a particle that moves at constant finite speed and changes its direction at random Poisson time instants. The initial (and each new) direction is taken at random according to some probability distribution on the unit sphere. Such stochastic motion is the basic model for describing many real finite-velocity transport phenomena arising in statistical physics, chemistry, biology, environmental science and financial markets. Markov random flights acts as an effective tool for modelling the slow and super-slow diffusion processes arising in various fields of science and technology. Features: Provides the first systematic presentation of the theory of Markov random flights in the Euclidean spaces of different dimensions. Suitable for graduate students and specialists and professionals in applied areas. Introduces a new unified approach based on the powerful methods of mathematical analysis, such as integral transforms, generalized, hypergeometric and special functions. Author Alexander D. Kolesnik is a professor, Head of Laboratory (2015-2019) and principal researcher (since 2020) at the Institute of Mathematics and Computer Science, Kishinev (Chisinau), Moldova. He graduated from Moldova State University in 1980 and earned his PhD from the Institute of Mathematics of the National Academy of Sciences of Ukraine, Kiev in 1991. He also earned a PhD Habilitation in mathematics and physics with specialization in stochastic processes, probability and statistics conferred by the Specialized Council at the Institute of Mathematics of the National Academy of Sciences of Ukraine and confirmed by the Supreme Attestation Commission of Ukraine in 2010. His research interests include: probability and statistics, stochastic processes, random evolutions, stochastic dynamic systems, random flights, diffusion processes, transport processes, random walks, stochastic processes in random environments, partial differential equations in stochastic models, statistical physics and wave processes. Dr. Kolesnik has published more than 70 scientific publications, mostly in high-standard international journals and a monograph. He has also acted as external referee for many outstanding international journals in mathematics and physics, being awarded by the "Certificate of Outstanding Contribution in Reviewing" from the journal "Stochastic Processes and their Applications." He was the visiting professor and scholarship holder at universities in Italy and Germany and member of the Board of Global Advisors of the International Federation of Nonlinear Analysts (IFNA), United States of America.

Stochastic hydrogeology, which emerged as a research area in the late 1970s, involves the study of subsurface, geological variability on flow and transport processes and the interpretation of observations using existing theories. Lacking, however, has been a rational framework for modeling the impact of the processes that take place in heterogeneous media and for incorporating it in predictions and decision-making. This book provides this important framework. It covers the fundamental and practical aspects of stochastic hydrogeology, coupling theoretical aspects with examples, case studies, and guidelines for applications.

This contributed volume provides an extensive account of research and expository papers in a broad domain of mathematical analysis and its various applications to a multitude of fields. Presenting the state-of-the-art knowledge in a wide range of topics, the book will be useful to graduate students and researchers in theoretical and applicable interdisciplinary research. The focus is on several subjects including: optimal control problems, optimal maintenance of communication networks, optimal emergency evacuation with uncertainty, cooperative and noncooperative partial differential systems, variational inequalities and general equilibrium models, anisotropic elasticity and harmonic functions, nonlinear stochastic differential equations, operator equations, max-product operators of Kantorovich type, perturbations of operators, integral operators, dynamical systems involving maximal monotone operators, the three-body problem, deceptive systems, hyperbolic equations, strongly generalized preinvex functions, Dirichlet characters, probability distribution functions, applied statistics, integral inequalities, generalized convexity, global hyperbolicity of spacetimes, Douglas-Rachford methods, fixed point problems, the general Rodrigues problem, Banach algebras, affine group, Gibbs semigroup, relator spaces, sparse data representation, Meier-Keeler sequential contractions, hybrid contractions, and polynomial equations. Some of the works published within this volume provide as well guidelines for further research and proposals for new directions and open problems.

Metaheuristic optimization is a higher-level procedure or heuristic designed to find, generate, or select a heuristic (partial search algorithm) that may provide a sufficiently good solution to an optimization problem, especially with incomplete or imperfect information or limited computation capacity. This is usually applied when two or more objectives are to be optimized simultaneously. This book is presented with two major objectives. Firstly, it features chapters by eminent researchers in the field providing the readers about the current status of the subject. Secondly, algorithm-based optimization or advanced optimization techniques, which are applied to mostly non-engineering problems, are applied to engineering problems. This book will also serve as an aid to both research and industry. Usage of these methodologies would enable the improvement in engineering and manufacturing technology and support an organization in this era of low product life cycle. Features: Covers the application of recent and new algorithms Focuses on the development aspects such as including surrogate modeling, parallelization, game theory, and hybridization Presents the advances of engineering applications for both single-objective and multi-objective optimization problems Offers recent developments from a variety of engineering fields Discusses Optimization using Evolutionary Algorithms and Metaheuristics applications in engineering

Serving as the foundation for a one-semester course in stochastic processes for students familiar with elementary probability theory and calculus, "Introduction to Stochastic Modeling, 4e, " bridges the gap between basic probability and an intermediate level course in stochastic processes. The objectives of the text are to introduce students to the standard concepts and methods of stochastic modeling, to illustrate the rich diversity of applications of stochastic processes in the applied sciences, and to provide exercises in the application of simple stochastic analysis to realistic problems.

New to this edition: Realistic applications from a variety of disciplines integrated throughout the text, including more biological applicationsPlentiful, completely updated problemsCompletely updated and reorganized end-of-chapter exercise sets, 250 exercises with answersNew chapters of stochastic differential equations and Brownian motion and related processesAdditional sections on Martingale and Poisson process

Realistic applications from a variety of disciplines integrated throughout the text.

Extensive end of chapter exercises sets, 250 with answers

Chapter 1-9 of the new edition are identical to the previous edition

New Chapter 10 - Random Evolutions

New Chapter 11- Characteristic functions and Their Applications "

Stochastic Modeling of Scientific Data combines stochastic modeling and statistical inference in a variety of standard and less common models, such as point processes, Markov random fields and hidden Markov models in a clear, thoughtful and succinct manner. The distinguishing feature of this work is that, in addition to probability theory, it contains statistical aspects of model fitting and a variety of data sets that are either analyzed in the text or used as exercises. Markov chain Monte Carlo methods are introduced for evaluating likelihoods in complicated models and the forward backward algorithm for analyzing hidden Markov models is presented. The strength of this text lies in the use of informal language that makes the topic more accessible to non-mathematicians. The combinations of hard science topics with stochastic processes and their statistical inference puts it in a new category of probability textbooks. The numerous examples and exercises are drawn from astronomy, geology, genetics, hydrology, neurophysiology and physics.