This is the revised and augmented edition of a now classic book which is an introduction to sub-Markovian kernels on general measurable spaces and their associated homogeneous Markov chains. The first part, an expository text on the foundations of the subject, is intended for post-graduate students. A study of potential theory, the basic classification of chains according to their asymptotic behaviour and the celebrated Chacon-Ornstein theorem are examined in detail.
The second part of the book is at a more advanced level and includes a treatment of random walks on general locally compact abelian groups. Further chapters develop renewal theory, an introduction to Martin boundary and the study of chains recurrent in the Harris sense. Finally, the last chapter deals with the construction of chains starting from a kernel satisfying some kind of maximum principle.

Current research results in stochastic and deterministic global optimization including single and multiple objectives are explored and presented in this book by leading specialists from various fields. Contributions include applications to multidimensional data visualization, regression, survey calibration, inventory management, timetabling, chemical engineering, energy systems, and competitive facility location. Graduate students, researchers, and scientists in computer science, numerical analysis, optimization, and applied mathematics will be fascinated by the theoretical, computational, and application-oriented aspects of stochastic and deterministic global optimization explored in this book. This volume is dedicated to the 70th birthday of Antanas Zilinskas who is a leading world expert in global optimization. Professor Zilinskas's research has concentrated on studying models for the objective function, the development and implementation of efficient algorithms for global optimization with single and multiple objectives, and application of algorithms for solving real-world practical problems.

In some cases, certain coherent structures can exist in stochastic dynamic systems almost in every particular realization of random parameters describing these systems. Dynamic localization in one-dimensional dynamic systems, vortexgenesis (vortex production) in hydrodynamic flows, and phenomenon of clustering of various fields in random media (i.e., appearance of small regions with enhanced content of the field against the nearly vanishing background of this field in the remaining portion of space) are examples of such structure formation. The general methodology presented in Volume 1 is used in Volume 2 Coherent Phenomena in Stochastic Dynamic Systems to expound the theory of these phenomena in some specific fields of stochastic science, among which are hydrodynamics, magnetohydrodynamics, acoustics, optics, and radiophysics. The material of this volume includes particle and field clustering in the cases of scalar (density field) and vector (magnetic field) passive tracers in a random velocity field, dynamic localization of plane waves in layered random media, as well as monochromatic wave propagation and caustic structure formation in random media in terms of the scalar parabolic equation.

This book is a thorough and self-contained treatise of martingales as a tool in stochastic analysis, stochastic integrals and stochastic differential equations. The book is clearly written and details of proofs are worked out.

This book presents in-depth coverage of laboratory experiments, theories, modeling techniques, and practices for the analysis and design of rock slopes in complex geological settings. It addresses new concepts in connection with the kinematical element method, discontinuity kinematical element method, integrated karst cave stochastic model-limit equilibrium method, improved strength reduction method, and fracture mechanics method, taking into account the relevant geological features. The book is chiefly intended as a reference guide for geotechnical engineering and engineering geology professionals, and as a textbook for related graduate courses.

This book provides a comprehensive up-to-date presentation of some of the classical areas of reliability, based on a more advanced probabilistic framework using the modern theory of stochastic processes. This framework allows analysts to formulate general failure models, establish formulae for computing various performance measures, as well as determine how to identify optimal replacement policies in complex situations.

In this second edition of the book, two major topics have been added to the original version: copula models which are used to study the effect of structural dependencies on the system reliability; and maintenance optimization which highlights delay time models under safety constraints.

Terje Aven is Professor of Reliability and Risk Analysis at University of Stavanger, Norway. Uwe Jensen is working as a Professor at the Institute of Applied Mathematics and Statistics of the University of Hohenheim in Stuttgart, Germany.

Review of first edition:

"This is an excellent book on mathematical, statistical and stochastic models in reliability. The authors have done an excellent job of unifying some of the stochastic models in reliability. The book is a good reference book but may not be suitable as a textbook for students in professional fields such as engineering. This book may be used for graduate level seminar courses for students who have had at least the first course in stochastic processes and some knowledge of reliability mathematics. It should be a good reference book for researchers in reliability mathematics."

--Mathematical Reviews (2000)

"

This textbook for advanced undergraduate and postgraduate students of Evolutionary Game Theory covers recent developments in the field, with an emphasis on economic contexts and applications. It begins with the basic ideas as they originated within the field of theoretical biology and then proceeds to the formulation of a theoretical framework that is suitable for the study of social and economic phenomena from an evolutionary perspective. Core topics include the Evolutionary Stable Strategy (EES) and Replicator Dynamics (RD), deterministic dynamic models, and stochastic perturbations. A set of short appendices presents some of the technical material referred to in the main text. Evolutionary theory is widely viewed as one of the most promising appraoches to understanding bounded rationality, learning, and change in complex social environments. New avenues of research are suggested by Vega-Redondo, and plentiful exmples illustrate the theory's potential applications. The recent boom experienced by this dscipline makes the book's systematic presentation of its essential contributions vital reading for newcomer to the field.

This volume presents the latest advances and trends in stochastic models and related statistical procedures. Selected peer-reviewed contributions focus on statistical inference, quality control, change-point analysis and detection, empirical processes, time series analysis, survival analysis and reliability, statistics for stochastic processes, big data in technology and the sciences, statistical genetics, experiment design, and stochastic models in engineering. Stochastic models and related statistical procedures play an important part in furthering our understanding of the challenging problems currently arising in areas of application such as the natural sciences, information technology, engineering, image analysis, genetics, energy and finance, to name but a few. This collection arises from the 12th Workshop on Stochastic Models, Statistics and Their Applications, Wroclaw, Poland.

Periodically Correlated Solutions to a Class of Stochastic Difference Equations.- On Nonlinear SDE'S whose Densities Evolve in a Finite-Dimensional Family.- Composition of Skeletons and Support Theorems.- Invariant Measure for a Wave Equation on a Riemannian Manifold.- Ergodic Distributed Control for Parameter Dependent Stochastic Semilinear Systems.- Dirichlet Forms, Caccioppoli Sets and the Skorohod Equation Masatoshi Fukushima.- Rate of Convergence of Moments of Spall's SPSA Method.- General Setting for Stochastic Processes Associated with Quantum Fields.- On a Class of Semilinear Stochastic Partial Differential Equations.- Parallel Numerical Solution of a Class of Volterra Integro-Differential Equations.- On the Laws of the Oseledets Spaces of Linear Stochastic Differential Equations.- On Stationarity of Additive Bilinear State-space Representation of Time Series.- On Convergence of Approximations of Ito-Volterra Equations.- Non-isotropic Ornstein-Uhlenbeck Process and White Noise Analysis.- Stochastic Processes with Independent Increments on a Lie Group and their Selfsimilar Properties.- Optimal Damping of Forced Oscillations Discrete-time Systems by Output Feedback.- Forecast of Levy's Brownian Motion as the Observation Domain Undergoes Deformation.- A Maximal Inequality for the Skorohod Integral.- On the Kinematics of Stochastic Mechanics.- Stochastic Equations in Formal Mappings.- On Fisher's Information Matrix of an ARMA Process.- Statistical Analysis of Nonlinear and NonGaussian Time Series.- Bilinear Stochastic Systems with Long Range Dependence in Continuous Time.- On Support Theorems for Stochastic Nonlinear Partial Differential Equations.- Excitation and Performance in Continuous-time Stochastic Adaptive LQ-control.- Invariant Measures for Diffusion Processes in Conuclear Spaces.- Degree Theory on Wiener Space and an Application to a Class of SPDEs.- On the Interacting Measure-Valued Branching Processes.

This book presents classical Markov Decision Processes (MDP) for real-life applications and optimization. MDP allows users to develop and formally support approximate and simple decision rules, and this book showcases state-of-the-art applications in which MDP was key to the solution approach. The book is divided into six parts. Part 1 is devoted to the state-of-the-art theoretical foundation of MDP, including approximate methods such as policy improvement, successive approximation and infinite state spaces as well as an instructive chapter on Approximate Dynamic Programming. It then continues with five parts of specific and non-exhaustive application areas. Part 2 covers MDP healthcare applications, which includes different screening procedures, appointment scheduling, ambulance scheduling and blood management. Part 3 explores MDP modeling within transportation. This ranges from public to private transportation, from airports and traffic lights to car parking or charging your electric car . Part 4 contains three chapters that illustrates the structure of approximate policies for production or manufacturing structures. In Part 5, communications is highlighted as an important application area for MDP. It includes Gittins indices, down-to-earth call centers and wireless sensor networks. Finally Part 6 is dedicated to financial modeling, offering an instructive review to account for financial portfolios and derivatives under proportional transactional costs. The MDP applications in this book illustrate a variety of both standard and non-standard aspects of MDP modeling and its practical use. This book should appeal to readers for practitioning, academic research and educational purposes, with a background in, among others, operations research, mathematics, computer science, and industrial engineering.

This textbook offers a readily comprehensible introduction to classical Newtonian gravitation, which is fundamental for an understanding of classical mechanics and is particularly relevant to Astrophysics. The opening chapter recalls essential elements of vectorial calculus, especially to provide the formalism used in subsequent chapters. In chapter two Classical Newtonian gravity theory for one point mass and for a generic number N of point masses is then presented and discussed. The theory for point masses is naturally extended to the continuous case. The third chapter addresses the paradigmatic case of spherical symmetry in the mass density distribution (central force), with introduction of the useful tool of qualitative treatment of motion. Subsequent chapters discuss the general case of non-symmetric mass density distribution and develop classical potential theory, with elements of harmonic theory, which is essential to understand the potential development in series of the gravitational potential, the subject of the fourth chapter. Finally, in the last chapter the specific case of motion of a satellite around the earth is considered. Examples and exercises are presented throughout the book to clarify aspects of the theory. The book is aimed at those who wish to progress further beyond an initial bachelor degree, onward to a master degree, and a PhD. It is also a valuable resource for postgraduates and active researchers in the field.

Brownian dynamics serve as mathematical models for the diffusive motion of microscopic particles of various shapes in gaseous, liquid, or solid environments. The renewed interest in Brownian dynamics is due primarily to their key role in molecular and cellular biophysics: diffusion of ions and molecules is the driver of all life. Brownian dynamics simulations are the numerical realizations of stochastic differential equations that model the functions of biological micro devices such as protein ionic channels of biological membranes, cardiac myocytes, neuronal synapses, and many more. Stochastic differential equations are ubiquitous models in computational physics, chemistry, biophysics, computer science, communications theory, mathematical finance theory, and many other disciplines. Brownian dynamics simulations of the random motion of particles, be it molecules or stock prices, give rise to mathematical problems that neither the kinetic theory of Maxwell and Boltzmann, nor Einstein's and Langevin's theories of Brownian motion could predict.This book takes the readers on a journey that starts with the rigorous definition of mathematical Brownian motion, and ends with the explicit solution of a series of complex problems that have immediate applications. It is aimed at applied mathematicians, physicists, theoretical chemists, and physiologists who are interested in modeling, analysis, and simulation of micro devices of microbiology. The book contains exercises and worked out examples throughout.

The book describes a systematic stochastic modeling approach for assessing thermal-fatigue crack-growth in mixing tees, based on the power spectral density of temperature fluctuation at the inner pipe surface. It shows the development of a frequency-temperature response function in the framework of single-input, single-output (SISO) methodology from random noise/signal theory under sinusoidal input. The frequency response of stress intensity factor (SIF) is obtained by a polynomial fitting procedure of thermal stress profiles at various instants of time. The method, which takes into account the variability of material properties, and has been implemented in a real-world application, estimates the probabilities of failure by considering a limit state function and Monte Carlo analysis, which are based on the proposed stochastic model. Written in a comprehensive and accessible style, this book presents a new and effective method for assessing thermal fatigue crack, and it is intended as a concise and practice-oriented guide for all undergraduate students, young scientists and researchers dealing with probabilistic assessment of structural integrity.

Networked control systems are increasingly ubiquitous today, with applications ranging from vehicle communication and adaptive power grids to space exploration and economics. The optimal design of such systems presents major challenges, requiring tools from various disciplines within applied mathematics such as decentralized control, stochastic control, information theory, and quantization.

A thorough, self-contained book, "Stochastic Networked Control Systems: Stabilization and Optimization under Information Constraints" aims to connect these diverse disciplines with precision and rigor, while conveying design guidelines to controller architects. Unique in the literature, it lays a comprehensive theoretical foundation for the study of networked control systems, and introduces an array of concrete tools for work in the field. Salient features included:

. Characterization, comparison and optimal design of information structures in static and dynamic teams. Operational, structural and topological properties of information structures in optimal decision making, with a systematic program for generating optimal encoding and control policies. The notion of signaling, and its utilization in stabilization and optimization of decentralized control systems.

. Presentation of mathematical methods for stochastic stability of networked control systems using random-time, state-dependent drift conditions and martingale methods.

. Characterization and study of information channels leading to various forms of stochastic stability such as stationarity, ergodicity, and quadratic stability; and connections with information and quantization theories. Analysis of various classes of centralized and decentralized control systems.

. Jointly optimal design of encoding and control policies over various information channels and under general optimization criteria, including a detailed coverage of linear-quadratic-Gaussian models.

. Decentralized agreement and dynamic optimization under information constraints.

This monograph is geared toward a broad audience of academic and industrial researchers interested in control theory, information theory, optimization, economics, and applied mathematics. It could likewise serve as a supplemental graduate text. The reader is expected to have some familiarity with linear systems, stochastic processes, and Markov chains, but the necessary background can also be acquired in part through the four appendices included at the end.

. Characterization, comparison and optimal design of information structures in static and dynamic teams. Operational, structural and topological properties of information structures in optimal decision making, with a systematic program for generating optimal encoding and control policies. The notion of signaling, and its utilization in stabilization and optimization of decentralized control systems.

. Presentation of mathematical methods for stochastic stability of networked control systems using random-time, state-dependent drift conditions and martingale methods.

. Characterization and study of information channels leading to various forms of stochastic stability such as stationarity, ergodicity, and quadratic stability; and connections with information and quantization theories. Analysis of various classes of centralized and decentralized control systems.

. Jointly optimal design of encoding and control policies over various information channels and under general optimization criteria, including a detailed coverage of linear-quadratic-Gaussian models.

. Decentralized agreement and dynamic optimization under information constraints.

This monograph is geared toward a broad audience of academic and industrial researchers interested in control theory, information theory, optimization, economics, and applied mathematics. It could likewise serve as a supplemental graduate text. The reader is expected to have some familiarity with linear systems, stochastic processes, and Markov chains, but the necessary background can also be acquired in part through the four appendices included at the end.

The purpose of this textbook is to bring together, in a self-contained introductory form, the scattered material in the field of stochastic processes and statistical physics. It offers the opportunity of being acquainted with stochastic, kinetic and nonequilibrium processes. Although the research techniques in these areas have become standard procedures, they are not usually taught in the normal courses on statistical physics. For students of physics in their last year and graduate students who wish to gain an invaluable introduction on the above subjects, this book is a necessary tool.

The systematic study of existence, uniqueness, and properties of solutions to stochastic differential equations in infinite dimensions arising from practical problems characterizes this volume that is intended for graduate students and for pure and applied mathematicians, physicists, engineers, professionals working with mathematical models of finance. Major methods include compactness, coercivity, monotonicity, in a variety of set-ups. The authors emphasize the fundamental work of Gikhman and Skorokhod on the existence and uniqueness of solutions to stochastic differential equations and present its extension to infinite dimension. They also generalize the work of Khasminskii on stability and stationary distributions of solutions. New results, applications, and examples of stochastic partial differential equations are included. This clear and detailed presentation gives the basics of the infinite dimensional version of the classic books of Gikhman and Skorokhod and of Khasminskii in one concise volume that covers the main topics in infinite dimensional stochastic PDE's. By appropriate selection of material, the volume can be adapted for a 1- or 2-semester course, and can prepare the reader for research in this rapidly expanding area.

The analysis of PDEs is a prominent discipline in mathematics research, both in terms of its theoretical aspects and its relevance in applications. In recent years, the geometric properties of linear and nonlinear second order PDEs of elliptic and parabolic type have been extensively studied by many outstanding researchers. This book collects contributions from a selected group of leading experts who took part in the INdAM meeting "Geometric methods in PDEs", on the occasion of the 70th birthday of Ermanno Lanconelli. They describe a number of new achievements and/or the state of the art in their discipline of research, providing readers an overview of recent progress and future research trends in PDEs. In particular, the volume collects significant results for sub-elliptic equations, potential theory and diffusion equations, with an emphasis on comparing different methodologies and on their implications for theory and applications.

This book presents state-of-the-art solution methods and applications of stochastic optimal control. It is a collection of extended papers discussed at the traditional Liverpool workshop on controlled stochastic processes with participants from both the east and the west. New problems are formulated, and progresses of ongoing research are reported. Topics covered in this book include theoretical results and numerical methods for Markov and semi-Markov decision processes, optimal stopping of Markov processes, stochastic games, problems with partial information, optimal filtering, robust control, Q-learning, and self-organizing algorithms. Real-life case studies and applications, e.g., queueing systems, forest management, control of water resources, marketing science, and healthcare, are presented. Scientific researchers and postgraduate students interested in stochastic optimal control,- as well as practitioners will find this book appealing and a valuable reference.

This volume presents a collection of contributions dedicated to applied problems in the financial and energy sectors that have been formulated and solved in a stochastic optimization framework. The invited authors represent a group of scientists and practitioners, who cooperated in recent years to facilitate the growing penetration of stochastic programming techniques in real-world applications, inducing a significant advance over a large spectrum of complex decision problems.

After the recent widespread liberalization of the energy sector in Europe and the unprecedented growth of energy prices in international commodity markets, we have witnessed a significant convergence of strategic decision problems in the energy and financial sectors. This has often resulted in common open issues and has induced a remarkable effort by the industrial and scientific communities to facilitate the adoption of advanced analytical and decision tools. The main concerns of the financial community over the last decade have suddenly penetrated the energy sector inducing a remarkable scientific and practical effort to address previously unforeseeable management problems. This proposal aims to include in a unified framework for the first time an extensive set of contributions related to real-world applied problems in finance and energy, leading to a common methodological approach and in many cases having similar underlying economic and financial implications.

During the spring and the summer of 2007 the School of Stochastic Programming held in Bergamo (www.unibg.it/sps2007), and the eleventh symposium on Stochastic Programming in Vienna (http: //www.univie.ac.at/spxi), offered two venues for the presentation of the chapters included in the volume. After the two events, during the fall of 2007, all the invited contributors enthusiastically accepted the invitation to present their original work in the projected volume, from which this proposal was developed.

The volume is structured in three parts, devoted to contributions related to financial applications - Part I, with 7 chapters; energy applications - Part II, with 8 chapters; and to specific theoretical and computational issues - Part III, with 6 chapters -- recently developed in the scientific community and explicitly related to the applied problems presented.

At the dawn of the 21st century, computational stochastic dynamics is an emerging research frontier. This book focuses on advanced computational methods and software tools which can highly assist in tackling complex problems in stochastic dynamic/seismic analysis and design of structures. The book is primarily intended for researchers and post-graduate students in the fields of computational mechanics and stochastic structural dynamics. Nevertheless, practice engineers as well could benefit from it as most code provisions tend to incorporate probabilistic concepts in the analysis and design of structures. The book addresses mathematical and numerical issues in stochastic structural dynamics and connects them to real-world applications. It consists of 16 chapters dealing with recent advances in a wide range of related topics (dynamic response variability and reliability of stochastic systems, risk assessment, stochastic simulation of earthquake ground motions, efficient solvers for the analysis of stochastic systems, dynamic stability, stochastic modelling of heterogeneous materials). Numerical examples demonstrating the significance of the proposed methods are presented in each chapter.

The second volume of this work continues the approach of the first volume, providing mathematical tools for the control engineer and examining such topics as random variables and sequences, iterative logarithmic and large number laws, differential equations, stochastic measurements and optimization, discrete martingales and probability space. It includes proofs of all theorems and contains many examples with solutions.
It is written for researchers, engineers and advanced students who wish to increase their familiarity with different topics of modern and classical mathematics related to system and automatic control theories. It also has applications to game theory, machine learning and intelligent systems.
* Provides comprehensive theory of matrices, real, complex and functional analysis
* Provides practical examples of modern optimization methods that can be effectively used in variety of real-world applications
* Contains worked proofs of all theorems and propositions presented

Stochastic Optimal Control (SOC)-a mathematical theory concerned with minimizing a cost (or maximizing a payout) pertaining to a controlled dynamic processunder uncertainty-has proven incredibly helpful to understanding and predicting debt crises and evaluating proposed financial regulation and risk management."Stochastic Optimal Control and the U.S. Financial Debt Crisis"analyzes SOC in relation to the 2008 U.S. financial crisis, and offers a detailed framework depicting why such a methodology is best suited for reducing financial risk and addressing key regulatory issues. Topics discussed include the inadequacies of the current approaches underlying financial regulations, the use of SOC to explain debt crises and superiority over existing approaches to regulation, and the domestic and international applications of SOC to financial crises. Principles in this book will appeal to economists, mathematicians, and researchers interested in the U.S. financial debt crisis and optimal risk management."

This volume is a collection of chapters covering recent advances in stochastic optimal control theory and algebraic systems theory. The book will be a useful reference for researchers and graduate students in systems and control, algebraic systems theory, and applied mathematics. Requiring only knowledge of undergraduate-level control and systems theory, the work may be used as a supplementary textbook in a graduate course on optimal control or algebraic systems theory.

For most cases of interest, exact solutions to nonlinear equations describing stochastic dynamical systems are not available. This book details the relatively simple and popular linearization techniques available, covering theory as well as application. It examines models with continuous external and parametric excitations, those that cover the majority of known approaches.

A unique interdisciplinary foundation for real-world problem solving

Stochastic search and optimization techniques are used in a vast number of areas, including aerospace, medicine, transportation, and finance, to name but a few. Whether the goal is refining the design of a missile or aircraft, determining the effectiveness of a new drug, developing the most efficient timing strategies for traffic signals, or making investment decisions in order to increase profits, stochastic algorithms can help researchers and practitioners devise optimal solutions to countless real-world problems.

Introduction to Stochastic Search and Optimization: Estimation, Simulation, and Control is a graduate-level introduction to the principles, algorithms, and practical aspects of stochastic optimization, including applications drawn from engineering, statistics, and computer science. The treatment is both rigorous and broadly accessible, distinguishing this text from much of the current literature and providing students, researchers, and practitioners with a strong foundation for the often-daunting task of solving real-world problems.

The text covers a broad range of today’s most widely used stochastic algorithms, including:

• Random search
• Recursive linear estimation
• Stochastic approximation
• Simulated annealing
• Genetic and evolutionary methods
• Machine (reinforcement) learning
• Model selection
• Simulation-based optimization
• Markov chain Monte Carlo
• Optimal experimental design

The book includes over 130 examples, Web links to software and data sets, more than 250 exercises for the reader, and an extensive list of references. These features help make the text an invaluable resource for those interested in the theory or practice of stochastic search and optimization.