|
Showing 1 - 2 of
2 matches in All Departments
A Markov Decision Process (MDP) is a natural framework for
formulating sequential decision-making problems under uncertainty.
In recent years, researchers have greatly advanced algorithms for
learning and acting in MDPs. This book reviews such algorithms,
beginning with well-known dynamic programming methods for solving
MDPs such as policy iteration and value iteration, then describes
approximate dynamic programming methods such as trajectory based
value iteration, and finally moves to reinforcement learning
methods such as Q-Learning, SARSA, and least-squares policy
iteration. It describes algorithms in a unified framework, giving
pseudocode together with memory and iteration complexity analysis
for each. Empirical evaluations of these techniques, with four
representations across four domains, provide insight into how these
algorithms perform with various feature sets in terms of running
time and performance. This tutorial provides practical guidance for
researchers seeking to extend DP and RL techniques to larger
domains through linear value function approximation. The practical
algorithms and empirical successes outlined also form a guide for
practitioners trying to weigh computational costs, accuracy
requirements, and representational concerns. Decision making in
large domains will always be challenging, but with the tools
presented here this challenge is not insurmountable.
An introduction to decision making under uncertainty from a
computational perspective, covering both theory and applications
ranging from speech recognition to airborne collision avoidance.
Many important problems involve decision making under
uncertainty-that is, choosing actions based on often imperfect
observations, with unknown outcomes. Designers of automated
decision support systems must take into account the various sources
of uncertainty while balancing the multiple objectives of the
system. This book provides an introduction to the challenges of
decision making under uncertainty from a computational perspective.
It presents both the theory behind decision making models and
algorithms and a collection of example applications that range from
speech recognition to aircraft collision avoidance. Focusing on two
methods for designing decision agents, planning and reinforcement
learning, the book covers probabilistic models, introducing
Bayesian networks as a graphical model that captures probabilistic
relationships between variables; utility theory as a framework for
understanding optimal decision making under uncertainty; Markov
decision processes as a method for modeling sequential problems;
model uncertainty; state uncertainty; and cooperative decision
making involving multiple interacting agents. A series of
applications shows how the theoretical concepts can be applied to
systems for attribute-based person search, speech applications,
collision avoidance, and unmanned aircraft persistent surveillance.
Decision Making Under Uncertainty unifies research from different
communities using consistent notation, and is accessible to
students and researchers across engineering disciplines who have
some prior exposure to probability theory and calculus. It can be
used as a text for advanced undergraduate and graduate students in
fields including computer science, aerospace and electrical
engineering, and management science. It will also be a valuable
professional reference for researchers in a variety of disciplines.
|
|