This book presents the latest findings on stochastic dynamic programming models and on solving optimal control problems in networks. It includes the authors' new findings on determining the optimal solution of discrete optimal control problems in networks and on solving game variants of Markov decision problems in the context of computational networks. First, the book studies the finite state space of Markov processes and reviews the existing methods and algorithms for determining the main characteristics in Markov chains, before proposing new approaches based on dynamic programming and combinatorial methods. Chapter two is dedicated to infinite horizon stochastic discrete optimal control models and Markov decision problems with average and expected total discounted optimization criteria, while Chapter three develops a special game-theoretical approach to Markov decision processes and stochastic discrete optimal control problems. In closing, the book's final chapter is devoted to finite horizon stochastic control problems and Markov decision processes. The algorithms developed represent a valuable contribution to the important field of computational network theory.

Richard Bellmann developed a theory of dynamic programming which is for many reasons still in the center of great interest. The authors present a new approach in the ?eld of the optimization and multi-objective control of time-discrete systems which is closely related to the work of Richard Bellmann. They develop their own concept and their extension to the optimization and multi-objective control of time-discrete systems as well as to dynamic networks and multilayered structures are very stimulating for further research. Di?erent perspectives of discrete control and optimal dynamic ?ow problems on networks are treated and characterized. Together with the algorithmic solutions a framework of multi-objective control problems is - rived. The conclusion with a real world example underlines the necessity and - portance of their theoretic framework. As they come back to the classical Bellmann concept of dynamic programming they stress and honor his basic concept without debase their own work. Multilayereddecisionprocessesaspartofthedesignandanalysisofcomplexsystems and networks will be essential in many ways and ?elds in the future.

This book presents the latest findings on stochastic dynamic programming models and on solving optimal control problems in networks. It includes the authors' new findings on determining the optimal solution of discrete optimal control problems in networks and on solving game variants of Markov decision problems in the context of computational networks. First, the book studies the finite state space of Markov processes and reviews the existing methods and algorithms for determining the main characteristics in Markov chains, before proposing new approaches based on dynamic programming and combinatorial methods. Chapter two is dedicated to infinite horizon stochastic discrete optimal control models and Markov decision problems with average and expected total discounted optimization criteria, while Chapter three develops a special game-theoretical approach to Markov decision processes and stochastic discrete optimal control problems. In closing, the book's final chapter is devoted to finite horizon stochastic control problems and Markov decision processes. The algorithms developed represent a valuable contribution to the important field of computational network theory.

Richard Bellmann developed a theory of dynamic programming which is for many reasons still in the center of great interest. The authors present a new approach in the ?eld of the optimization and multi-objective control of time-discrete systems which is closely related to the work of Richard Bellmann. They develop their own concept and their extension to the optimization and multi-objective control of time-discrete systems as well as to dynamic networks and multilayered structures are very stimulating for further research. Di?erent perspectives of discrete control and optimal dynamic ?ow problems on networks are treated and characterized. Together with the algorithmic solutions a framework of multi-objective control problems is - rived. The conclusion with a real world example underlines the necessity and - portance of their theoretic framework. As they come back to the classical Bellmann concept of dynamic programming they stress and honor his basic concept without debase their own work. Multilayereddecisionprocessesaspartofthedesignandanalysisofcomplexsystems and networks will be essential in many ways and ?elds in the future.