Foundations and Trends® in Stochastic Systems > Vol 1 > Issue 1

Monotonicity in Markov Reward and Decision Chains: Theory and Applications

  • Koole, Ger 1

[1]Koole, Ger, Department of Mathematics, VU University Amsterdam, The Netherlands, koole@few.vu.nl

Published: 7 June 2007© 2007 G. Koole

Short description

Monotonicity in Markov Reward and Decision Chains: Theory and Applications focuses on monotonicity results for dynamic systems that take values in the natural numbers or in more-dimensional lattices.

Keywords

Stochastic Optimization | Queuing Theory

Download a free copy

(Full text PDF)

Table of contents

1 Introduction
2 Typical Results
3 The Framework
4 Criteria and the Environment
5 Operators
6 Inequalities
7 Propagation Results
8 Monotonicity of Control
9 Models
10 Other State Spaces
11 Comparing Systems
12 Conclusions and Future Research Directions
References

Foundations and Trends® in Stochastic Systems

(Vol 1, Issue 1, 2006, pp 1-76)

DOI: 10.1561/0900000002

Abstract

This paper focuses on monotonicity results for dynamic systems that take values in the natural numbers or in more-dimensional lattices. The results are mostly formulated in terms of controlled queueing systems, but there are also applications to maintenance systems, revenue management, and so forth. We concentrate on results that are obtained by inductively proving properties of the dynamic programming value function. We give a framework for using this method that unifies results obtained for different models. We also give a comprehensive overview of the results that can be obtained through it, in which we discuss not only (partial) characterizations of optimal policies but also applications of monotonicity to optimization problems and the comparison of systems.

Buy e-Book
100.00 USD
(Full text PDF)
Buy Book
75.00 USD
(Physical copy)

Table of contents

1: Introduction
2: Typical Results
3: The Framework
4: Criteria and the Environment
5: Operators
6: Inequalities
7: Propagation Results
8: Monotonicity of Control
9: Models
10: Other State Spaces
11: Comparing Systems
12: Conclusion and Future Research Directions
References
Cover image for Monotonicity in Markov Reward and Decision Chains

Monotonicity in Markov Reward and Decision Chains:

Theory and Applications

88 pages

DOI: 10.1561/9781601980298

E-ISBN: 978-1-60198-029-8

ISBN: 978-1-60198-028-1

Description

Monotonicity in Markov Reward and Decision Chains: Theory and Applications focuses on monotonicity results for dynamic systems that take values in the natural numbers or in more-dimensional lattices. The results are mostly formulated in terms of controlled queueing systems, but there are also applications to maintenance systems, revenue management, and so forth. The focus is on results that are obtained by inductively proving properties of the dynamic programming value function. A framework is provided for using this method that unifies results obtained for different models. The author also provides a comprehensive overview of the results that can be obtained through it, in which he discusses not only (partial) characterizations of optimal policies but also applications of monotonicity to optimization problems and the comparison of systems. Monotonicity in Markov Reward and Decision Chains: Theory and Applications is an invaluable resource for anyone planning or conducting research in this particular area. The essentials of the topic are presented in an accessible manner and an extensive bibliography guides towards further reading.