On Discounted Dynamic Programming with Unbounded Returns

Matkowski, Janusz and Nowak, Andrzej S. (2008): On Discounted Dynamic Programming with Unbounded Returns.

Abstract

In this paper, we apply the idea of $k$-local contraction of \cite{zec, zet} to study discounted stochastic dynamic programming models with unbounded returns. Our main results concern the existence of a unique solution to the Bellman equation and are applied to the theory of stochastic optimal growth. Also a discussion of some subtle issues concerning k-local and global contractions is included.

• Item Type: MPRA Paper
• Language: English
• Keywords: Stochastic dynamic programming, Bellman functional equation, contraction mapping, stochastic optimal growth
• Subjects: D - Microeconomics > D9 - Intertemporal Choice > D90 - General; D - Microeconomics > D9 - Intertemporal Choice > D91 - Intertemporal Household Choice ; Life Cycle Models and Saving; C - Mathematical and Quantitative Methods > C6 - Mathematical Methods ; Programming Models ; Mathematical and Simulation Modeling > C61 - Optimization Techniques ; Programming Models ; Dynamic Analysis

References

Berge, C.(1963) Topological Spaces. MacMillan, New York.

Bertsekas, D.P., Shreve, S.E.(1978) Stochastic Optimal Control: the Discrete Time Case. Academic Press, New York

Blackwell, D. (1965) Discounted dynamic programming. Annals of Mathematical Statistics 36: 226-235.

Boyd III, J.H. (1990) Recursive utility and the Ramsey problem. Journal of Economic Theory 50: 326-345.

Boyd III, J.H., Becker, R.A. (1997) Capital Theory, Equilibrium Analysis and Recursive Utility. Blackwell Publishers, New York.

Brown, L.D., Purves, R.(1973) Measurable selections of extrema. Annals of Statistics 1: 902-912.

Brock, W.A., Mirman, L.J.(1972) Optimal economic growth and uncertainty: the discounted case. Journal of Economic Theory 4: 479-513.

Dana, R.A., Le Van, C., Mitra, T., Nishimura, K., (Eds) (2006) Handbook of Optimal Growth 1. Springer, Berlin.

Dutta, P.K., Mitra, T. (1989). On continuity of the utility function in intertemporal allocation models: an example. International Economic Review 30: 527-536.

Filipe Martins-da-Rocha, V., Vailakis, Y. (2008) Existence and uniqueness of fixed-point for local contractions. Personal communication, submitted for publication.

Hernandez-Lerma, O., Lasserre, J.B. (1999) Further Topics on Discrete-Time Markov Control Processes. Springer-Verlag, New York.

Himmelberg, C.J., (1975) Measurable relations. Fundamenta Mathematicae 87: 53-72.

Kuratowski, K., Ryll-Nardzewski, C. (1965) A general theorem on selectors. Bulletin de l'Academie Polonaise des Sciences (Ser. Mathematique) 13: 397-403.

Le Van, C., Morhaim, L. (2002) Optimal growth models with bounded or unbounded returns: a unifying approach. Journal of Economic Theory 105: 158-187.

Le Van, C., Vailakis, Y. (2005) Recursive utility and optimal growth with bounded or unbounded returns. Journal of Economic Theory 123: 187-20

Neveu, J. (1965) Mathematical Foundations of the Calculus of Probability. Holden-Day, San Francisco.

Nowak, A.S. (1984) On zero-sum stochastic games with general state space I. Probability and Mathematical Statistics 4: 13-32.

Nowak, A.S. (1985) Universally measurable strategies in zero-sum stochastic games. Annals of Probability 13: 269-287.

Puterman, M. (2005) Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley-Interscience, New York.

Rinc\'{o}n-Zapatero, J. P., Rodrigues-Palmero, C. (2003) Existence and uniqueness of solutions to the Bellman equation in the unbounded case. Econometrica 71: 1519-1555.

Rinc\'{o}n-Zapatero, J. P., Rodrigues-Palmero, C. (2007) Recursive utility with unbounded aggregators. Economic Theory 33: 381-391.

Schal, M. (1975) Conditions for optimality in dynamic programming and for the limit of n-stage optimal policies to be optimal. Zeischrift fur Wahrscheinlichkeitstheorie und verwandte Gebiete 32: 179-196.

Stokey, N.L., Lucas, R.E. with Prescott, E. (1989) Recursive Methods in Economic Dynamics. Harvard University Press, Cambridge, MA.