On Variable Discounting in Dynamic Programming: Applications to Resource Extraction and Other Economic Models

Jaśkiewicz, Anna and Matkowski, Janusz and Nowak, Andrzej S. (2011): On Variable Discounting in Dynamic Programming: Applications to Resource Extraction and Other Economic Models. Forthcoming in: Annals of Operations Research : pp. 1-15.

Abstract

This paper generalizes the classical discounted utility model introduced by Samuelson by replacing a constant discount rate with a function. The existence of recursive utilities and their constructions are based on Matkowski's extension of the Banach Contraction Principle. The derived utilities go beyond the class of recursive utilities studied in the literature and enable a discussion on subtle issues concerning time preferences in the theory of finance, economics or psychology. Moreover, our main results are applied to the theory of optimal growth with unbounded utility functions.

• Item Type: MPRA Paper
• Language: English
• Keywords: Dynamic programming Variable discounting Bellman equation
• Subjects: D - Microeconomics > D9 - Intertemporal Choice > D90 - General; C - Mathematical and Quantitative Methods > C6 - Mathematical Methods ; Programming Models ; Mathematical and Simulation Modeling > C61 - Optimization Techniques ; Programming Models ; Dynamic Analysis

References

Alvarez F, Stokey N (1998) Dynamic programming with homogeneous functions. J Econ Theory 82: 167-189

Becker RA, Boyd III JH (1997) Capital theory, equilibrium analysis and recursive utility. Blackwell Publishers, New York

Benzion U, Rapoport A, Yagil J (1989) Discount rates inferred from decisions: an experimental study. Manage Sci 35: 270--284

Berge C (1963) Topological spaces. MacMillan, New York

Bhattacharya R, Majumdar M (2007) Random dynamical systems: theory and applications. Cambridge University Press, Cambridge, MA

Blackwell D (1965) Discounted dynamic programming. Ann Math Statist 36: 226-235

Boyd III JH (1990) Recursive utility and the Ramsey problem. J Econ Theory 50: 326-345

Boyd III JH (2006) Disrete-time recursive utility. In: Dana RA, Le Van C, Mitra T, Nishimura K (eds) Handbook of optimal growth 1, discrete time. Springer-Verlag, New York pp. 251-272

Denardo EV (1967) Contraction mappings in the theory underlying dynamic programming. SIAM Rev 9: 165--177

Dugundji J, Granas A (2003) Fixed point theory. Springer-Verlag, New York

Epstein L, Hynes JA (1983) The rate of time preference and dynamic economic analysis. J Polit Econ 91: 611-635

Feinberg EA (2002) Total reward criteria. In: Feinberg EA, Shwartz A (eds) Handbook of Markov decision processes. Kluwer Publishers, Boston pp. 173--207

Frederick S, Loewenstein G, O'Donoghue T (2002) Time discounting and time preference: a critical review. J Econ Lit 11: 351--401

Green L, Myerson J, McFadden E (1997) Rate of temporal discounting decreases with amount of reward. Memory and Cognition 25: 715--723

Hernandez-Lerma O, Lasserre JB (1996) Discrete-time Markov control processes: basic optimality criteria. Springer-Verlag, New York

Hernandez-Lerma O, Lasserre JB (1999) Further topics on discrete-time Markov control processes. Springer-Verlag, New York

Hinderer K (1970) Foundations of non-stationary dynamic programming with discrete time parameter. Lecture Notes in Oper Res 33, Springer-Verlag, New York

Kirby KN (1997) Bidding on the future: evidence against normative discounting of delayed rewards, J Exper Psych: General 126: 54--70

Koopmans TC (1960) Stationary ordinal utility and impatience. Econometrica 28: 287--309

Kreps DM, Porteus EL (1978) Temporal resolution of uncertainty and dynamic choice theory. Econometrica 46: 185-200

Le Van C, Morhaim L (2002) Optimal growth models with bounded or unbounded returns: a unifying approach. J Econ Theory 105: 158-187

Le Van C, Vailakis Y (2005) Recursive utility and optimal growth with bounded or unbounded returns. J Econ Theory 123: 187-209

Lucas RE, Stokey N (1984) Optimal growth with many consumers. J Econ Theory 32: 139--171

Marinacci M, Montrucchio L (2010) Unique solutions for stochastic recursive utilities. J Econom Theory 145: 1776--1804

Matkowski J (1975) Integral solutions of functional equations. Dissertationes Mathematicae 127: 1--68

Ramsey FP (1928) A mathematical theory of saving. Econ J 38: 543--599

Rincon-Zapatero JP, Rodriguez-Palmero C (2003) Existence and uniqueness of solutions to the Bellman equation in the unbounded case. Econometrica 71: 1519-1555

Rincon-Zapatero JP, Rodriguez-Palmero C (2009) Corrigendum to ``Existence and uniqueness of solutions to the Bellman equation in the unbounded case''. Econometrica 71 (2003), 1519-1555. Econometrica 77: 317-318

Samuelson P (1937) A note on measurement of utility. Rev of Econ Stud 4: 155--161

Schal M (1975) Conditions for optimality in dynamic programming and for the limit of n-stage optimal policies to be optimal. Z Wahrsch verw Geb 32: 179-196

Stokey NL, Lucas RE, Prescott E (1989) Recursive methods in economic dynamics. Harvard University Press, Cambridge, MA

Strauch R (1966) Negative dynamic programming. Ann Math Statist 37: 871--890

Thaler RH (1981) Some empirical evidence on dynamic inconsistency. Econ Lett 8: 201--207

Uzawa H (1968) Time preference, the consumption function, and optimum asset holding. In: Wolfe JN (ed) Value, capital and growth: papers in honor of Sir John Hicks. Edinburgh University Press, Edinburgh pp. 485--504