Interpolating Value Functions in Discrete Choice Dynamic Programming Models

Sullivan, Paul (2006): Interpolating Value Functions in Discrete Choice Dynamic Programming Models. Unpublished.

Abstract

Structural discrete choice dynamic programming models have been shown to be a valuable tool for analyzing a wide range of economic behavior. A major limitation on the complexity and applicability of these models is the computational burden associated with computing the high dimensional integrals that typically characterize an agent's decision rules. This paper develops a regression based approach to interpolating value functions during the solution of dynamic programming models that alleviates this burden. This approach is suitable for use in models that incorporate unobserved state variables that are serially correlated across time and correlated across choices within a time period. The key assumption is that one unobserved state variable, or error term, in the model is distributed extreme value. Additional error terms that allow for correlation between unobservables across time or across choices within a given time period may be freely incorporated in the model. Value functions are simulated at a fraction of the state space and interpolated at the remaining points using a new regression function based on the extreme value closed form solution for the expected maxima of the value function. This regression function is well suited for use in models with large choice sets and complicated error structures. The performance of the interpolation method appears to be excellent, and it greatly reduces the computational burden of estimating the parameters of a dynamic programming model.

• Item Type: MPRA Paper
• Institution: Bureau of Labor Statistics
• Language: English
• Keywords: dynamic programming models; interpolation; simulation methods; estimation of dynamic programming models
• Subjects: C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C50 - General; C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods: General > C15 - Statistical Simulation Methods; Monte Carlo Methods; Bootstrap Methods

References

Arcidiacono, Peter (2004). "Ability Sorting and the Returns to College Major." Journal of Econometrics, v. 121: 343-375.

Bellman, Richard (1957). "Dynamic Programming." Princeton University Press.

Berkovec, James, and Steven Stern (1991). "Job Exit Behavior of Older Men." Econometrica, v. 59, no. 1: 189-210.

Berry, Steven (1992). "Estimation of a Model of Entry in the Airline Industry." Econometrica, v. 60, no. 4: 889-917.

Berry, Steven, James Levinsohn, and Ariel Pakes (1995). "Automobile Prices in Market Equilibrium." Econometrica, v. 63, no. 4: 841-890.

Brien, Michael, Lee Lillard and Steven Stern (2006). "Cohabitation, Marriage and Divorce in a Model of Match Quality." International Economic Review, v. 47, no. 2.

Eckstein, Zvi and Kenneth Wolpin (1989). "The Specification and Estimation of Discrete Choice Dynamic Programming Models." Journal of Human Resources, v. 24: 562-598.

Geweke, John (1988). "Antithetic Acceleration of Monte Carlo Integration in Bayesian Inference." Journal of Econometrics, v. 38: 73-89.

Heckman, James, and Burton Singer (1984). "A Method for Minimizing the Impact of Distributional Assumptions in Econometric Models for Duration Data." Econometrica, v. 52: 271-320.

Hotz, Joseph and Robert Miller (1993). "Conditional Choice Probabilities and the Estimation of Dynamic Programming Models." Review of Economics and Statistics, v. 60: 497-530.

Keane, Michael and Kenneth Wolpin (1994). "The Solution and Estimation of Discrete Choice Dynamic Programming Models by Simulation and Interpolation: Monte Carlo Evidence." The Review of Economics and Statistics, v. 76, no. 4: 648-672.

Keane, Michael, and Kenneth Wolpin (1997). "The Career Decisions of Young Men." Journal of Political Economy, v. 105 : 474-521.

Manski, Charles. "Nonparametric Estimation of Expectations in the Analysis of Discrete Choice Under Uncertainty." In Nonparametric and Semiparametric Methods in Econometrics and Statistics, edited by W. Barnett, J. Powell, and G. Tauchen. pp. 259-276. MIT Press, Cambridge.

McFadden, Daniel (1981). "Econometric Models of Probabilistic Choice." In C. Manski and D. McFadden (eds.) Structural Analysis of Discrete Data with Econometric Applications, pp. 198-272. MIT Press, Cambridge.

Miller, Robert A. (1984). "Job Matching and Occupational Choice." Journal of Political Economy, v. 92: 1086-1120.

Rust, John (1987). "Optimal Replacement of GMC Bus Engines: An Empirical Model of Howard Zurcher." Econometrica, v. 55: 999-1033.

Rust, John (1997). "Using Randomization to Break the Curse of Dimensionality." Econometrica, v. 65, no. 3.

Rust, John (2005). "Structural Estimation of Markov Decision Processes." In Handbook of Econometrics Vol. 4, edited by R. Engle and D. McFadden. North-Holland.

rust+phelan : Rust, John and Christopher Phelan (1997). "How Social Security and Medicare Affect Retirement Behavior in a World of Incomplete Markets." Econometrica, v. 65: 781-831.

Stern, Steven (1997). "Simulation Based Estimation." Journal of Economic Literature, v. 35, no. 4.

Stinebrickner, Todd (2000). "Serially Correlated Variables in Dynamic Discrete Choice Models." Journal of Applied Econometrics, v. 15: 595-624.

Stinebrickner, Todd (2001). "Compensation Policies and Teacher Decisions." International Economic Review, v. 42, No. 3: 751-779.

Sullivan, Paul (2006). "A Dynamic Analysis of Educational Attainment, Occupational Choices, and Job Search." Working paper.