Maximum likelihood estimation of time series models: the Kalman filter and beyond

Tommaso, Proietti and Alessandra, Luati (2012): Maximum likelihood estimation of time series models: the Kalman filter and beyond.

Abstract

The purpose of this chapter is to provide a comprehensive treatment of likelihood inference for state space models. These are a class of time series models relating an observable time series to quantities called states, which are characterized by a simple temporal dependence structure, typically a first order Markov process.

The states have sometimes substantial interpretation. Key estimation problems in economics concern latent variables, such as the output gap, potential output, the non-accelerating-inflation rate of unemployment, or NAIRU, core inflation, and so forth. Time-varying volatility, which is quintessential to finance, is an important feature also in macroeconomics. In the multivariate framework relevant features can be common to different series, meaning that the driving forces of a particular feature and/or the transmission mechanism are the same.

The objective of this chapter is reviewing this algorithm and discussing maximum likelihood inference, starting from the linear Gaussian case and discussing the extensions to a nonlinear and non Gaussian framework.

• Item Type: MPRA Paper
• Language: English
• Keywords: Time series models; Unobserved components
• Subjects: C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C13 - Estimation: General; C - Mathematical and Quantitative Methods > C2 - Single Equation Models ; Single Variables > C22 - Time-Series Models ; Dynamic Quantile Regressions ; Dynamic Treatment Effect Models ; Diffusion Processes

References

Amisano, G. and Tristani, O. (2010). Euro area inflation persistence in an estimated nonlinear DSGE model. Journal of Economic Dynamics and Control, 34, 1837–1858.

Anderson, B.D.O., and J.B. Moore (1979). Optimal Filtering. Englewood Cliffs: Prentice-Hall.

Brockwell, P.J. and Davis, R.A. (1991), Time Series: Theory and Methods, Springer.

Bryson, A.E., and Ho, Y.C. (1969). Applied optimal control: optimization, estimation, and control. Blaisdell Publishing, Waltham, Mass.

Burridge, P. and Wallis, K.F. (1988). Prediction Theory for Autoregressive-Moving Average Processes. Econometric Reviews, 7, 65-9.

Caines P.E. (1988). Linear Stochastic Systems. Wiley Series in Probability and Mathematical Statistics, John Wiley & Sons, New York.

Canova, F. (2007), Methods for Applied Macroeconomic Research. Princeton University Press,

Capp´e, O., Moulines, E., and Ryd´en, T. (2005). Inference in hidden markov models. Springer Series in Statistics. Springer, New York.

Chang, Y., Miller, J.I., and Park, J.Y. (2009), Extracting a Common Stochastic Trend: Theory with some Applications, Journal of Econometrics, 15, 231–247.

Clark, P.K. (1987). The Cyclical Component of U. S. Economic Activity, The Quarterly Journal of Economics, 102, 4, 797–814.

Cogley, T., Primiceri, G.E., Sargent, T.J. (2010), Inflation-Gap Persistence in the U.S., American Economic Journal: Macroeconomics, 2(1), January 2010, 43–69.

Creal,D. , (2012) A survey of sequential Monte Carlo methods for economics and finance, Econometric Reviews, 31, 3, 245–296.

Creal, D., Koopman, S.J. and Lucas A. (2011a), Generalized Autoregressive Score Models with Applications, Journal of Applied Econometrics, forthcoming.

Creal, D., Koopman, S.J. and Lucas A. (2011b), A Dynamic Multivariate Heavy-Tailed Model for Time- Varying Volatilities and Correlations, Journal of Business and Economics Statistics, 29, 4, 552–563.

de Jong, P. (1988a). The likelihood for a state space model. Biometrika 75: 165-9. 24

de Jong, P. (1989). Smoothing and interpolation with the state space model. Journal of the American Statistical Association, 84, 1085-1088.

de Jong, P (1991). The diffuse Kalman filter. Annals of Statistics 19, 1073-83.

de Jong, P., and Chu-Chun-Lin, S. (1994). Fast Likelihood Evaluation and Prediction for Nonstationary State Space Models. Biometrika, 81, 133-142.

de Jong, P. and Penzer, J. (2004), The ARMA model in state space form, Statistics and Probability Letters, 70, 119–125

Dempster, A. P., Laird, N. M., and Rubin, D. B. (1977). Maximum likelihood estimation from incomplete data. Journal of the Royal Statistical Society, 14, 1:38.

Doran, E. (1992). Constraining Kalman Filter and Smoothing Estimates to Satisfy Time-Varying Restrictions. Review of Economics and Statistics, 74, 568-572.

Doucet, A., de Freitas, J. F. G. and Gordon, N. J. (2001). Sequential Monte Carlo Methods in Practice. New York: Springer-Verlag.

Durbin, J., and S.J. Koopman (1997). Monte Carlo maximum likelihood estimation for non-Gaussian state space models. Biometrika 84, 669-84.

Durbin, J., and Koopman, S.J. (2000). Time series analysis of non-Gaussian observations based on state-space models from both classical and Bayesian perspectives (with discussion). Journal of Royal Statistical Society, Series B, 62, 3-56.

Durbin, J., and S.J. Koopman (2001). Time Series Analysis by State Space Methods. Oxford University Press, Oxford.

Durbin, J., and S.J. Koopman (2002). A simple and efficient simulation smoother for state space time series analysis. Biometrika, 89, 603-615.

Farhmeir, L. and Tutz G. (1994). Multivariate Statistical Modelling Based Generalized Linear Models, Springer-Verlag, New-York.

Fernndez-Villaverde, J. and Rubio-Ramrez, J.F. (2005), Estimating Dynamic Equilibrium Economies: Linear versus Non-Linear Likelihood, Journal of Applied Econometrics, 20, 891910.

Fernndez-Villaverde, J. and Rubio-Ramrez, J.F. (2007). Estimating Macroeconomic Models: A Likelihood Approach. Review of Economic Studies, 74, 1059–1087.

Fernndez-Villaverde, J. (2010), The Econometrics of DSGE Models, Journal of the Spanish Economic Association 1, 3–49.

Frale, C., Marcellino, M., Mazzi, G. and Proietti, T. (2011), EUROMIND: A Monthly Indicator of the Euro Area Economic Conditions, Journal of the Royal Statistical Society - Series A, 174, 2, 439–470.

Francke, M.K., Koopman, S.J., de Vos, A. (2010), Likelihood functions for state space models with diffuse initial conditions, Journal of Time Series Analysis 31, 407–414. 25

Fr¨uhwirth-Schnatter, S. (2006). Finite Mixture and Markov Switching Models. Springer Series in Statistics. Springer, New York.

Gamerman, D. and Lopes H. F. (2006). Markov Chain Monte Carlo. Stochastic Simulation for Bayesian Inference, Second edition, Chapman & Hall, London.

Geweke, J.F., and Singleton, K.J. (1981). Maximum likelihood confirmatory factor analysis of economic time series. International Economic Review, 22, 1980.

Golub, G.H., and van Loan, C.F. (1996), Matrix Computations, third edition, The John Hopkins University Press.

Gordon, N. J., Salmond, D. J. and Smith, A. F. M. (1993). A novel approach to non-linear and non- Gaussian Bayesian state estimation. IEE-Proceedings F 140, 107-113.

Hannan, E.J., and Deistler, M. (1988). The Statistical Theory of Linear Systems. Wiley Series in Probability and Statistics, John Wiley & Sons.

Harvey, A.C. (1989). Forecasting, Structural Time Series and the Kalman Filter. Cambridge University Press, Cambridge, UK.

Harvey, A.C. (2001). Testing in Unobserved Components Models. Journal of Forecasting, 20, 1-19.

Harvey, A.C., (2010), Exponential Conditional Volatility Models , working paper CWPE 1040.

Harvey, A.C., and Chung, C.H. (2000). Estimating the underlying change in unemployment in the UK. Journal of the Royal Statistics Society, Series A, Statistics in Society, 163, Part 3, 303-339.

Harvey, A.C., and J¨ager, A. (1993). Detrending, stylized facts and the business cycle. Journal of Applied Econometrics, 8, 231-247.

Harvey, A.C., and Proietti, T. (2005). Readings in Unobserved Components Models. Advanced Texts in Econometrics. Oxford University Press, Oxford, UK.

Harvey, A.C., and Chakravarty, T. (2008). Beta-t(E)GARCH, working paper, CWPE 0840.

Harville, D. A. (1977) Maximum likelihood approaches to variance component estimation and to related problems, Journal of the American Statistical Association, 72, 320–340.

Hodrick, R., and Prescott, E.C. (1997). Postwar U.S. Business Cycle: an Empirical Investigation, Journal of Money, Credit and Banking, 29, 1, 1-16.

Ionides, E. L., Breto, C. and King, A. A. (2006), Inference for nonlinear dynamical systems, Proceedings of the National Academy of Sciences 103, 18438–18443.

Ionides, E. L, Bhadra, A., Atchade, Y. and King, A. A. (2011), Iterated filtering, Annals of Statistics, 39, 1776–1802.

Jazwinski, A.H. (1970). Stochastic Processes and Filtering Theory. Academic Press, New York. 26

Julier S.J., and Uhlmann, J.K. (1996), A General Method for Approximating Nonlinear Transformations of Probability Distributions, Robotics Research Group, Oxford University, 4, 7, 1–27.

Julier S.J., and Uhlmann, J.K. (1997), A New Extension of the Kalman Filter to Nonlinear Systems, Proceedings of AeroSense: The 11th International Symposium on Aerospace/Defense Sensing, Simulation and Controls.

Jungbacker, B., Koopman, S.J., and van der Wel, M., (2011), Maximum likelihood estimation for dynamic factor models with missing data, Journal of Economic Dynamics and Control, 35, 8, 1358– 1368.

Kailath, T., Sayed, A.H., and Hassibi, B. (2000), Linear Estimation, Prentice Hall, Upper Saddle River, New Jersey.

Kalman, R.E. (1960). A new approach to linear filtering and prediction problems. Journal of Basic Engineering, Transactions ASME. Series D 82: 35-45.

Kalman, R.E., and R.S. Bucy (1961). New results in linear filtering and prediction theory, Journal of Basic Engineering, Transactions ASME, Series D 83: 95-108.

Kim, C.J. and C. Nelson (1999). State-Space Models with Regime-Switching. Cambridge MA: MIT Press.

Kitagawa, G. (1987). Non-Gaussian State-Space Modeling of Nonstationary Time Series (with discussion), Journal of the American Statistical Association, 82, 10321063.

Kitagawa, G. (1998). A self-organising state-space model, Journal of the American Statistical Association, 93, 1203-1215.

Kitagawa, G. (1996). Monte Carlo Filter and Smoother for Non-Gaussian Nonlinear State-Space Models, Journal of Computational and Graphical Statistics, 5, 125.

Kitagawa, G., andWGersch (1996). Smoothness priors analysis of time series. Berlin: Springer-Verlag.

Koopman, S.J., and Durbin, J. (2000). Fast filtering and smoothing for multivariate state space models, Journal of Time Series Analysis, 21, 281–296.

Luati, A. and Proietti, T. (2010). Hyper-spherical and Elliptical Stochastic Cycles, Journal of Time Series Analysis, 31, 169–181.

Morley, J.C., Nelson, C.R., and Zivot, E. (2002). Why are Beveridge-Nelson and Unobserved-Component Decompositions of GDP So Different?, Review of Economics and Statistics, 85, 235-243.

Nelson, C.R., and Plosser, C.I. (1982). Trends and random walks in macroeconomic time series: some evidence and implications. Journal of Monetary Economics, 10, 139-62.

Nerlove, M., Grether, D. M., and Carvalho, J. L. (1979), Analysis of Economic Time Series: A Synthesis, New York: Academic Press.

Nyblom, J. (1986). Testing for deterministic linear trend in time series. Journal of the American Statistical Association, 81: 545-9.

Nyblom, J.(1989). Testing for the constancy of parameters over time. Journal of the American Statistical Association, 84, 223-30.

Nyblom, J., and Harvey, A.C. (2000). Tests of common stochastic trends, Econometric Theory, 16, 176-99.

Nyblom J., M¨akel¨ainen T. (1983). Comparison of tests for the presence of random walk coefficients in a simple linear model. Journal of the American Statistical Association, 78, 856864.

Ord J.K., Koehler A.B., and Snyder, R.D. (1997). Estimation and prediction for a class of Dynamic nonlinear statistical models. Journal of the American Statistical Association, 92, 1621-1629.

Pagan, A. (1980). Some Identification and Estimation Results for Regression Models with Stochastically Varying Coefficients Journal of Econometrics, 13, 341–363.

Patterson, H.D. and Thompson, R. (1971) Recovery of inter-block information when block sizes are unequal, Biometrika, 58, 545–554. Pearlman, J. G. (1980). An Algorithm for the Exact Likelihood of a High-Order Autoregressive-Moving Average Process. Biometrika, 67: 232-233.

Pitt, M.K. and Shephard, N. (1999). Filtering via simulation: auxiliary particle filters. Journal of the American Statistical Association, 94, 590-599.

Poyiadjis, G and Doucet, A and Singh, SS (2011) Particle approximations of the score and observed information matrix in state space models with application to parameter estimation. Biometrika, 98, 65–80.

Primiceri, G.E. (2005), Time Varying Structural Vector Autoregressions and Monetary Policy, The Review of Economic Studies, 72, 821–852

Proietti T. (1999). Characterising Business Cycle Asymmetries by Smooth Transition Structural Time Series Models. Studies in Nonlinear Dynamics and Econometrics, 3, 141–156.

Proietti T. (2006), Trend–Cycle Decompositions with Correlated Components. Econometric Reviews, 25, 61-84

Richard, J.F. and Zhang, W. (2007), Efficient high-dimensional importance sampling, Journal of Econometrics 127, , 1385–1411.

Rosenberg, B. (1973). Random coefficient models: the analysis of a cross-section of time series by stochastically convergent parameter regression. Annals of Economic and Social Measurement, 2, 399-428.

Rubin, D. B. (1987). A noniterative sampling/importance resampling alternative to the data augmentation algorithm for creating a few imputations when the fraction of missing information is modest: the SIR algorithm. Discussion of Tanner and Wong (1987). Journal of the American Statistical Association, 82, 543-546.

Sargent, T.J. (1989), Two Models of Measurements and the Investment Accelerator, Journal of Political Economy, 97, 2, 251–287,

Sargent, T.J., and C.A. Sims (1977), Business Cycle Modeling Without Pretending to Have Too Much A-Priori Economic Theory, in New Methods in Business Cycle Research, ed. by C. Sims et al., Minneapolis: Federal Reserve Bank of Minneapolis.

Smets, F. and Wouters, R. (2003), An Estimated Dynamic Stochastic General Equilibrium Model of the Euro Area, Journal of the European Economic Association,1, 5, 1123–1175.

Shephard, N. (2005). Stochastic Volatility: Selected Readings. Advanced Texts in Econometrics. Oxford University Press, Oxford, UK.

Shephard, N. and Pitt, M. K. (1997). Likelihood analysis of non-Gaussian measurement time series. Biometrika, 84, 653-667.

Shumway, R.H., and Stoffer, D.S. (1982). An approach to time series smoothing and forecasting using the EM algorithm. Journal of Time Series Analysis, 3, 253-264.

Stock, J.H., and M.W. Watson (1989), New Indexes of Coincident and Leading Economic Indicators, NBER Macroeconomics Annual 1989, 351-393.

Stock, J.H., and Watson M.W. (1991). A probability model of the coincident economic indicators. In Leading Economic Indicators, Lahiri K, Moore GH (eds), Cambridge University Press, New York.

Stock, J.H. and Watson, M.W. (2007), Why Has U.S. Inflation Become Harder to Forecast?, Journal of Money, Credit and Banking, 39(1), 3-33.

Tunnicliffe-Wilson, G. (1989). On the use of marginal likelihood in time series model estimation. Journal of the Royal Statistical Society, Series B, 51, 15-27.

van der Merwe, R., Doucet, A., De Freitas, N., Wan, E. (2000), The Unscented Particle Filter, Advances in Neural Information Processing Systems, 13, 584-590. Watson, M.W. (1986). Univariate detrending methods with stochastic trends. Journal of Monetary Economics, 18, 49-75.

West, M. and P.J.Harrison (1989). Bayesian Forecasting and Dynamic Models. New York: Springer- Verlag.

Winschel, W. and Kr¨atzig, M. (2010), Solving, Estimating, and Selecting Nonlinear Dynamic Models without the Curse of Dimensionality, Econometrica, 39, 1, 3–33.