Mixtures of g-priors for Bayesian model averaging with economic applications

Ley, Eduardo and Steel, Mark F. J. (2011): Mixtures of g-priors for Bayesian model averaging with economic applications.

Abstract

We examine the issue of variable selection in linear regression modeling, where we have a potentially large amount of possible covariates and economic theory offers insufficient guidance on how to select the appropriate subset. In this context, Bayesian Model Averaging presents a formal Bayesian solution to dealing with model uncertainty. Our main interest here is the effect of the prior on the results, such as posterior inclusion probabilities of regressors and predictive performance. We combine a Binomial-Beta prior on model size with a g-prior on the coefficients of each model. In addition, we assign a hyperprior to g, as the choice of $g$ has been found to have a large impact on the results. For the prior on g, we examine the Zellner-Siow prior and a class of Beta shrinkage priors, which covers most choices in the recent literature. We propose a benchmark Beta prior, inspired by earlier findings with fixed g, and show it leads to consistent model selection. Inference is conducted through a Markov chain Monte Carlo sampler over model space and g. We examine the performance of the various priors in the context of simulated and real data. For the latter, we consider two important applications in economics, namely cross-country growth regression and returns to schooling. Recommendations to applied users are provided.

• Item Type: MPRA Paper
• Language: English
• Keywords: Consistency; Model uncertainty; Posterior odds; Prediction; Robustness
• Subjects: O - Economic Development, Innovation, Technological Change, and Growth > O4 - Economic Growth and Aggregate Productivity > O47 - Empirical Studies of Economic Growth ; Aggregate Productivity ; Cross-Country Output Convergence; C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C11 - Bayesian Analysis: General

References

Andrews, D.R., and C.L. Mallows (1974) "Scale Mixtures of Normal Distributions," Journal of the Royal Statistical Society}}, B, 36: 99--102.

Berger, J.O. (1985) Statistical Decision Theory and Bayesian Analysis, 2nd ed., New York: Springer.

Bernardo, J.M., and A.F.M. Smith (1994) Bayesian Theory, Chichester: John Wiley.

Berger, J.O. and L. Pericchi (1996) "The Intrinsic Bayes Factor for model selection and prediction," Journal of the American Statistical Association, 91: 109-122.

Berger, J.O., L. Pericchi and J. Varshavsky (1998) "Bayes factors and marginal distributions in invariant situations," Sankhya, Ser. A, 60: 307-321

Bottolo L., and Richardson S. (2008), "Fully Bayesian Variable Selection Using g-Priors," Working paper, Imperial College.

Bottolo L., and Richardson S. (2010), "Evolutionary Stochastic Search for Bayesian Model Exploration," Bayesian Analysis, 5: 583--618.

Brock, W., S. Durlauf and K. West (2003) "Policy Evaluation in Uncertain Economic Environments," (with discussion) Brookings Papers of Economic Activity, 1: 235--322.

Brown, P.J., M. Vannucci and T. Fearn (1998) "Bayesian Wavelength Selection in Multicomponent Analysis," Journal of Chemometrics, 12: 173--182.

Carvalho, C.M., N.G. Polson and J.G. Scott (2010) "The Horseshoe Estimator for Sparse Signals," Biometrika, 97: 465--480.

Chib, S. (1995) "Marginal Likelihood from the Gibbs Output," Journal of the American Statistical Association, 90: 1313--1321.

Ciccone, A. and Jarocinski, M. (2010). Determinants of Economic Growth: Will Data Tell? American Economic Journal: Macroeconomics, 2: 222--246.

Clyde, M.A., and E.I. George (2004) "Model Uncertainty," Statistical Science, 19: 81--94.

Clyde, M.A., J. Ghosh and M. Littman (2010) "Bayesian adaptive sampling for variable selection and model averaging", Journal of Computational and Graphical Statistics, forthcoming.

Cui, W., and George, E. I. (2008) "Empirical Bayes vs. fully Bayes variable selection," Journal of Statistical Planning and Inference, 138: 888--900.

Eicher, T.S., C. Papageorgiou and A.E. Raftery (2010) "Determining Growth Determinants: Default Priors and Predictive Performance in Bayesian Model Averaging, With Applications to Growth Determinants," Journal of Applied Econometrics, forthcoming.

Feldkircher, M. and S. Zeugner (2009) "Benchmark Priors Revisited: On Adaptive Shrinkage and the Supermodel Effect in Bayesian Model Averaging," IMF Working Paper 09/202.

Feldkircher, M. and S. Zeugner (2010) "The Impact of Data Revisions on the Robustness of Growth Determinants: A Note on `Determinants of Economic Growth. Will Data Tell'?," Working Paper 2010-12, University of Salzburg.

Fernandez, C., E. Ley and M.F.J. Steel (2001a) "Benchmark Priors for Bayesian Model Averaging," Journal of Econometrics, 100: 381--427.

Fernandez, C., E. Ley and M.F.J. Steel (2001b) "Model Uncertainty in Cross-Country Growth Regressions," Journal of Applied Econometrics, 16: 563--76.

Fernandez, C., and M.F.J. Steel (2000) "Bayesian Regression Analysis With Scale Mixtures of Normals," Econometric Theory, 16: 80-101

Foster, D.P., and E.I. George (1994), "The Risk Inflation Criterion for multiple regression," Annals of Statistics, 22: 1947--1975.

Forte, A., M.J. Bayarri, J.O. Berger and G. Garcia-Donato (2010), "Closed-form objective Bayes factors for variable selection in linear models", poster presentation Frontiers of Statistical Decision Making and Bayesian Analysis in honour of Jim Berger.

Gneiting, T. and A.E. Raftery (2007) "Strictly Proper Scoring Rules, Prediction and Estimation," Journal of the American Statistical Association, 102: 359--378.

Gradshteyn, I.S. and I.M. Ryzhik (1994) Table of Integrals, Series and Products, San Diego: Academic Press, 5th ed.

Hoeting, J.A., D. Madigan, A.E. Raftery and C.T. Volinsky (1999) "Bayesian model averaging: A tutorial," Statistical Science 14: 382--401.

Jeffreys, H. (1961) Theory of Probability, Oxford: Clarendon Press, 3rd ed.

Kass, R.E. and L. Wasserman (1995) "A reference Bayesian test for nested hypotheses and its relationship to the Schwarz criterion," Journal of the American Statistical Association, 90: 928--934.

Liang, F., R. Paulo, G. Molina, M.A. Clyde, and J.O. Berger (2008) "Mixtures of $g$-priors for Bayesian Variable Selection," Journal of the American Statistical Association, 103: 410--423.

Ley, E. and M.F.J. Steel (2009) "On the Effect of Prior Assumptions in Bayesian Model Averaging With Applications to Growth Regression," Journal of Applied Econometrics, 24: 651--674.

Madigan, D. and J. York (1995) "Bayesian graphical models for discrete data," International Statistical Review, 63: 215--232.

Maruyama, Y. and E.I. George (2010) "gBF: A Fully Bayes Factor with a Generalized $g$-prior," Technical Report, University of Pennsylvania, available at http://arxiv.org/abs/0801.4410

Mitchell, T.J. and J.J. Beauchamp (1988) "Bayesian variable selection in linear regression (with discussion), " Journal of the American Statistical Association, 83: 1023--1036.

Nott, D.J. and R. Kohn (2005) "Adaptive Sampling for Bayesian Variable Selection," Biometrika, 92: 747--763

Raftery, A.E., D. Madigan, and J. A. Hoeting (1997) "Bayesian Model Averaging for Linear Regression Models," Journal of the American Statistical Association, 92: 179--191.

Sala-i-Martin, X.X., G. Doppelhofer and R.I. Miller (2004) "Determinants of Long-Term Growth: A Bayesian Averaging of Classical Estimates (BACE) Approach," American Economic Review, 94: 813--835.

Strawderman, W. (1971) "Proper Bayes minimax estimators of the multivariate normal mean," Annals of Statistics, 42: 385--388.

Tobias, J.L. and Li, M. (2004) "Returns to Schooling and Bayesian Model Averaging; A Union of Two Literatures," Journal of Economic Surveys, 18: 153--180.

Verdinelli, I. and Wasserman, L. (1995) "Computing Bayes Factors Using a Generalization of the Savage-Dickey Density Ratio," Journal of the American Statistical Association, 90: 614--618.

Zellner, A. (1971) An Introduction to Bayesian Inference in Econometrics, New York: Wiley.

Zellner, A. (1986) "On assessing prior distributions and Bayesian regression analysis with g-prior distributions," in Bayesian Inference and Decision Techniques: Essays in Honour of Bruno de Finetti, eds. P.K. Goel and A. Zellner, Amsterdam: North-Holland, pp. 233--243.

Zellner, A. and Siow, A. (1980) "Posterior odds ratios for selected regression hypotheses," (with discussion) in Bayesian Statistics, eds. J.M. Bernardo, M.H. DeGroot, D.V. Lindley and A.F.M. Smith, Valencia: University Press, pp. 585--603.