Ley, Eduardo and Steel, Mark F. J. (2011): Mixtures of gpriors for Bayesian model averaging with economic applications. Forthcoming in: Journal of Econometrics
This is the latest version of this item.

PDF
MPRA_paper_36817.pdf Download (938kB)  Preview 
Abstract
We examine the issue of variable selection in linear regression modeling, where we have a potentially large amount of possible covariates and economic theory offers insufficient guidance on how to select the appropriate subset. In this context, Bayesian Model Averaging presents a formal Bayesian solution to dealing with model uncertainty. Our main interest here is the effect of the prior on the results, such as posterior inclusion probabilities of regressors and predictive performance. We combine a BinomialBeta prior on model size with a gprior on the coefficients of each model. In addition, we assign a hyperprior to g, as the choice of g has been found to have a large impact on the results. For the prior on g, we examine the ZellnerSiow prior and a class of Beta shrinkage priors, which covers most choices in the recent literature. We propose a benchmark Beta prior, inspired by earlier findings with fixed g, and show it leads to consistent model selection. The effect of this prior structure on penalties for complexity and lack of fit is described in some detail. Inference is conducted through a Markov chain Monte Carlo sampler over model space and g. We examine the performance of the various priors in the context of simulated and real data. For the latter, we consider two important applications in economics, namely crosscountry growth regression and returns to schooling. Recommendations to applied users are provided.
Item Type:  MPRA Paper 

Original Title:  Mixtures of gpriors for Bayesian model averaging with economic applications 
Language:  English 
Keywords:  Complexity penalty; Consistency; Model uncertainty; Posterior odds; Prediction; Robustness 
Subjects:  O  Economic Development, Technological Change, and Growth > O4  Economic Growth and Aggregate Productivity > O47  Measurement of Economic Growth; Aggregate Productivity; CrossCountry Output Convergence C  Mathematical and Quantitative Methods > C1  Econometric and Statistical Methods and Methodology: General > C11  Bayesian Analysis: General 
Item ID:  36817 
Depositing User:  Eduardo Ley 
Date Deposited:  21. Feb 2012 04:52 
Last Modified:  20. Feb 2013 10:23 
References:  Andrews, D.R., and C.L. Mallows (1974) Scale Mixtures of Normal Distributions, Journal of the Royal Statistical Society, B, 36: 99102. Berger, J.O. (1985) Statistical Decision Theory and Bayesian Analysis, 2nd ed., New York: Springer. Berger, J.O. and L. Pericchi (1996) The Intrinsic Bayes factor for model selection and prediction, Journal of the American Statistical Association, 91: 109122. Berger, J.O., L. Pericchi and J. Varshavsky (1998) Bayes factors and marginal distributions in invariant situations, Sankhy\={a, Ser.~A, 60: 307321. Bernardo, J.M., and A.F.M. Smith (1994) Bayesian Theory, Chichester: John Wiley. Bottolo L., and Richardson S. (2008), Fully Bayesian Variable Selection Using gPriors, Working paper, Imperial College. Bottolo L., and Richardson S. (2010), Evolutionary Stochastic Search for Bayesian Model Exploration, Bayesian Analysis, 5: 583618. Brock, W., S. Durlauf and K. West (2003) Policy Evaluation in Uncertain Economic Environments, (with discussion) Brookings Papers of Economic Activity, 1: 235322. Brown, P.J., M. Vannucci and T. Fearn (1998) Bayesian Wavelength Selection in Multicomponent Analysis, Journal of Chemometrics, 12: 173182. Carvalho, C.M., N.G. Polson and J.G. Scott (2010) The Horseshoe Estimator for Sparse Signals, Biometrika, 97: 465480. Chib, S. (1995) Marginal Likelihood from the Gibbs Output, Journal of the American Statistical Association, 90: 13131321. Ciccone, A. and Jarocinski, M. (2010). Determinants of Economic Growth: Will Data Tell? American Economic Journal: Macroeconomics, 2: 222246. Clyde, M.A., and E.I. George (2004) Model Uncertainty, Statistical Science, 19: 8194. Clyde, M.A., J. Ghosh and M. Littman (2011) Bayesian adaptive sampling for variable selection and model averaging , Journal of Computational and Graphical Statistics, 20: 80101. Cui, W., and George, E. I. (2008) Empirical Bayes vs. fully Bayes variable selection, Journal of Statistical Planning and Inference, 138: 888900. Eicher, T.S., C. Papageorgiou and A.E. Raftery (2011) Default Priors and Predictive Performance in Bayesian Model Averaging, With Application to Growth Determinants, Journal of Applied Econometrics, 26: 3055. Feldkircher, M. and S. Zeugner (2009) Benchmark Priors Revisited: On Adaptive Shrinkage and the Supermodel Effect in Bayesian Model Averaging, IMF Working Paper 09/202. Feldkircher, M. and S. Zeugner (2012) The Impact of Data Revisions on the Robustness of Growth Determinants: A Note on `Determinants of Economic Growth. Will Data Tell'?, Journal of Applied Econometrics, forthcoming. Fernandez, C., E. Ley and M.F.J. Steel (2001a) Benchmark Priors for Bayesian Model Averaging, Journal of Econometrics, 100: 381427. Fernandez, C., E. Ley and M.F.J. Steel (2001b) Model Uncertainty in CrossCountry Growth Regressions, Journal of Applied Econometrics, 16: 56376. Fernandez, C., and M.F.J. Steel (2000) Bayesian Regression Analysis With Scale Mixtures of Normals, Econometric Theory, 16: 80101 Foster, D.P., and E.I. George (1994), The Risk Inflation Criterion for multiple regression, Annals of Statistics, 22: 19471975. Forte, A., M.J. Bayarri, J.O. Berger and G. Garc{{\iaDonato (2010), Closedform objective Bayes factors for variable selection in linear models , poster presentation Frontiers of Statistical Decision Making and Bayesian Analysis in honour of Jim Berger. GarciaDonato, G. and M.A. Mart{{\inezBeneito (2011), Inferences in Bayesian variable selection problems with large model spaces, Technical Report, arXiv:1101.4368v1. Gneiting, T. and A.E. Raftery (2007) Strictly Proper Scoring Rules, Prediction and Estimation, Journal of the American Statistical Association, 102: 359378. Gradshteyn, I.S. and I.M. Ryzhik (1994) Table of Integrals, Series and Products, San Diego: Academic Press, 5th ed. Hoeting, J.A., D. Madigan, A.E. Raftery and C.T. Volinsky (1999) Bayesian model averaging: A tutorial, Statistical Science\/ 14: 382401. Jeffreys, H. (1961) Theory of Probability, Oxford: Clarendon Press, 3rd ed. Kass, R.E. and L. Wasserman (1995) A reference Bayesian test for nested hypotheses and its relationship to the Schwarz criterion, Journal of the American Statistical Association, 90: 928934. Liang, F., R. Paulo, G. Molina, M.A. Clyde, and J.O. Berger (2008) Mixtures of gpriors for Bayesian Variable Selection, Journal of the American Statistical Association, 103: 410423. Ley, E. and M.F.J. Steel (2009) On the Effect of Prior Assumptions in Bayesian Model Averaging With Applications to Growth Regression, Journal of Applied Econometrics, 24: 651674. Madigan, D. and J. York (1995) Bayesian graphical models for discrete data, International Statistical Review, 63: 215232. Maruyama, Y. and E.I. George (2011) Fully Bayes Factors with a Generalized gprior, Annals of Statistics, forthcoming, arXiv:0801.4410. Mitchell, T.J. and J.J. Beauchamp (1988) Bayesian variable selection in linear regression (with discussion), Journal of the American Statistical Association, 83: 10231036. Nott, D.J. and R. Kohn (2005) Adaptive Sampling for Bayesian Variable Selection, Biometrika, 92: 747763 Raftery, A.E., D. Madigan, and J. A. Hoeting (1997) Bayesian Model Averaging for Linear Regression Models, Journal of the American Statistical Association, 92: 179191. SalaiMartin, X.X., G. Doppelhofer and R.I. Miller (2004) Determinants of LongTerm Growth: A Bayesian Averaging of Classical Estimates (BACE) Approach, American Economic Review, 94: 813835. Scott, J.G. and J.O. Berger (2010) Bayes and empirical Bayes multiplicity adjustment in the variableselection problem, Annals of Statistics, 38: 25872619. Strawderman, W. (1971) Proper Bayes minimax estimators of the multivariate normal mean, Annals of Mathematical Statistics, 42: 385388. Tobias, J.L. and Li, M. (2004) Returns to Schooling and Bayesian Model Averaging; A Union of Two Literatures, Journal of Economic Surveys, 18: 153180. Verdinelli, I. and Wasserman, L. (1995) Computing Bayes Factors Using a Generalization of the SavageDickey Density Ratio, Journal of the American Statistical Association, 90: 614ñ618. Zellner, A. (1971) An Introduction to Bayesian Inference in Econometrics, New York: Wiley. Zellner, A. (1986) On assessing prior distributions and Bayesian regression analysis with gprior distributions, in Bayesian Inference and Decision Techniques: Essays in Honour of Bruno de Finetti, eds.~P.K. Goel and A. Zellner, Amsterdam: NorthHolland, pp. 233243. Zellner, A. and Siow, A. (1980) Posterior odds ratios for selected regression hypotheses, (with discussion) in Bayesian Statistics, eds.~J.M. Bernardo, M.H. DeGroot, D.V. Lindley and A.F.M. Smith, Valencia: University Press, pp. 585603. 
URI:  http://mpra.ub.unimuenchen.de/id/eprint/36817 
Available Versions of this Item
 Mixtures of gpriors for Bayesian model averaging with economic applications. (deposited 21. Feb 2012 04:52) [Currently Displayed]