Leeb, Hannes and Pötscher, Benedikt M. (2005): Can One Estimate the Unconditional Distribution of PostModelSelection Estimators ?
This is the latest version of this item.

PDF
MPRA_paper_1895.pdf Download (321Kb)  Preview 
Abstract
We consider the problem of estimating the unconditional distribution of a postmodelselection estimator. The notion of a postmodelselection estimator here refers to the combined procedure resulting from first selecting a model (e.g., by a model selection criterion like AIC or by a hypothesis testing procedure) and then estimating the parameters in the selected model (e.g., by leastsquares or maximum likelihood), all based on the same data set. We show that it is impossible to estimate the unconditional distribution with reasonable accuracy even asymptotically. In particular, we show that no estimator for this distribution can be uniformly consistent (not even locally). This follows as a corollary to (local) minimax lower bounds on the performance of estimators for the distribution; performance is here measured by the probability that the estimation error exceeds a given threshold. These lower bounds are shown to approach 1/2 or even 1 in large samples, depending on the situation considered. Similar impossibility results are also obtained for the distribution of linear functions (e.g., predictors) of the postmodelselection estimator.
Item Type:  MPRA Paper 

Original Title:  Can One Estimate the Unconditional Distribution of PostModelSelection Estimators ? 
Language:  English 
Keywords:  Inference after model selection; Postmodelselection estimator; Pretest estimator; Selection of regressors; Akaike's information criterion AIC; Thresholding; Model uncertainty; Consistency; Uniform consistency; Lower risk bound 
Subjects:  C  Mathematical and Quantitative Methods > C5  Econometric Modeling > C51  Model Construction and Estimation C  Mathematical and Quantitative Methods > C2  Single Equation Models; Single Variables > C20  General C  Mathematical and Quantitative Methods > C1  Econometric and Statistical Methods and Methodology: General > C12  Hypothesis Testing: General C  Mathematical and Quantitative Methods > C1  Econometric and Statistical Methods and Methodology: General > C13  Estimation: General C  Mathematical and Quantitative Methods > C5  Econometric Modeling > C52  Model Evaluation, Validation, and Selection 
Item ID:  1895 
Depositing User:  Benedikt Poetscher 
Date Deposited:  24. Feb 2007 
Last Modified:  17. Feb 2013 19:42 
References:  Ahmed, S. E. & A. K. Basu (2000): Least squares, preliminary test and Steintype estimation in general vector AR(p) models. Statistica Neerlandica 54, 4766. Bauer, P., Pötscher, B. M. & P. Hackl (1988): Model selection by multiple test procedures. Statistics 19, 3944. Billingsley, P. (1995): Probability and Measure, (3rd ed.), Wiley. Brownstone, D. (1990): Bootstrapping improved estimators for linear regression models. Journal of Econometrics 44, 171187. Danilov, D. L. & J. R. Magnus (2004): On the harm that ignoring pretesting can cause. Journal of Econometrics 122, 2746. Dijkstra, T. K. & J. H. Veldkamp (1988): `Datadriven selection of regressors and the bootstrap'. Lecture Notes in Economics and Mathematical Systems 307, 1738. Dukić, V. M. & E. A Peña (2005): Variance estimation in a model with gaussian submodels. Journal of the American Statistical Association 100, 296309. Freedman, D. A., Navidi, W. & S. C. Peters (1988): `On the impact of variable selection in fitting regression equations'. Lecture Notes in Economics and Mathematical Systems 307, 116. Hansen, P. R. (2003): Regression analysis with many specifications: a bootstrap method for robust inference. Working Paper, Department of Economics, Brown University. Hjort, N. L. & G. Claeskens (2003): Frequentist model average estimators. Journal of the American Statistical Association 98, 879899. Kabaila, P. (1995): The effect of model selection on confidence regions and prediction regions. Econometric Theory 11, 537549. Kapetanios, G. (2001): Incorporating lag order selection uncertainty in parameter inference for AR models. Economics Letters 72, 137144. Kilian, L. (1998): Accounting for lag order uncertainty in autoregressions: the endogenous lag order bootstrap algorithm. Journal of Time Series Analysis 19, 531548. Knight, K. (1999): Epiconvergence in distribution and stochastic equisemicontinuity. Working Paper, Department of Statistics, University of Toronto. Kulperger, R. J. & S. E. Ahmed (1992): A bootstrap theorem for a preliminary test estimator. Communications in Statistics: Theory and Methods 21, 20712082. Leeb, H. (2002): On a differential equation with advanced and retarded arguments. Communications on Applied Nonlinear Analysis 9, 7786. Leeb, H. (2005): The distribution of a linear predictor after model selection: conditional finitesample distributions and asymptotic approximations. Journal of Statistical Planning and Inference 134, 6489. Leeb, H. (2003): The distribution of a linear predictor after model selection: unconditional finitesample distributions and asymptotic approximations. IMS Lecture NotesMonograph Series 49, 291311. Leeb, H. & B. M. Pötscher (2003): The finitesample distribution of postmodelselection estimators and uniform versus nonuniform approximations. Econometric Theory 19, 100142. Leeb, H. & B. M. Pötscher (2005a): Model Selection and Inference: Facts and Fiction. Econometric Theory 21, 2159. Leeb, H. & B. M. Pötscher (2005b): Can one estimate the conditional distribution of postmodelselection estimators? Working Paper, Department of Statistics, University of Vienna. Leeb, H. & B. M. Pötscher (2006a): Performance limits for estimators of the risk or distribution of shrinkagetype estimators, and some general lower risk bound results. Econometric Theory 22, 6997. (Corrigendum, ibid., forthcoming.) Leeb, H. & B. M. Pötscher (2006b): Can one estimate the conditional distribution of postmodelselection estimators? Annals of Statistics 34, 25542591. Lehmann, E.L. & G. Casella (1998): Theory of Point Estimation, 2nd edition, Springer Texts in Statistics. SpringerVerlag. Nickl, R. (2003): Asymptotic Distribution Theory of PostModelSelection Estimators. Masters Thesis, Department of Statistics, University of Vienna. Pötscher, B. M. (1991): Effects of model selection on inference. Econometric Theory 7, 163185. Pötscher, B. M. (1995): Comment on `The effect of model selection on confidence regions and prediction regions' by P. Kabaila. Econometric Theory 11, 550559. Pötscher, B. M. & A. J. Novak (1998): The distribution of estimators after model selection: large and small sample results. Journal of Statistical Computation and Simulation 60, 1956. Rao, C. R. & Y. Wu (2001): `On model selection,' IMS Lecture Notes Monograph Series 38, 157. Sen, P. K. (1979): Asymptotic properties of maximum likelihood estimators based on conditional specification. Annals of Statistics 7, 10191033. Sen P. K. & A. K. M. E. Saleh (1987): On preliminary test and shrinkage Mestimation in linear models. Annals of Statistics 15, 15801592. van der Vaart, A. W. (1998): Asymptotic Statistics. Cambridge University Press. 
URI:  http://mpra.ub.unimuenchen.de/id/eprint/1895 
Available Versions of this Item

Can One Estimate the Unconditional Distribution of PostModelSelection Estimators ? (deposited 12. Oct 2006)
 Can One Estimate the Unconditional Distribution of PostModelSelection Estimators ? (deposited 24. Feb 2007) [Currently Displayed]