Leeb, Hannes and Pötscher, Benedikt M. (2005): Can One Estimate the Unconditional Distribution of PostModelSelection Estimators ?
There is a more recent version of this item available. 

PDF
MPRA_paper_72.pdf Download (309kB)  Preview 
Abstract
We consider the problem of estimating the unconditional distribution of a postmodelselection estimator. The notion of a postmodelselection estimator here refers to the combined procedure resulting from first selecting a model (e.g., by a model selection criterion like AIC or by a hypothesis testing procedure) and then estimating the parameters in the selected model (e.g., by leastsquares or maximum likelihood), all based on the same data set. We show that it is impossible to estimate the unconditional distribution with reasonable accuracy even asymptotically. In particular, we show that no estimator for this distribution can be uniformly consistent (not even locally). This follows as a corollary to (local) minimax lower bounds on the performance of estimators for the distribution. These lower bounds are shown to approach 1/2 or even 1 in large samples, depending on the situation considered. Similar impossibility results are also obtained for the distribution of linear functions (e.g., predictors) of the postmodelselection estimator.
Item Type:  MPRA Paper 

Original Title:  Can One Estimate the Unconditional Distribution of PostModelSelection Estimators ? 
Language:  English 
Keywords:  Inference after model selection; Postmodelselection estimator; Pretest estimator; Selection of regressors; Akaike's information criterion AIC; Thresholding; Model uncertainty; Consistency; Uniform consistency; Lower risk bound 
Subjects:  C  Mathematical and Quantitative Methods > C2  Single Equation Models ; Single Variables > C20  General C  Mathematical and Quantitative Methods > C1  Econometric and Statistical Methods and Methodology: General > C13  Estimation: General C  Mathematical and Quantitative Methods > C5  Econometric Modeling > C52  Model Evaluation, Validation, and Selection C  Mathematical and Quantitative Methods > C1  Econometric and Statistical Methods and Methodology: General > C12  Hypothesis Testing: General C  Mathematical and Quantitative Methods > C5  Econometric Modeling > C51  Model Construction and Estimation 
Item ID:  72 
Depositing User:  Benedikt Poetscher 
Date Deposited:  12. Oct 2006 
Last Modified:  25. May 2015 03:48 
References:  Ahmed, S. E. & A. K. Basu (2000): Least squares, preliminary test and Steintype estimation in general vector AR(p) models. Statistica Neerlandica 54, 4766. Bauer, P., Pötscher, B. M. & P. Hackl (1988): Model selection by multiple test procedures. Statistics 19, 3944. Brownstone, D. (1990): Bootstrapping improved estimators for linear regression models. Journal of Econometrics 44, 171187. Danilov, D. L. & J. R. Magnus (2004): On the harm that ignoring pretesting can cause. Journal of Econometrics 122, 2746. Dijkstra, T. K. & J. H. Veldkamp (1988): `Datadriven selection of regressors and the bootstrap'. Lecture Notes in Economics and Mathematical Systems 307, 1738. Dukić, V. M. & E. A Peña (2002): Estimation after model selection in a gaussian model. Manuscript. Freedman, D. A., Navidi, W. & S. C. Peters (1988): `On the impact of variable selection in fitting regression equations'. Lecture Notes in Economics and Mathematical Systems 307, 116. Hansen, P. R. (2003): Regression analysis with many specifications: a bootstrap method for robust inference. Working Paper, Department of Economics, Brown University. Hjort, N. L. & G. Claeskens (2003): Frequentist model average estimators. Journal of the American Statistical Association 98, 879899. Kabaila, P. (1995): The effect of model selection on confidence regions and prediction regions. Econometric Theory 11, 537549. Kapetanios, G. (2001): Incorporating lag order selection uncertainty in parameter inference for AR models. Economics Letters 72, 137144. Kilian, L. (1998): Accounting for lag order uncertainty in autoregressions: the endogenous lag order bootstrap algorithm. Journal of Time Series Analysis 19, 531548. Knight, K. (1999): Epiconvergence in distribution and stochastic equisemicontinuity. Working Paper, Department of Statistics, University of Toronto. Kulperger, R. J. & S. E. Ahmed (1992): A bootstrap theorem for a preliminary test estimator. Communications in Statistics: Theory and Methods 21, 20712082. Leeb, H. (2002): On a differential equation with advanced and retarded arguments. Communications on Applied Nonlinear Analysis 9, 7786. Leeb, H. (2003): The distribution of a linear predictor after model selection: unconditional finitesample distributions and asymptotic approximations. Working Paper, Department of Statistics, University of Vienna. Leeb, H. (2005): The distribution of a linear predictor after model selection: conditional finitesample distributions and asymptotic approximations. Journal of Statistical Planning and Inference 134, 6489. Leeb, H. & B. M. Pötscher (2002): Performance limits for estimators of the risk or distribution of shrinkagetype estimators, and some general lower risk bound results. Working Paper, Department of Statistics, University of Vienna. (Forthcoming in Econometric Theory.) Leeb, H. & B. M. Pötscher (2003): The finitesample distribution of postmodelselection estimators and uniform versus nonuniform approximations. Econometric Theory 19, 100142. Leeb, H. & B. M. Pötscher (2005a): Model Selection and Inference: Facts and Fiction. Econometric Theory 21, 2159. Leeb, H. & B. M. Pötscher (2005b): Can one estimate the conditional distribution of postmodelselection estimators? Working Paper, Department of Statistics, University of Vienna. Pötscher, B. M. (1991): Effects of model selection on inference. Econometric Theory 7, 163185. Pötscher, B. M. (1995): Comment on `The effect of model selection on confidence regions and prediction regions' by P. Kabaila. Econometric Theory 11, 550559. Pötscher, B. M. & A. J. Novak (1998): The distribution of estimators after model selection: large and small sample results. Journal of Statistical Computation and Simulation 60, 1956. Rao, C. R. & Y. Wu (2001): `On model selection,' IMS Lecture Notes Monograph Series 83, 157. Sen, P. K (1979): Asymptotic properties of maximum likelihood estimators based on conditional specification. Annals of Statistics 7, 10191033. Sen P. K & A. K. M. E. Saleh (1987): On preliminary test and shrinkage Mestimation in linear models. Annals of Statistics 15, 15801592. 
URI:  http://mpra.ub.unimuenchen.de/id/eprint/72 
Available Versions of this Item
 Can One Estimate the Unconditional Distribution of PostModelSelection Estimators ? (deposited 12. Oct 2006) [Currently Displayed]