Confidence Sets Based on Sparse Estimators Are Necessarily Large

Pötscher, Benedikt M. (2007): Confidence Sets Based on Sparse Estimators Are Necessarily Large. Unpublished.

Abstract

Confidence sets based on sparse estimators are shown to be large compared to more standard confidence sets, demonstrating that sparsity of an estimator comes at a substantial price in terms of the quality of the estimator. The results are set in a general parametric or semiparametric framework.

• Item Type: MPRA Paper
• Language: English
• Keywords: sparse estimator, consistent model selection, post-model-selection estimator, penalized maximum likelihood, confidence set, coverage probability
• Subjects: C - Mathematical and Quantitative Methods > C4 - Econometric and Statistical Methods: Special Topics > C44 - Statistical Decision Theory; Operations Research; C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods: General

References

Beran, R. (1992): The radial process for confidence sets. Probability in Banach spaces, 8 (Brunswick, ME, 1991), 479--496, Progress in Probability 30, Birkhäuser Boston, Boston, MA.

Bunea, F. (2004): Consistent covariate selection and post model selection inference in semiparametric regression. Annals of Statistics 32, 898-927.

Bunea, F. & I. W. McKeague (2005): Covariate selection for semiparametric hazard function regression models. Journal of Multivariate Analysis 92, 186-204.

Fan, J. & R. Li (2001): Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association 96, 1348-1360.

Fan, J. & R. Li (2002): Variable selection for Cox's proportional hazards model and frailty model. Annals of Statistics 30, 74-99.

Fan, J. & R. Li (2004): New estimation and model selection procedures for semiparametric modeling in longitudinal data analysis. Journal of the American Statistical Association 99, 710-723.

Kabaila, P. (1995): The effect of model selection on confidence regions and prediction regions. Econometric Theory 11, 537--549.

Kabaila, P. (1998): Valid confidence intervals in regression after variable selection. Econometric Theory 14, 463--482.

Kabaila, P. & H. Leeb (2006): On the large-sample minimal coverage probability of confidence intervals after model selection. Journal of the American Statistical Association 101, 619-629.

Kale, B. K. (1985): A note on the super efficient estimator. Journal of Statistical Planning and Inference 12, 259-263.

Leeb, H. & B. M. Pötscher (2005): Model selection and inference: facts and fiction. Econometric Theory 21, 21--59.

Leeb, H. & B. M. Pötscher (2008): Sparse estimators and the oracle property, or the return of Hodges' estimator. Journal of Econometrics 142, 201-211 .

Li, R. & H. Liang (2008): Variable selection in semiparametric regression modeling. Annals of Statistics 36, 261-286.

Pötscher, B. M. (1991): Effects of model selection on inference. Econometric Theory 7, 163--185.

Pötscher, B. M. (1995): Comment on `The effect of model selection on confidence regions and prediction regions'. Econometric Theory 11, 550--559.

Pötscher, B. M. & H. Leeb (2007): On the distribution of penalized maximum likelihood estimators: the LASSO, SCAD, and thresholding. Working Paper, Department of Statistics, University of Vienna.

Pötscher, B. M. & U. Schneider (2009): On the distribution of the adaptive LASSO estimator. Journal of Statistical Planning and Inference 139, 2775-2790.

Wang, H. & C. Leng (2007): Unified LASSO estimation via least squares approximation. Journal of the American Statistical Association 102, 1039-1048.

Wang, H., Li, G. & C. L. Tsai (2007): Regression coefficient and autoregressive order shrinkage and selection via the lasso. Journal of the Royal Statistical Society B 69, 63-78.

Wang, H., Li, R. & C. L. Tsai (2007): Tuning parameter selectors for the smoothly clipped absolute deviation method. Biometrika 94, 553-568.

Yang, Y. (2005): Can the strength of AIC and BIC be shared? A conflict between model identification and regression estimation. Biometrika 92, 937-950.

Zhang, H. H. & W. Lu (2007): Adaptive lasso for Cox's proportional hazards model. Biometrika 94, 691-703.

Zou, H. (2006): The adaptive lasso and its oracle properties. Journal of the American Statistical Association 101, 1418-1429.

Zou, H. & M. Yuan (2008): Composite quantile regression and the oracle model selection theory. Annals of Statistics 36, 1108-1126.