Pötscher, Benedikt M. and Leeb, Hannes (2007): On the distribution of penalized maximum likelihood estimators: The LASSO, SCAD, and thresholding.
There is a more recent version of this item available. 

PDF
MPRA_paper_5615.pdf Download (355kB)  Preview 
Abstract
We study the distributions of the LASSO, SCAD, and thresholding estimators, in finite samples and in the largesample limit. The asymptotic distributions are derived for both the case where the estimators are tuned to perform consistent model selection and for the case where the estimators are tuned to perform conservative model selection. Our findings complement those of Knight and Fu (2000) and Fan and Li (2001). We show that the distributions are typically highly nonnormal regardless of how the estimator is tuned, and that this property persists in large samples. An impossibility result regarding estimation of the estimators' distribution function is also provided.
Item Type:  MPRA Paper 

Institution:  University of Vienna 
Original Title:  On the distribution of penalized maximum likelihood estimators: The LASSO, SCAD, and thresholding. 
Language:  English 
Keywords:  Penalized maximum likelihood; LASSO; SCAD; thresholding; postmodelselection estimator; finitesample distribution; asymptotic distribution; estimation of distribution; uniform consistency 
Subjects:  C  Mathematical and Quantitative Methods > C1  Econometric and Statistical Methods and Methodology: General > C13  Estimation: General C  Mathematical and Quantitative Methods > C2  Single Equation Models ; Single Variables 
Item ID:  5615 
Depositing User:  Benedikt Poetscher 
Date Deposited:  06 Nov 2007 
Last Modified:  27 Sep 2019 00:23 
References:  Bauer, P., Pötscher, B.M. & P. Hackl (1988): Model selection by multiple test procedures. Statistics 19, 3944. Beran, R. (1997): Diagnosing bootstrap success. Annals of the Institute of Statistical Mathematics 49, 124. Bruce, A.G. & H. Gao (1996): Understanding WaveShrink: Variance and bias estimation. Biometrika 83, 727745. Efron, B., Hastie, T., Johnstone, I. & R. Tibshirani (2004): Least angle regression. Annals of Statistics 32, 407499. Fan, J. & R. Li (2001): Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association} 96, 13481360. Frank, I.E. & J.H. Friedman (1993): A statistical view of some chemometrics regression tools (with discussion). Technometrics 35, 109148. Kabaila, P. (1995): The effect of model selection on confidence regions and prediction regions. Econometric Theory 11, 537549. Knight, K. & W. Fu (2000): Asymptotics of lassotype estimators. Annals of Statistics 28, 13561378. Knight, K. (2008): Shrinkage estimation for nearlysingular designs. Econometric Theory 24, forthcoming. Kulperger, R.J. & S.E. Ahmed (1992): A bootstrap theorem for a preliminary test estimator. Communications in Statistics: Theory and Methods 21, 20712082. Judge, G.G. & M.E. Bock (1978): The Statistical Implications of Pretest and SteinRule Estimators in Econometrics. NorthHolland. Leeb, H. & B.M. Pötscher (2003): The finitesample distribution of postmodelselection estimators and uniform versus nonuniform approximations. Econometric Theory 19, 100142. Leeb, H. & B.M. Pötscher (2005): Model selection and inference: Facts and fiction. Econometric Theory 21, 2159. Leeb, H. & B.M. Pötscher (2006a): Performance limits for estimators of the risk or distribution of shrinkagetype estimators, and some general lower riskbound results. Econometric Theory 22, 2159. (Corrections: ibidem, forthcoming). Leeb, H. & B.M. Pötscher (2006b): Can one estimate the conditional distribution of postmodelselection estimators? Annals of Statistics 34, 25542591. Leeb, H. & B.M. Pötscher (2007): Sparse estimators and the oracle property, or the return of Hodges' estimator. Journal of Econometrics, doi:10.1016/j.jeconom.2007.05.017. Leeb, H. & B.M. Pötscher (2008): Can one estimate the unconditional distribution of postmodelselection estimators? Econometric Theory 24, forthcoming. Lehmann, E.L. & G. Casella (1998): Theory of Point Estimation. Springer Texts in Statistics. New York: SpringerVerlag. Pötscher, B.M. (1991): Effects of model selection on inference. Econometric Theory 7, 163185. Pötscher, B.M. (2006): The distribution of model averaging estimators and an impossibility result regarding its estimation. IMS Lecture NotesMonograph Series 52, 113129. Samworth, R. (2003): A note on methods of restoring consistency of the bootstrap. Biometrika 90, 985990. Sen, P.K. (1979): Asymptotic properties of maximum likelihood estimators based on conditional specification. Annals of Statistics 7, 10191033. Tibshirani, R. (1996): Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society Series B 58, 267288. Zhao, P. & B. Yu (2006): On model selection consistency of lasso. Journal of Machine Learning Research 7, 25412563. Zou, H. (2006): The adaptive lasso and its oracle properties. Journal of the American Statistical Association 101, 14181429. 
URI:  https://mpra.ub.unimuenchen.de/id/eprint/5615 
Available Versions of this Item
 On the distribution of penalized maximum likelihood estimators: The LASSO, SCAD, and thresholding. (deposited 06 Nov 2007) [Currently Displayed]