Pötscher, Benedikt M. and Schneider, Ulrike (2011): Distributional results for thresholding estimators in highdimensional Gaussian regression models.
This is the latest version of this item.

PDF
MPRA_paper_34706.pdf Download (770kB)  Preview 
Abstract
We study the distribution of hard, soft, and adaptive softthresholding estimators within a linear regression model where the number of parameters k can depend on sample size n and may diverge with n. In addition to the case of known errorvariance, we define and study versions of the estimators when the errorvariance is unknown. We derive the finitesample distribution of each estimator and study its behavior in the largesample limit, also investigating the effects of having to estimate the variance when the degrees of freedom nk does not tend to infinity or tends to infinity very slowly. Our analysis encompasses both the case where the estimators are tuned to perform consistent variable selection and the case where the estimators are tuned to perform conservative variable selection. Furthermore, we discuss consistency, uniform consistency and derive the minimax rate under either type of tuning.
Item Type:  MPRA Paper 

Original Title:  Distributional results for thresholding estimators in highdimensional Gaussian regression models 
Language:  English 
Keywords:  Thresholding, Lasso, adaptive Lasso, penalized maximum likelihood, variable selection, finitesample distribution, asymptotic distribution, variance estimation, minimax rate, highdimensional model, oracle property 
Subjects:  C  Mathematical and Quantitative Methods > C1  Econometric and Statistical Methods and Methodology: General > C13  Estimation: General C  Mathematical and Quantitative Methods > C2  Single Equation Models; Single Variables > C20  General C  Mathematical and Quantitative Methods > C5  Econometric Modeling > C52  Model Evaluation, Validation, and Selection 
Item ID:  34706 
Depositing User:  Benedikt Poetscher 
Date Deposited:  21. Nov 2011 17:15 
Last Modified:  16. Feb 2013 05:19 
References:  Alliney, S. & S. A. Ruzinsky (1994): An algorithm for the minimization of mixed l₁ and l₂ norms with applications to Bayesian estimation. IEEE Transactions on Signal Processing 42, 618627. Bauer, P., Pötscher, B. M. & P. Hackl (1988): Model selection by multiple test procedures. Statistics 19, 3944. Donoho, D. L., Johnstone, I. M., Kerkyacharian, G., D. Picard (1995): Wavelet shrinkage: asymptopia? With discussion and a reply by the authors. Journal of the Royal Statistical Society Series B 57, 301369. Fan, J. & R. Li (2001): Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association 96, 13481360. Fan, J. & H. Peng (2004): Nonconcave penalized likelihood with a diverging number of parameters. Annals of Statistics 32, 928961. Feller, W. (1957): An Introduction to Probability Theory and Its Applications, Volume 1. 2nd ed., Wiley, New York. Frank, I. E. & J. H. Friedman (1993): A statistical view of some chemometrics regression tools (with discussion). Technometrics 35, 109148. Ibragimov, I. A. (1956): On the composition of unimodal distributions. Theory of Probability and its Applications 1, 255260. Knight, K. & W. Fu (2000): Asymptotics for lassotype estimators. Annals of Statistics 28, 13561378. Leeb, H. & B. M. Pötscher (2003): The finitesample distribution of postmodelselection estimators and uniform versus nonuniform approximations. Econometric Theory 19, 100142. Leeb, H. & B. M. Pötscher (2005): Model selection and inference: facts and fiction. Econometric Theory 21, 2159. Leeb, H. & B. M. Pötscher (2008): Sparse estimators and the oracle property, or the return of Hodges' estimator. Journal of Econometrics 142, 201211. Pötscher, B. M. (1991): Effects of model selection on inference. Econometric Theory 7, 163185. Pötscher, B. M. (2006): The distribution of model averaging estimators and an impossibility result regarding its estimation. IMS Lecture NotesMonograph Series 52, 113129. Pötscher, B. M. & H. Leeb (2009): On the distribution of penalized maximum likelihood estimators: the LASSO, SCAD, and thresholding. Journal of Multivariate Analysis 100, 20652082. Pötscher, B. M. & U. Schneider (2009): On the distribution of the adaptive LASSO estimator. Journal of Statistical Planning and Inference 139, 27752790. Pötscher, B. M. & U. Schneider (2010): Confidence sets based on penalized maximum likelihood estimators in Gaussian regression. Electronic Journal of Statistics 10, 334360. Sen, P. K. (1979): Asymptotic properties of maximum likelihood estimators based on conditional specification. Annals of Statistics 7, 10191033. Tibshirani, R. (1996): Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society Series B 58, 267288. Zhang, C.H. (2010): Nearly unbiased variable selection under minimax concave penalty. Annals of Statistics 38, 894942. Zou, H. (2006): The adaptive lasso and its oracle properties. Journal of the American Statistical Association 101, 14181429. 
URI:  http://mpra.ub.unimuenchen.de/id/eprint/34706 
Available Versions of this Item

Distributional results for thresholding estimators in highdimensional Gaussian regression models. (deposited 28. Jun 2011 13:40)

Distributional results for thresholding estimators in highdimensional Gaussian regression models. (deposited 08. Nov 2011 20:13)
 Distributional results for thresholding estimators in highdimensional Gaussian regression models. (deposited 21. Nov 2011 17:15) [Currently Displayed]

Distributional results for thresholding estimators in highdimensional Gaussian regression models. (deposited 08. Nov 2011 20:13)