Zhu, Ying (2013): Sparse Linear Models and TwoStage Estimation in HighDimensional Settings with Possibly Many Endogenous Regressors.
There is a more recent version of this item available. 

PDF
MPRA_paper_49846.pdf Download (953kB)  Preview 
Abstract
This paper explores the validity of the twostage estimation procedure for sparse linear models in highdimensional settings with possibly many endogenous regressors. In particular, the number of endogenous regressors in the main equation and the instruments in the firststage equations can grow with and exceed the sample size n. The analysis concerns the exact sparsity case, i.e., the maximum number of nonzero components in the vectors of parameters in the firststage equations, k1, and the number of nonzero components in the vector of parameters in the secondstage equation, k2, are allowed to grow with n but slowly compared to n. I consider the highdimensional version of the twostage least square estimator where one obtains the fitted regressors from the firststage regression by a least square estimator with l_1regularization (the Lasso or Dantzig selector) when the firststage regression concerns a large number of instruments relative to n, and then construct a similar estimator using these fitted regressors in the secondstage regression. The main theoretical results of this paper are nonasymptotic bounds from which I establish sufficient scaling conditions on the sample size for estimation consistency in l_2norm and variableselection consistency (i.e., the twostage highdimensional estimators correctly select the nonzero coefficients in the main equation with high probability). A technical issue regarding the socalled "restricted eigenvalue (RE) condition" for estimation consistency and the "mutual incoherence (MI) condition" for selection consistency arises in the twostage estimation from allowing the number of regressors in the main equation to exceed n and this paper provides analysis to verify these RE and MI conditions. Depending on the underlying assumptions that are imposed, the upper bounds on the l_2error and the sample size required to obtain these consistency results differ by factors involving k1 and/or k2. Simulations are conducted to gain insight on the finite sample performance of the highdimensional twostage estimator.
Item Type:  MPRA Paper 

Original Title:  Sparse Linear Models and TwoStage Estimation in HighDimensional Settings with Possibly Many Endogenous Regressors 
Language:  English 
Keywords:  Highdimensional statistics; Lasso; sparse linear models; endogeneity; twostage estimation 
Subjects:  C  Mathematical and Quantitative Methods > C1  Econometric and Statistical Methods and Methodology: General C  Mathematical and Quantitative Methods > C1  Econometric and Statistical Methods and Methodology: General > C13  Estimation: General C  Mathematical and Quantitative Methods > C3  Multiple or Simultaneous Equation Models ; Multiple Variables > C31  CrossSectional Models ; Spatial Models ; Treatment Effect Models ; Quantile Regressions ; Social Interaction Models C  Mathematical and Quantitative Methods > C3  Multiple or Simultaneous Equation Models ; Multiple Variables > C36  Instrumental Variables (IV) Estimation 
Item ID:  49846 
Depositing User:  Ms Ying Zhu 
Date Deposited:  18. Sep 2013 12:45 
Last Modified:  18. Sep 2013 13:39 
References:  [1] Ackerberg, D. A., and G. S. Crawford (2009). "Estimating Price Elasticities in Differentiated Product Demand Models with Endogenous Characteristics". Working Paper. [2] Amemiya, T. (1974). "The NonLinear TwoStage Least Squares Estimator". Journal of Econometrics, 2, 105110. [3] Angrist, J. D., and A. B. Krueger (1991). "Does Compulsory School Attendance Affect Schooling and Earnings?" Quarterly Journal of Economics, 106, 9791014. [4] Benkard, C. L., and P. Bajari (2005). "Hedonic Price Indexes with Unobserved Product Characteristics, and Application to Personal Computers". Journal of Business and Economic Statistics, 23, 6175. [5] Belloni, A., V. Chernozhukov, and L. Wang (2011). "SquareRoot Lasso: Pivotal Recovery of Sparse Signals Via Conic Programming". Biometrika, 98(4): 791806. [6] Belloni, A., and V. Chernozhukov (2011a). "L1Penalized Quantile Regression in HighDimensional Sparse Models". The Annals of Statistics, 39, 82130. [7] Belloni, A., and V. Chernozhukov (2011b). "High Dimensional Sparse Econometric Models: an Introduction", in: Inverse Problems and High Dimensional Estimation, Stats in the Château 2009, Alquier, P., E. Gautier, and G. Stoltz, Eds., Lecture Notes in Statistics, 203, 127162, Springer, Berlin. [8] Belloni, A., D. Chen, V. Chernozhukov, and C. Hansen (2012). "Sparse Models and Methods for Instrumental Regression, with an Application to Eminent Domain". Econometrica, 80, 23692429. [9] Belloni, A., V. Chernozhukov, and C. Hansen (2012). "Inference on Treatment Effects after Selection amongst HighDimensional Controls". Working Paper. cemmap. [10] Belloni, A., and V. Chernozhukov (2013). "Least Squares after Model Selection in HighDimensional Sparse Models". Bernoulli, 19, 521547. [11] Berry, S. T., J. A. Levinsohn, and A. Pakes (1995). "Automobile Prices in Market Equilibrium". Econometrica, 63, 841890. [12] Bickel, P., J. Y. Ritov, and A. B. Tsybakov (2009). "Simultaneous Analysis of Lasso and Dantzig Selector". The Annals of Statistics, 37, 17051732. [13] Bühlmann, P., and S. A. van de Geer (2011). Statistics for HighDimensional Data. Springer, NewYork. [14] Caner, M. (2009). "LASSO Type GMM Estimator". Econometric Theory, 25, 123. [15] Candès, E., and T. Tao (2007). "The Dantzig Selector: Statistical Estimation when p is Much Larger Than n". The Annals of Statistics, 35, 23132351. [16] Carrasco, M., and J. P. Florens (2000). "Generalization of GMM to a Continuum of Moment Conditions". Econometric Theory, 16, 797834. [17] Carrasco, M. (2012). "A Regularization Approach to the Many Instruments Problem". Journal of Econometrics, 170, 383398. [18] Dalalyan, A., and A. B. Tsybakov (2008). "Aggregation by Exponential Weighting, Sharp PACBayesian Bounds and Sparsity". Journal of Machine Learning Research, 72, 3961. [19] Donoho, D. L., M. Elad, and V. N. Temlyakov (2006). "Stable Recovery of Sparse Overcomplete Representations in the Presence of Noise". IEEE Transactions on Information Theory, 52, 618. [20] Fan, J., and R. Li (2001). "Variable Selection via Nonconcave Penalized Likelihood and Its Oracle Properties". Journal of American Statistical Association, 96, 13481360. [21] Fan, J., and Y. Liao (2011). "Ultra High Dimensional Variable Selection with Endogenous Covariates". Manuscript. Princeton University. [22] Fan, J., and J. Lv (2010). "A Selective Overview of Variable Selection in High Dimensional Feature Space". Statistica Sinica, 20, 101148. [23] Fan, J., and J. Lv (2011). "NonConcave Penalized Likelihood with NPDimensionality". IEEE Transactions on Information Theory, 57, 54675484. [24] Fan, J., J. Lv, and L, Qi (2011). "Sparse High Dimensional Models in Economics". Annual Review of Economics, 3, 291317. [25] Garen, J. (1984), "The Returns to Schooling: A Selectivity Bias Approach with a Continuous Choice Variable". Econometrica, 52, 11991218. [26] Gautier, E., and A. B. Tsybakov (2011). "Highdimensional Instrumental Variables Regression and Confidence Sets". Manuscript. CREST (ENSAE). [27] Hansen, C., J. Hausman, and W. K. Newey (2008). "Estimation with Many Instrumental Variables". Journal of Business and Economic Statistics, 26, 398422. [28] Koltchinskii, V. (2009). "The Dantzig Selector and Sparsity Oracle Inequalities". Bernoulli, 15, 799828. [29] Koltchinskii, V. (2011). "Oracle Inequalities in Empirical Risk Minimization and Sparse Recovery Problems". Forthcoming in Lecture Notes in Mathematics, Springer, Berlin. [30] Ledoux, M. (2001). The concentration of measure phenomenon. Mathematical Surveys and Monographs. American Mathematical Society, Providence, RI. [31] Ledoux, M., and M. Talagrand (1991). Probability in Banach Spaces: Isoperimetry and Processes. SpringerVerlag, New York, NY. [32] Lin, Y., and H. H. Zhang (2006). "Component Selection and Smoothing in Multivariate Nonparametric Regression". The Annals of Statistics, 34(5): 22722297. [33] Loh, P., and M. Wainwright (2012). "HighDimensional Regression with Noisy and Missing data: Provable Guarantees with Nonconvexity". Annals of Statistics, 40(3): 16371664. [34] Lounici, K. (2008). "SupNorm Convergence Rate and Sign Concentration Property of the Lasso and Dantzig Selector". Electronic Journal of Statistics, 2, 90102. [35] Manresa, E. (2013). "Recovery of Networks using Panel Data". Working paper. CEMFI. [36] Meinshausen, N., and B. Yu (2009). "Lassotype Recovery of Sparse Representations for Highdimensional Data". The Annals of Statistics, 37(1): 246270. [37] Negahban, S., P. Ravikumar, M. J. Wainwright, and B. Yu (2012). "A Unified Framework for HighDimensional Analysis of MEstimators with Decomposable Regularizers". Statistical Science, 27, 538557. [38] Nevo, A. (2001). "Measuring Market Power in the ReadytoEat Cereal Industry". Econometrica, 69, 307342. [39] Ravikumar, P., H. Liu, J. Lafferty, and L. Wasserman (2009). "Sparse Additive Models". Journal of the Royal Statistical Society, Series B, 71, 10091030. [40] Ravikumar, P., M. J. Wainwright, and J. Lafferty (2010). "Highdimensional Ising Model Selection Using l1Regularized Logistic Regression". The Annals of Statistics, 38(3): 12871319. [41] Raskutti, G., M. J. Wainwright, and B. Yu (2010). "Restricted Eigenvalue Conditions for Correlated Gaussian Designs". Journal of Machine Learning Research, 11: 22412259. [42] Raskutti, G., M. J. Wainwright, and B. Yu (2011). "Minimax Rates of Estimation for Highdimensional Linear Regression over lqBalls". IEEE Trans. Information Theory, 57(10): 69766994. [43] Rigollet, P., and A. B. Tsybakov (2011). "Exponential Screening and Optimal Rates of Sparse Estimation". The Annals of Statistics, 35, 731771. [44] Rosenbaum, M., and A. B. Tsybakov (2010). "Sparse Recovery Under Matrix Uncertainty". The Annals of Statistics, 38, 26202651. [45] Rosenbaum, M., and A. B. Tsybakov (2013). "Improved Matrix Uncertainty Selector", in: From Probability to Statistics and Back: HighDimensional Models and Processes  A Festschrift in Honor of Jon A. Wellner, Banerjee, M. et al. Eds, IMS Collections, 9, 276290, Institute of Mathematical Statistics. [46] M. Rudelson, and S. Zhou (2011). "Reconstruction from Anisotropic Random Measurements". Technical report, University of Michigan. [47] SalaiMartin, X. (1997). "I Just Ran Two Million Regressions". The American Economic Review, 87, 178183. [48] Tibshirani, R. (1996). "Regression Shrinkage and Selection via the Lasso". Journal of the Royal Statistical Society, Series B, 58(1): 267288. [49] Vershynin, R. (2012). "Introduction to the NonAsymptotic Analysis of Random Matrices", in Eldar, Y. and G. Kutyniok, Eds, Compressed Sensing: Theory and Applications, 210268, Cambridge. [50] Wainwright, J. M. (2009). "Sharp Thresholds for Highdimensional and Noisy Sparsity Recovery Using l1 Constrained Quadratic Programming (Lasso)". IEEE Trans. Information Theory, 55: 21832202. [51] Wainwright, J. M. (2014). HighDimensional Statistics: A NonAsymptotic Viewpoint. In preparation. University of California, Berkeley. [52] Wooldridge, J. M. (2002). Econometric Analysis of Cross Section and Panel Data. MIT Press, Cambridge. [53] Ye, F., and C.H. Zhang (2010). "Rate Minimaxity of the Lasso and Dantzig Selector for the lq Loss in lr Balls". Journal of Machine Learning Research, 11, 35193540. [54] Zhao, P., and Yu, B. (2007). On model selection consistency of Lasso. Journal of Machine Learning Research, 7, 25412567. 
URI:  https://mpra.ub.unimuenchen.de/id/eprint/49846 
Available Versions of this Item
 Sparse Linear Models and TwoStage Estimation in HighDimensional Settings with Possibly Many Endogenous Regressors. (deposited 18. Sep 2013 12:45) [Currently Displayed]