Statistical inference and feasibility determination: a nonasymptotic approach

Zhu, Ying (2018): Statistical inference and feasibility determination: a nonasymptotic approach.

Abstract

We develop non-asymptotically justified methods for hypothesis testing about the p-dimensional coefficients in (possibly nonlinear) regression models, where the hypotheses can also be nonlinear in the coefficients. Our (nonasymptotic) control on the Type I and Type II errors holds for fixed n and does not rely on well-behaved estimation error or prediction error; in particular, when the number of restrictions in the null hypothesis is large relative to p-n, we show it is possible to bypass the sparsity assumption on the coefficients (for both Type I and Type II error control), regularization on the estimates of the coefficients, and other inherent challenges in an inverse problem. We also demonstrate an interesting link between our framework and Farkas' lemma (in math programming) under uncertainty, which points to some potential applications of our method outside traditional hypothesis testing.

• Item Type: MPRA Paper
• Language: English
• Keywords: Nonasymptotic validity; hypothesis testing; confidence regions; concentration inequalities; high dimensional regressions; Farkas' lemma
• Subjects: C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General; C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C12 - Hypothesis Testing: General; C - Mathematical and Quantitative Methods > C2 - Single Equation Models ; Single Variables; C - Mathematical and Quantitative Methods > C2 - Single Equation Models ; Single Variables > C21 - Cross-Sectional Models ; Spatial Models ; Treatment Effect Models ; Quantile Regressions

References

Arlot, S., G. Blanchard, and E. Roquain (2010). “Some Nonasymptotic Results on Resampling in High Dimension, I: Confidence Regions.” Annals of Statistics, 38, 51-82.

Bertsimas, D. and J. Tsitsiklis (1997). Introduction to Linear Optimization, Athena Scientific.

Ben-Tal, A. and A. Nemirovski (2000). “Robust Solutions of Linear Programming Problems Contaminated with Uncertain Data.” Mathematical Programming, 88, 411-424.

Bertsimas, D. and M. Sim (2004). “The Price of Robustness.” Operations Research, 52, 35-53.

Bickel, P., J. Y. Ritov, and A. B. Tsybakov (2009). “Simultaneous Analysis of Lasso and Dantzig Selector.” Annals of Statistics, 37, 1705-1732.

Bobkov, S. G. and M. Ledoux (2000). “From Brunn-Minkowski to Brascamp-Lieb and to Logarithmic Sobolev Inequalities.” Geometric and Functional Analysis. 10, 1028-1052.

Boucheron, S, G. Lugosi, and P. Massart (2013). Concentration Inequalities: A Nonasymptotic Theory of Independence. Oxford University Press. Oxford.

Chernozhukov, V., D. Chetverikov, and K. Kato (2013). “Gaussian Approximations and Multiplier Bootstrap for Maxima of Sums of High-Dimensional Random Vectors.” Annals of Statistics, 41, 2786-2819.

Dezeure, R., P. Bühlmann, and C.-H. Zhang (2017). “High-Dimensional Simultaneous Inference with the Bootstrap.” Test, 26, 685-719.

Gautier, E. and A. B. Tsybakov (2011). “High-Dimensional Instrumental Variables Regression and Confidence Sets.” Manuscript. CREST (ENSAE).

Horowitz, J. L. (2017). “Non-Asymptotic Inference in Instrumental Variables Estimation.” Manuscript. Northwestern University.

Javanmard, A. and A. Montanari (2014). “Confidence Intervals and Hypothesis Testing for High- Dimensional Regression.” Journal of Machine Learning Research, 15, 2869-2909.

Ledoux, M., and M. Talagrand (1991). Probability in Banach Spaces: Isoperimetry and Processes. Springer-Verlag, New York, NY.

Maurey, B. (1991). “Some Deviation Inequalities.” Geometric and Functional Analysis. 1, 188-197.

Ning, Y and H. Liu (2017). “A General Theory of Hypothesis Tests and Confidence Regions for Sparse High Dimensional Models.” Annals of Statistics, 45, 158-195.

Papke, L. E. and J. M. Wooldridge (2008). “Panel Data Methods for Fractional Response Variables with an Application to Test Pass Rates.” Journal of Econometrics, 145, 121-133.

Saumard, A. and J. A. Wellner (2014). “Log-Concavity and Strong Log-Concavity: A Review.” Statistics Surveys, 8, 45-114.

Soyster, A. L. (1973). “Convex Programming with Set-Inclusive Constraints and Applications to Inexact Linear Programming.” Operations Research, 21, 1154-1157.

van de Geer, S., P. Bühlmann, Y. Ritov, and R. Dezeure (2014). “On Asymptotically Optimal Confidence Regions and Tests for High-Dimensional Models.” Annals of Statistics, 42, 1166-1202.

Wainwright, M. J. (2015). High-Dimensional Statistics: A Non-Asymptotic Viewpoint. University of California, Berkeley.

Wooldridge, J. M. (2010). Econometric Analysis of Cross Section and Panel Data. MIT Press, Cambridge.

Wooldridge, J. M. and Y. Zhu (2017). “Inference in Approximately Sparse Correlated Random Effects Probit Models.” Forthcoming in Journal of Business and Economic Statistics.

Zhang C.-H. and S. S. Zhang (2014). “Confidence Intervals for Low Dimensional Parameters in High Dimensional Linear Models.” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 76, 217-242.

Zhang, X. and Cheng, G. (2017). “Simultaneous Inference for High-Dimensional Linear Models.” Journal of the American Statistical Association - Theory & Methods, 112, 757-768.

Zhu, Y. and J. Bradic (2017). “Breaking the Curse of Dimensionality in Regression.” arXiv:1708.00430.

Zhu, Y. and J. Bradic (2018). “Linear Hypothesis Testing in Dense High-Dimensional Linear Models.” Journal of the American Statistical Association - Theory & Methods, 113, 1583-1600.