Band Width Selection for High Dimensional Covariance Matrix Estimation

Qiu, Yumou and Chen, Song Xi (2014): Band Width Selection for High Dimensional Covariance Matrix Estimation. Forthcoming in: Journal of the American Statistical Association

Abstract

The banding estimator of Bickel and Levina (2008a) and its tapering version of Cai, Zhang and Zhou (2010), are important high dimensional covariance estimators. Both estimators require choosing a band width parameter. We propose a band width selector for the banding covariance estimator by minimizing an empirical estimate of the expected squared Frobenius norms of the estimation error matrix. The ratio consistency of the band width selector to the underlying band width is established. We also provide a lower bound for the coverage probability of the underlying band width being contained in an interval around the band width estimate. Extensions to the band width selection for the tapering estimator and threshold level selection for the thresholding covariance estimator are made. Numerical simulations and a case study on sonar spectrum data are conducted to confirm and demonstrate the proposed band width and threshold estimation approaches.

• Item Type: MPRA Paper
• Language: English
• Keywords: Bandable covariance; Banding estimator; Large $p$, small $n$; Ratio-consistency; Tapering estimator; Thresholding estimator.
• Subjects: C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General; C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C13 - Estimation: General; C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C14 - Semiparametric and Nonparametric Methods: General; C - Mathematical and Quantitative Methods > C5 - Econometric Modeling

References

Anderson, T. W. (2003). An Introduction to Multivariate Statistical Analysis. Wiley, New York.

Bai, Z.D., Silverstein, J.W. and Yin, Y.Q. (1998), A Note on the Largest Eigenvalue of a Large-dimensional Sample Covariance Matrix. Journal of Multivariate Analysis 26, 166-168.

Bai, Z.D. and Yin, Y.Q. (1993), Limit of the Smallest Eigenvalue of a Large Dimensional Sample Covariance Matrix. The Annals of Probability 21 1276-1294.

Bickel, P. J. and Levina, E. (2008a), Regularized Estimation of Large Covariance Matrices. The Annals of Statistics 36 199-227.

Bickel, P. and Levina, E. (2008b), Covariance Regularization by Thresholding. The Annals of Statistics 36 2577-2604.

Cai, T. T., Zhang, C.H. and Zhou, H. (2010), Optimal Rates of Convergence for Covariance Matrix Estimation. The Annals of Statistics 38 2118-2144.

Cai, T. T. and Liu, W. (2011), Adaptive Thresholding for Sparse Covariance Matrix Estimation. Journal of the American Statistical Association 494 672-684.

Cai, T. T., Liu, W. and Luo, X. (2011), A Constrained $\ell_{1}$ Minimization Approach to Sparse Precision Matrix Estimation. Journal of the American Statistical Association 494 594-607.

Cai, T. T., Liu, W. and Xia, Y. (2013), Two-Sample Covariance Matrix Testing and Support Recovery in High-Dimensional and Sparse Settings. Journal of the American Statistical Association 501 265-277.

Cai, T. and Yuan, M. (2012), Adaptive covariance matrix estimation through block thresholding. The Annals of Statistics 40 2014-2042.

Fan, J., Fan, Y. and Lv, J. (2008), High Dimensional Covariance Matrix Estimation Using a Factor Model. Journal of Econometrics 147 186-197.

Gorman, R. and Sejnowski, T. (1988a), Analysis of Hidden Units in a Layered Network Trained to Classify Sonar Targets. Neural Networks 1 75-89.

Gorman, R. and Sejnowski, T. (1988b), Learned Classification of Sonar Targets Using a Massively Parallel Network . IEEE Transactions on Acoustics, Speech and Signal Processing 36 1135-1140.

Hall, P. and Jin, J. (2010), Innovated Higher Criticism for Detecting Sparse Signals in Correlated Noise. The Annals of Statistics 38 1686-1732.

Huang, J., Liu, N., Pourahmadi, M., and Liu, L.} (2006), Covariance Matrix Selection and Estimation via Penalised Normal Likelihood. Biometrika 93 85-98.

Jing, B.Y., Shao, Q.M. and Wang, Q.Y. (2003), Self-normalized Cramer Type Large Deviations for Independent Random Variables. The Annals of Probability 31 2167-2215.

Johnstone, I. (2001), On the Distribution of the Largest Eigenvalue in Principal Components Analysis. \textit{The Annals of Statistics 29 295-327.

Levina, E., Rothman, A. and Zhu, J. (2008), Sparse Estimation of Large Covariance Matrices Via a Nested Lasso Penalty. The Annals of Applied Statistics 2 245-263.

Qiu, Y. and Chen, S. (2012), Test for Bandedness of High-dimensional Covariance Matrices and Bandwidth Estimation. The Annals of Statistics 40 1285-1314.

Rothman, A. J., Levina, E. and Zhu, J. (2009), Generalized Thresholding of Large Covariance Matrices. Journal of the American Statistical Association 104 177-186.

Rothman, A. J., Levina, E. and Zhu, J. (2010), A new approach to Cholesky-based covariance regularization in high dimensions. Biometrika 97 539-550.

van der Vaart, A.W.} (2000), Asymptotic Statistics. Cambridge University Press, Cambridge.

Wu, W. B. and Pourahmadi, M. (2003), Nonparametric Estimation of Large Covariance Matrices of Longitudinal Data. Biometrika 93, 831-844.

Xue, L. and Zou, H. (2012), Regularized Rank-based Estimation of High-dimensional Nonparanormal Graphical Models. The Annals of Statistics 40 2541-2571.

Xue, L. and Zou, H. (2013), Minimax Optimal Estimation of General Bandable Covariance Matrices. Journal of Multivariate Analysis 116 45-51.

Yi, F. and Zou, H. (2013), SURE-tuned Tapering Estimation of Large Covariance Matrices. Computational Statistics and Data Analysis 58 339-351.