Testing in the Presence of Nuisance Parameters: Some Comments on Tests Post-Model-Selection and Random Critical Values

Leeb, Hannes and Pötscher, Benedikt M. (2013): Testing in the Presence of Nuisance Parameters: Some Comments on Tests Post-Model-Selection and Random Critical Values.

Abstract

We point out that the ideas underlying some test procedures recently proposed for testing post-model-selection (and for some other test problems) in the econometrics literature have been around for quite some time in the statistics literature. We also sharpen some of these results in the statistics literature. Furthermore, we show that some intuitively appealing testing procedures, that have found their way into the econometrics literature, lead to tests that do not have desirable size properties, not even asymptotically.

• Item Type: MPRA Paper
• Language: English
• Keywords: Tests in presence of nuisance parameters; inference post model selection
• Subjects: C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C52 - Model Evaluation, Validation, and Selection; C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C12 - Hypothesis Testing: General; C - Mathematical and Quantitative Methods > C0 - General > C01 - Econometrics

References

Andrews, D. W. K. & P. Guggenberger (2009): Hybrid and Size-Corrected Subsampling Methods. Econometrica 77, 721-762.

Bickel, P. J. & K. A. Doksum (1977): Mathematical Statistics: Basic Ideas and Selected Topics. Holden-Day, Oakland.

Berger, R. L. & D. D. Boos (1994): P Values Maximized Over a Confidence Set for the Nuisance Parameter. Journal of the American Statistical Association 89, 1012-1016.

DiTraglia, F. J. (2011): Using Invalid Instruments on Purpose: Focused Moment Selection and Averaging for GMM. Working Paper, Version November 9, 2011.

Kabaila, P. & H. Leeb (2006): On the Large-Sample Minimal Coverage Probability of Confidence Intervals after Model Selection. Journal of the American Statistical Association 101, 619-629.

Leeb, H. & B. M. Pötscher (2003): The Finite-Sample Distribution of Post-Model-Selection Estimators and Uniform Versus Non-Uniform Approximations. Econometric Theory 19, 100-142.

Leeb, H. & B. M. Pötscher (2005): Model Selection and Inference: Facts and Fiction. Econometric Theory 21, 29-59.

Loh, W.-Y. (1985): A New Method for Testing Separate Families of Hypotheses. Journal of the American Statistical Association 80, 362-368.

Liu, C.-A. (2011): A Plug-In Averaging Estimator for Regressions with Heteroskedastic Errors, Working Paper, Version October 29, 2011.

McCloskey, A. (2011): Powerful Procedures with Correct Size for Test Statistics with Limit Distributions that are Discontinuous in Some Parameters. Working Paper, Version October 2011.

McCloskey, A. (2012): Bonferroni-based Size Correction for Nonstandard Testing Problems. Working Paper, Brown University.

Romano, J. P. & A. Shaikh, M. Wolf (2014): A Practical Two-Step Method for Testing Moment Inequalities. Working Paper, University of Zurich.

Silvapulle, M. J. (1996): A Test in the Presence of Nuisance Parameters. Journal of the American Statistical Association 91, 1690-1693. (Correction, ibidem 92 (1997), 801.)

Williams, D. A. (1970): Discrimination Between Regression Models to Determine the Pattern of Enzyme Synthesis in Synchronous Cell Cultures. Biometrics 26, 23-32.