esposito, francesco paolo and cummins, mark (2015): Multiple hypothesis testing of market risk forecasting models. Forthcoming in: Journal of Forecasting

PDF
MPRA_paper_64986.pdf Download (503kB)  Preview 
Abstract
Extending previous risk model backtesting literature, we construct multiple hypothesis testing (MHT) with the stationary bootstrap. We conduct multiple tests which control for the generalized confidence level and employ the bootstrap MHT to design multiple comparison testing. We consider absolute and relative predictive ability to test a range of competing risk models, focusing on ValueatRisk (VaR) and Expected Shortfall (ExS). In devising the test for the absolute predictive ability, we take the route of recent literature and construct balanced simultaneous confidence sets that control for the generalized familywise error rate, which is the joint probability of rejecting true hypotheses. We implement a stepdown method which increases the power of the MHT in isolating false discoveries. In testing for the ExS model predictive ability, we design a new simple test to draw inference about recursive model forecasting capability. In the second suite of statistical testing, we develop a novel device for measuring the relative predictive ability in the bootstrap MHT framework. The device, we coin multiple comparison mapping, provides a statistically robust instrument designed to answer the question: ''which model is the best model?''.
Item Type:  MPRA Paper 

Original Title:  Multiple hypothesis testing of market risk forecasting models 
Language:  English 
Keywords:  valueatrisk, expected shortfall, bootstrap multiple hypothesis testing, generalized familywise error rate, multiple comparison map 
Subjects:  C  Mathematical and Quantitative Methods > C1  Econometric and Statistical Methods and Methodology: General > C12  Hypothesis Testing: General 
Item ID:  64986 
Depositing User:  Francesco Esposito 
Date Deposited:  11 Jun 2015 14:00 
Last Modified:  29 Sep 2019 06:40 
References:  Artzner, P., P.Delbaen, J.M. Eber, and D.Heath, 1999, Coherent measures of risk, Mathematical Finance 9. Bao, Y., T.Lee, and B.Saltoglu, 2006, Evaluating predictive performance of valueatrisk models in emerging markets: A reality check, Journal of Forecasting 25. Bao, Y., T.Lee, and B.Saltoglu, 2007, Comparing density forecast models,Journal of Forecasting 26. Beran, R., 1988, Balanced simultaneous confidence sets, Journal of the American Statistical Association 83. Beran, R., 1990, Refining bootstrap simultaneous confidence sets, Journal of the American Statistical Association 85. Beran, R., 2003, The impact of bootstrap on statistical algorithms and theory, Statistical Science 18. Berkowitz, J., 2001, Testing density forecasts with applications to risk management, Journal of Business and Economic Statistics . Bierens, H.J., 1987, Kernel estimators of regression functions, in Advances in Econometrics: Fifth World Congress, volume1 (Palgrave, London). BIS, 2011, Messages from the academic literature on risk measurement for the trading book, Technical Report19, Bank for International Settlements. Bollerslev, T., 1986, Generalized autoregressive conditional heteroskedasticity, Journal of Econometrics 3. Christoffersen, P., 1998, Evaluating interval forecasts, International Economic Review 39. Corradi, V., and N.R. Swanson, 2006, Bootstrap conditional distribution test in the presence of dynamic misspecification, Journal of Econometrics 133. Corradi, V., and N.R. Swanson, 2007, Nonparametric bootstrap procedures for predictive inference based on recursive estimation schemes, International Economic Review . Diebold, F.X., and R.S. Mariano, 1995, Comparing predictive accuracy, Journal of Business and Economic Statistics . Diebold, F.X., A.G. Todd, and A.S. Tay, 1998, Evaluating density forecasts with applications to financial risk management, International Economic Review . Efron, B., 1979, Bootstrap methods: Another look at the jackknife, The Annals of Statistics 7. Engle, R., and S.Manganelli, 2004, Caviar: Conditional autoregressive valueatrisk by regression quantile, Journal of Business and Economic Statistics 22. Engle, R.F., 1982, Autoregressive conditional heteroskedasticity with estimates of the variance of the united kingdom inflation, Econometrica 50. Glosten, L.R., R.Jagannathan, and D.E. Runkle, 1993, On the relation between the expected value and the volatility of the nominal excess return on stocks, The Journal of Finance 48. GonzalesRivera, G., L.TaeHwy, and M.Santosh, 2003, Forecasting volatility: A reality check based on option pricing, utility functions, valueatrisk and predictive likelihood, Working Paper . Hansen, P.R., 2001, An unbiased and powerful test for superior predictive ability, Working Paper . Hansen, P.R., and A.Lunde, 2001, A comparison of volatility models: Does anything beat a garch(1,1)?, Working Paper . Horowitz, J.L., 2000, The bootstrap, Working Paper . J.P.Morgan, 1996, Riskmetrics$^{TM}$, Technical Report4, J.P.Morgan, Reuters. Kerkhof, J., and B.Melenberg, 2003, Backtesting for riskbased regulatory capital, Working Paper . Kerkhof, J., B.Melenberg, and H.Schumacher, 2009, Model risk and capital reserves, Working Paper . Koenker, B., and G.Bassett, 1978, Regression quantiles, Econometrica 46. Kupiec, P., 1995, Techniques for verifying the accuracy of risk management models, Journal of Derivatives 3. Kusch, H.R., 1989, The jackknife and the bootstrap for general stationary observations, The Annals of Statistics 17. Lahiri, S.N., 1999, Theoretical comparisons of block bootstrap methods, The Annals of Statistics 27. Lehmann, E.L., and J.Romano, 2005, Generalization of the familiwise error rate, The Annals of Statistics 33. Liu, R.Y., and K.Singh, 1992, Moving blocks jackknife and bootstrap capture weak dependence, in Exploring the Limits of Bootstrap (John Wiley and Sons). Longin, F.M., 1996, The asymptotic distribution of extreme stock market returns, The Journal of Business 69. Mancini, L., and F.Trojani, 2011, Robust value at risk prediction, The Journal of Financial Econometrics 9. McNeil, A.J., and R.Frey, 2000, Estimation of tailrelated risk measures for heteroskedastic financial time series: an extreme value approach, Working Paper . Nadaraya, E.A., 1964, On estimating regression, Theory of Probability and its Applications 9. Nelson, D.B., 1991, Conditional heteroskedasticity in asset returns: a new approach, Econometrica 59. Nordman, D.J., 2009, A note on the stationary bootstrap's variance, The Annals of Statistics 37. Patton, A., D.N. Politis, and H.White, 2009, Correction to: Automatic blocklength selection for dependent bootstrap, Econometric Reviews 28. Politis, D.N., and J.P. Romano, 1994a, Limit theorems for weakly dependent hilbert space valued random variables with applications to the stationary bootstrap, Statistica Sinica 4. Politis, D.N., and J.P. Romano, 1994b, The stationary bootstrap, Journal of the American Statistical Association 89. Politis, D.N., and H.White, 2004, Automatic blocklength selection for dependent bootstrap, Econometric Reviews 23. Romano, J., M.S. Azeem, and M.Wolf, 2009, Hypothesis testing in econometrics, Working Paper Series, Institute for Empirical Research, University of Zurich 444. Romano, J., and M.Wolf, 2005, Exact and approximate stepdown methods for multiple hypothesis testing, Journal of the American Statistical Association 100. Romano, J., and M.Wolf, 2007, Control of generalized error rates in multiple testing, The Annals of Statistics 35. Romano, J., and M.Wolf, 2010, Balanced control of generalized error rates, The Annals of Statistics 38. Rosenblatt, M., 1952, Remarks on a multivariate transformation, The Annals of Mathmatical Statistics 23. Vander Vaart, A.W., 1998, Functional delta method, in Asympotic Statistics, chapter20 (Cambridge University Press). Watson, G.S., 1964, Smooth regression analysis, Sankya: The Indian Journal of Statistics 26. West, K.D., 1996, Asymptotic inference about predictive ability, Econometrica 64. White, H., 2000, A reality check for data snooping, Econometrica 68. 
URI:  https://mpra.ub.unimuenchen.de/id/eprint/64986 