Bank Net Interest Margin Forecasting and Capital Adequacy Stress Testing by Machine Learning Techniques

Brummelhuis, Raymond and Luo, Zhongmin (2019): Bank Net Interest Margin Forecasting and Capital Adequacy Stress Testing by Machine Learning Techniques.

Abstract

The 2007-09 financial crisis revealed that the investors in the financial market were more concerned about the future as opposed to the current capital adequacy for banks. Stress testing promises to complement the regulatory capital adequacy regimes, which assess a bank's current capital adequacy, with the ability to assess its future capital adequacy based on the projected asset-losses and incomes from the forecasting models from regulators and banks. The effectiveness of stress-test rests on its ability to inform the financial market, which depends on whether or not the market has confidence in the model-projected asset-losses and incomes for banks. Post-crisis studies found that the stress-test results are uninformative and receive insignificant market reactions; others question its validity on the grounds of the poor forecast accuracy using linear regression models which forecast the banking-industry incomes measured by Aggregate Net Interest Margin. Instead, our study focuses on NIM forecasting at an individual bank's level and employs both linear regression and non-linear Machine Learning techniques. First, we present both the linear and non-linear Machine Learning regression techniques used in our study. Then, based on out-of-sample tests and literature-recommended forecasting techniques, we compare the NIM forecast accuracy by 162 models based on 11 different regression techniques, finding that some Machine Learning techniques as well as some linear ones can achieve significantly higher accuracy than the random-walk benchmark, which invalidates the grounds used by the literature to challenge the validity of stress-test. Last, our results from forecast accuracy comparisons are either consistent with or complement those from existing forecasting literature. We believe that the paper is the first systematic study on forecasting bank-specific NIM by Machine Learning Techniques; also, it is a first systematic study on forecast accuracy comparison including both linear and non-linear Machine Learning techniques using financial data for a critical real-world problem; it is a multi-step forecasting example involving iterative forecasting, rolling-origins, recalibration with forecast accuracy measure being scale-independent; robust regression proved to be beneficial for forecasting in presence of outliers. It concludes with policy suggestions and future research directions.

• Item Type: MPRA Paper
• Language: English
• Keywords: Regression, Machine Learning, Time Series Analysis, Bank Capital, Stress Test, Net Interest Margin, Forecasting, PPNR, CCAR
• Subjects: C - Mathematical and Quantitative Methods > C4 - Econometric and Statistical Methods: Special Topics; C - Mathematical and Quantitative Methods > C4 - Econometric and Statistical Methods: Special Topics > C45 - Neural Networks and Related Topics; C - Mathematical and Quantitative Methods > C5 - Econometric Modeling; C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C58 - Financial Econometrics; C - Mathematical and Quantitative Methods > C6 - Mathematical Methods ; Programming Models ; Mathematical and Simulation Modeling; G - Financial Economics > G0 - General > G01 - Financial Crises

References

[1] Acharya V., R. Engle and D. Pierret. 2014. ”Testing macroprudential stress tests: The risk of regulatory risk weights”. Journal of Monetary Economics, Vol. 65 (July) p36-53 [2] Ahmed, N., A. Atiya, N. Gayar and H. El-Shishiny. (2010). ”An Empirical Comparison of Machine Learning Models for Time Series Forecasting”. Econometric Reviews. (September) 29:5,594-621. [3] Armstrong, J. S., and F. Collopy. (1992). ”Error Measures for Generalizing About Forecasting Methods: Empirical Comparisons”. International Journal of Forecasting, Volume 8, Issue 1 (June), Pages 69-80 [4] Basel Committee on Banking Supervision (BCBS). (June 2011). ”Basel III: A global regulatory framework for more resilient banks and banking systems”, BIS. [5] Bergmeir C., R. J. Hyndman and B. Koo (2018). ”A note on the validity of cross-validation for evaluating autoregressive time series prediction”, Computational Statistics and Data Analysis 120, p70–83 [6] Berndt, A., R. Douglas, D. Duffie, M. Ferguson and D. Schranz. (2005). ”Measure Default Risk Premia from Default Swap Rates and EDFs”, BIS Working Papers 173. [7] Bolotnyy, V., R. Edge and L. Guerrieri. (May 2015). ”Revenue Forecasts, Capital Adequacy and the Uncertainty of Stress Test Results”, Harvard Univ. and Federal Reserve Board working paper. [8] Breiman L., J. Friedman, R. Olshen and C. Stone. (1984). Classification and Regression Trees, New York:Wadsworth. [9] Brummelhuis, R. and Z. Luo. (2019a). ”CDS Proxy Construction via Machine Learning Techniques: Methodology and Results”. forthcoming in Journal of Financial Data Science Spring 2019. [10] Brummelhuis, R. and Z. Luo. (2019b). ”CDS Proxy Construction via Machine Learning Techniques: Parametrization, Correlation and Benchmarking”. forthcoming in Journal of Financial Data Science Spring 2019. [11] Brummelhuis, R. and Z. Luo. (2018c). ”Arbitrage Opportunities in CDS Term Structure: Theory and Implications for OTC Derivatives”. SSRN Electronic Journal. [12] Brummelhuis, R., A. Cordoba, M. Quintanilla and L. Seco. (2002). ”Principal Component Value at Risk”. Mathematical Finance, Vol. 12, pp. 23-43. [13] Busch, R. and C. Memmel. (2014). ”Quantify the Components of banks’ Net Interest Margin”, Financial Markets and Portfolio Management (November) Volume 30, Issue 4, pp 371–396. [14] Cortes C. and V. Vapnik. (September 1995). ”Support-vector networks”, Journal of Machine Learning, Vol. 20, Issue 3, pp (273-297) [15] Chen C, J. Twycross and JM Garibaldi. (2017). ”A new accuracy measure based on bounded relative error for time series forecasting”. PLOS ONE 12(3): e0174202. https://doi.org/10.1371/journal.pone.0174202 [16] Covas, F., B. Rump and E. Zakrajsek. (2014). ”Stress-testing U.S. Bank Holiding Companies: A Dynamic Panel Quantile Regression Approach”. International Journal of Forecasting 30(3), 691-713. [17] Delgado M. and D. Amorim. (2014). ”Do we need Hundreds of Classifiers to Solve Real World Classification Problems?”, Journal of Machine Learning Research 15, 3133-318. [18] European Central Bank. (2010). ”Beyond ROE: How to Measure Bank Performance”, Appendix to the report on EU banking structures. [19] Fox, J. and S. Weisburg. (2013). ”Robust Regression”. School of Statistics, Univ. of Minnesota [20] Glasserman, P. and G. Tangirala. (January 2016). ”Are the Fed’s Stress Test Results Predictable?” The Journal of Alternative Investments. Vol. 18, Issue 4, (Spring): pp.82-97. [21] A. Graves. (June 2014). ”Generating Sequences With Recurrent Neural Networks”, Department of Computer Science, University of Toronto [22] Greene, W. Econometric Analysis. (1997). 3rd ed. New Jersey:Prentice Hall. [23] Grover, S. and M. McCracken. (2014). ”Factor-Based Prediction of Aggregate Bank Stress”. Federal Reserve Bank of St. Louis Review (Second Quarter) 96(2), pp. 173-93. [24] Guerrieri, L. and M. Welch. (July 2012). ”Can Macro Variables Used in Stress Testing Forecast the Performance of Banks?”. FEDS Working Paper No. 2012-49. [25] Hastie, T., R. Tisbshirani and J. Friedman. (2009). The Elements of Statistical Learning. 2nd ed. Springer Science+Business Media LLC. [26] Hirtle B., A. Kovner, J. Vickery and M. Bhanot. (2016). ”Assessing financial stability: The Capital and Loss Assessment under Stress Scenarios (CLASS) model”. Journal of Banking & Finance 69 S35–S55 [27] Ho, T.S.Y. and A. Saunders. (1981). ”The Determinants of Bank Interest Rate Margins: Theory and Empirical Evidence”. Journal of Financial and Quantitative Analysis 16(4), 581-600. [28] Hyndman Rob J. and Anne B. Koehler. (1992). ”Another look at measures of forecast accuracy”, International Journal of Forecasting Volume 8, Issue 1 (June) Pages 69-80 [29] Kilian L. and M. Taylor. (2003). ”why is it so difficult to beat the random walk forecast of exchange rates?”, Journal of Int’l Economics 60 (2003) 85–107. [30] Kim, S.andC.Scott.(September2012).”RobustKernelDensityEstimation”, The Journal of Machine Learning Research, Vol. 13 Issue 1, pp2529-2565 . [31] Kupiec, P. (2018). ”On the accuracy of alternative approaches for calibrating bank stress test models”. Journal of Financial Stability. JFS-639. [32] Lin T., B. Horne, Peter Ti and C. Giles. (1996). ”Learning long-term dependencies in NARX recurrent neural networks”, IEEE Transactions on Neural Networks , vol. 7, no. 6, p.1329. [33] Loh, W. (2002). ”Regression Trees with Unbiased Variable Selection and Interaction Detection”. Statistica Sinica 12. 361-386. [34] Marcellino M, J. Stock and M. Watson. (2006). ”A Comparison of Direct and Iterated Multistep AR Methods for Forecasting Macroeconomic Time Series”. Journal of Econometrics. 135 :499-526. [35] MacKay, DJC. (1992). ”Bayesian interpolation.” Neural computation. Vol. 4, No. 3, pp. 415-447. [36] Matheron, G. (1963). ”Principles of geostatistics”. Economic Geology, 58, pp 1246-1266. [37] McNeil, A. J., R. Frey and P. Embrechts. (2015). Quantitative risk management: Concepts, techniques and tools, Princeton Univ. Press, 2nd edition [38] Platt J. (1998). ”Fast training of support vector machines using sequential minimal optimization”, In Scholkopf, B., C.J.C. Burges and A.J. Smola (eds). Advances in Kernel Methods - Support Vector Learning, pp. 185-208, MIT Press [39] Rasmussen C. and C. Williams. (2006). Gaussian Processes for Machine Learning. The MIT Press. [40] Scholkopf B., P. Bartlett., A. Smola and R. Williamson. (1998). ”Shrinking the tube: a new support vector regression algorithm”, In Kearns M. S., Solla M. S., and Cohn D. A., editors, Advances in Neural Information Processing Systems, 11. MIT Press. [41] Schuermann, T. (2014). ”Stress Testing Banks”. International Journal of Forecasting. 30. 717-728. [42] Tashman, Leonard J. (2000). ”Out-of-sample tests of forecasting accuracy: an analysis and review”, International Journal of Forecasting 16, 437–450. [43] Tukey, J. (1977). Exploratory Data Analysis. Pearson. 1 ed. 1 (January). [44] Zhang, G., B. Patuwo and M. Hu. (1998). ”Forecasting with artificial neural networks: The state of the art”, International Journal of Forecasting 14 35-62