Comparing different early warning systems: Results from a horse race competition among members of the Macro-prudential Research Network

Alessi, Lucia and Antunes, Antonio and Babecky, Jan and Baltussen, Simon and Behn, Markus and Bonfim, Diana and Bush, Oliver and Detken, Carsten and Frost, Jon and Guimaraes, Rodrigo and Havranek, Tomas and Joy, Mark and Kauko, Karlo and Mateju, Jakub and Monteiro, Nuno and Neudorfer, Benjamin and Peltonen, Tuomas and Rodrigues, Paulo and Rusnak, Marek and Schudel, Willem and Sigmund, Michael and Stremmel, Hanno and Smidkova, Katerina and van Tilburg, Ruben and Vasicek, Borek and Zigraiova, Diana (2015): Comparing different early warning systems: Results from a horse race competition among members of the Macro-prudential Research Network.

Abstract

Over the recent decades researchers in academia and central banks have developed early warning systems (EWS) designed to warn policy makers of potential future economic and financial crises. These EWS are based on diverse approaches and empirical models. In this paper we compare the performance of nine distinct models for predicting banking crises resulting from the work of the Macroprudential Research Network (MaRs) initiated by the European System of Central Banks. In order to ensure comparability, all models use the same database of crises created by MaRs and comparable sets of potential early warning indicators. We evaluate the models’ relative usefulness by comparing the ratios of false alarms and missed crises and discuss implications for pratical use and future research. We find that multivariate models, in their many appearances, have great potential added value over simple signalling models. One of the main policy recommendations coming from this exercise is that policy makers can benefit from taking a broad methodological approach when they develop models to set macro-prudential instruments.

• Item Type: MPRA Paper
• Language: English
• Keywords: Early warning systems, financial crisis, Bayesian model averaging
• Subjects: G - Financial Economics > G0 - General > G01 - Financial Crises

References

Abiad, A. (2003). “Early-Warning Systems: A Survey and a Regime-Switching Approach.” IMF Working Paper No. 03/32.

Aikman, David; Alessandri, Piergiorgio; Eklund, Bruno; Gai, Prasanna; Kapadia, Sujit; Martin, Elizabeth; Mora, Nada; Sterne, Gabriel; Willison, Matthew (2009). “Funding Liquidity Risk in a Quantitative Model of Systemic Stability”. Bank of England Working Paper, No. 372, London, June 2009.

Alessi, L. and C. Detken (2011). “Quasi Real Time Early Warning Indicators for Costly Asset Price Boom/Bust Cycles. A Role for Global Liquidity”. European Journal of Political Economy, Vol. 27, Issue: 3, Pages: 520-533, 2011.

Alessi, L. and C. Detken (2014). “Identifying excessive credit growth and leverage”. ECB Working Paper, forthcoming.

Antunes, A., D. Bonfim, N. Monteiro, and P. Rodrigues (2014). “Early Warning Indicators of Banking Crises: Exploring New Data and Tools”. Economic Bulletin Banco de Portugal April 2014.

Babecký, Jan; Havránek, Tomáš; Matějů, Jakub; Rusnák, Marek; Šmídková, Kateřina; Vašíček, Bořek (2012). “Banking, Debt and Currency Crises: Early Warning Indicators for Developed Countries.” Working Paper Series 1485, European Central Bank.

Anselin, Luc (2002). “Under the Hood: Issues in the Specification and Interpretation of Spatial Regression Models.” Agricultural economics, 27(3), pp.247-267.

Bank of England (2008, April). Financial Stability Report, London, April 2008.

Basel Committee on Banking Supervision (2010). Guidance for National Authorities Operating the Countercyclical Capital Buffer. December 2010.

Berg, A. and C. Patillo (1998). “Are Currency Crises Predictable? A Test.” IMF Working Paper No. 98/154.

Bilson, J. F. O. (1979). “Leading Indicators of Currency Devaluations.” Columbia Journal of World Business 14, pp. 62–76.

Breiman, Leo (2001). “Random Forests.” Machine Learning 45 (1): 5–32.

Bussiere, M. and Fratzscher, M. (2006). “Towards a New Early Warning System of Financial Crises.” Journal of International Money and Finance, 25, 953–973.

Davis, Philip E.; Karim, Dilruba (2008). “Comparing Early Warning Systems for Banking Crises, Journal of Financial Stability.” Vol. 4 (2008), Iss. 2, pp. 89-120.

Demirgüç-Kunt, Asli; Detragiache, Enrica (2005). “Cross-Country Empirical Studies of Systemic Bank Distress: A Survey”. IMF Working Paper, No. 96/05, Washington 2005.

Eichengreen, Barry; Rose, Andrew K; Wyplosz, Charles (1996). “Contagious Currency Crisis”. NBER Working Paper No. 5681.

Elsinger, Helmut; Lehar, Alfred; Summer, Martin (2006). “Using Market Information for Banking System Risk Assessment”. International Journal of Central Banking, Vol. 2 (2006), No. 1, pp. 137-165.

Fabian Valencia & Luc Laeven (2012). “Systemic Banking Crises Database: An Update.” IMF Working Papers 12/163, International Monetary Fund.

Forbes, K.J., Warnock, F.E. (2012). “Capital Flow Waves: Surges, Stops, Flight, and Retrenchment.” Journal of International Economics 88, pp.235-251.

Frankel, J. A. and A. K. Rose (1996). “Currency Crashes in Emerging Markets: An Empirical Treatment.” Journal of International Economics 41(3–4), pp. 351–366.

Frankel, J. A. and G. Saravelos (2010). “Are Leading Indicators of Financial Crisis Useful for Assessing Country Vulnerability? Evidence from the 2008–09 Global Crisis.” NBER Working Paper No. 16047.

Gaytán, Alejandro; Johnson, Christian A. (2002). “A Review of the Literature on Early Warning Systems for Banking Crises.” Central Bank of Chile Working Paper, No. 183, Santiago, October 2002.

Jiménez-Rodríguez, R. and Sánchez, M. (2004). “Oil Price Shocks and Real GDP Growth; Empirical Evidence for OECD Countries.” Working Paper Series 362, European Central Bank.

Jordà, Òscar; Schularick, Moritz and Taylor Alan M. (2011). “When Credit Bites Back: Leverage, Business Cycles, and Crises.” Working Paper Series 2011-27, Federal Reserve Bank of San Francisco.

Kakamu, Kazuhiko, and Hajime Wago (2010). “Estimation of Regional Business Cycle in Japan Using Bayesian Panel Spatial Autoregressive Probit Model.” Working paper. Available at: http://www. mssanz.org. au/modsim05/proceedings/papers/kakamu_2. pdf, 2007.

Kaminsky, G. L. and C. M. Reinhart (1996). “The Twin Crises: The Causes of Banking and Balance-of-Payments Problems.” Board of Governors of the Federal Reserve System, International Finance Discussion Papers 544. Kaminsky, Graciela L.; Lizondo, Saul; Reinhart, Carmen M. (1998). “Leading Indicators of Currency Crises”. IMF Staff Paper, Vol. 45 (1998), No. 1, pp. 1-48.

Kaminsky, Graciela L.; Reinhart, Carmen M. (1999). “The Twin Crises: The Causes of Banking and Balance-of-Payments Problems”. American Economic Review, Vol. 89 (1999), No. 3, pp. 473-500.

Kauko, Karlo (2014). “How to Foresee Banking Crises, a Survey of Empirical Literature.” Economic Systems (forthcoming).

Krugman, P. (1979). “A Model of Balance-of-Payments Crises.” Journal of Money, Credit and Banking 11(3), pp. 311–325.

Laeven, L. and Valencia, F. (2008). “Systemic Banking Crises: A New Database.” International Monetary Fund, WP/08/224.

Luc Laeven & Fabian Valencia (2013). “Systemic Banking Crises Database.” IMF Economic Review, Palgrave Macmillan, vol. 61(2), pages 225-270, June.

Lilien, D.M. (1982). “Sectoral Shifts and Cyclical Unemployment.” Journal of Political Economy 90 (4), pp. 777-793.

Lin, Chin-Shien; Khan, Haider A.; Wang, Ying-Chieh; Chang, Ruei-Yuan (2006). “A New Approach to Modelling Early Warning Systems for Currency Crises: Can a Machine-Learning Fuzzy Expert System Predict the Currency Crises Effectively?” Center for International Research on the Japanese Economy Discussion Paper, CIRJE-F-411, Tokyo, April 2006.

Mayes, David G. & Hanno Stremmel, (2014).”The Effectiveness of Capital Adequacy Measures in Predicting Bank Distres.” SUERF Studies: 2014/1.

Novo, Álvaro (2003). “Contagious currency crises: a spatial probit approach,” Working Paper 5-03, Banco de Portugal. Peria, M. S. M. (2002). “A Regime-Switching Approach to the Study of Speculative Attacks: A Focus on EMS Crises.” In James D. Hamilton and Baldev Raj, editors, Advances in Markov-Switching Models. Heidelberg: Physica-Verlag.

Rose, A. K. and M. M. Spiegel (2009). “Cross-Country Causes and Consequences of the 2008 Crisis: Early Warning.” NBER Working Paper No. 15357.