Adaptive Conformal Inference for computing Market Risk Measures: an Analysis with Four Thousands Crypto-Assets

Fantazzini, Dean (2024): Adaptive Conformal Inference for computing Market Risk Measures: an Analysis with Four Thousands Crypto-Assets. Forthcoming in: Journal of Risk and Financial Management (2024)

Abstract

This paper investigates the estimation of the Value-at-Risk (VaR) across various probability levels for the log-returns of a comprehensive dataset comprising four thousand crypto-assets. Employing four recently introduced Adaptive Conformal Inference (ACI) algorithms, we aim to provide robust uncertainty estimates crucial for effective risk management in financial markets. We contrast the performance of these ACI algorithms with that of traditional benchmark models, including GARCH models and daily range models. Despite the substantial volatility observed in the majority of crypto-assets, our findings indicate that ACI algorithms exhibit notable efficacy. In contrast, daily range models, and to a lesser extent, GARCH models, encounter challenges related to numerical convergence issues and structural breaks. Among the ACI algorithms, the Fully Adaptive Conformal Inference (FACI) and the Scale-Free Online Gradient Descent (SF-OGD) stand out for their ability to provide precise VaR estimates across all quantiles examined. Conversely, the Aggregated Adaptive Conformal Inference (AgACI) and the Strongly Adaptive Online Conformal Prediction (SAOCP) demonstrate proficiency in estimating VaR for extreme quantiles but tend to be overly conservative for higher probability levels. These conclusions withstand robustness checks encompassing the market capitalization of crypto-assets, time series size, and different forecasting methods for asset log-returns. This study underscores the promise of ACI algorithms in enhancing risk assessment practices in the context of volatile and dynamic crypto-asset markets.

• Item Type: MPRA Paper
• Language: English
• Keywords: Value at Risk (VaR); Adaptive Conformal Inference (ACI); Aggregated Adaptive Conformal Inference (AgACI); Fully Adaptive Conformal Inference (FACI); Scale-Free Online Gradient Descent (SF-OGD); Strongly Adaptive Online Conformal Prediction (SAOCP), GARCH; Daily Range; Risk Management
• Subjects: C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C14 - Semiparametric and Nonparametric Methods: General; C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C51 - Model Construction and Estimation; C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C53 - Forecasting and Prediction Methods ; Simulation Methods; C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C58 - Financial Econometrics; G - Financial Economics > G1 - General Financial Markets > G17 - Financial Forecasting and Simulation; G - Financial Economics > G3 - Corporate Finance and Governance > G32 - Financing Policy ; Financial Risk and Risk Management ; Capital and Ownership Structure ; Value of Firms ; Goodwill

References

Abad, Pilar, Sonia Benito, and Carmen López. 2014. A comprehensive review of value at risk methodologies. The Spanish Review of Financial Economics 12: 15–32.

Alexander, Carol, and Michael Dakos. 2023. Assessing the accuracy of exponentially weighted moving average models for value-at-risk and expected shortfall of crypto portfolios. Quantitative Finance 23: 393–427.

Angelopoulos, Anastasios, and Stephen Bates. 2023. Conformal prediction: A gentle introduction. Foundations and Trends in Machine Learning 16: 494–591.

Barber, Rina Foygel, Emmanuel J. Candès, Aaditya Ramdas, and Ryan J. Tibshirani. 2021. Predictive inference with the jackknife+. The Annals of Statistics 49: 486–507.

Barber, Rina Foygel, Emmanuel J. Candes, Aaditya Ramdas, and Ryan J. Tibshirani. 2023. Conformal prediction beyond exchangeability. The Annals of Statistics 51: 816–45.

Bhatnagar, Aadyot, Huan Wang, Caiming Xiong, and Yu Bai. 2023. Improved online conformal prediction via strongly adaptive online learning. Paper presented at 40th International Conference on Machine Learning, Honolulu, HI, USA, July 23–29.

Bianchi, Carluccio, Maria Elena De Giuli, Dean Fantazzini, and Mario Maggi. 2011. Small sample properties of copula-GARCH modelling: A Monte Carlo study. Applied Financial Economics 21: 1587–97.

Cai, Yong, and Kalimuthu Krishnamoorthy. 2006. Exact size and power properties of five tests for multinomial proportions. Communications in Statistics: Simulation and Computation 35: 149–60.

Cauchois, Maxime, Suyash Gupta, and John C. Duchi. 2021. Knowing what you know: Valid and validated confidence sets in multiclass and multilabel prediction. The Journal of Machine Learning Research 22: 3681–722.

Cesa-Bianchi, Nicolo, and Gábor Lugosi. 2006. Prediction, Learning, and Games. Cambridge: Cambridge University Press.

Chernozhukov, Victor, Kaspar Wüthrich, and Yinchu Zhu. 2018. Exact and robust conformal inference methods for predictive machine learning with dependent data. Paper presented at Conference On Learning Theory, Stockholm, Sweden, July 6–9; pp. 732–49.

Chou, Ray Yeutien, Hengchih Chou, and Nathan Liu. Range Volatility: A Review of Models and Empirical Studies. New York: Springer, pp. 2029–50.

Christoffersen, Peter F. 1998. Evaluating interval forecasts. International Economic Review 39: 841–62.

Corsi, Fulvio. 2009. A simple approximate long-memory model of realized volatility. Journal of Financial Econometrics 7: 174–96.

Dashevskiy, Mikhail, and Zhiyuan Luo. 2008. Network traffic demand prediction with confidence. Paper presented at IEEE GLOBECOM 2008—-2008 IEEE Global Telecommunications Conference, New Orleans, LA, USA, November 30–December 4; pp. 1–5.

Emmer, Susanne, Marie Kratz, and Dirk Tasche. 2015. What is the best risk measure in practice? A comparison of standard measures. Journal of Risk 18: 31–60.

Engle, Robert F., and Simone Manganelli. 2004. Caviar: Conditional autoregressive value at risk by regression quantiles. Journal of Business & Economic Statistics 22: 367–81.

Fantazzini, Dean. 2009. The effects of misspecified marginals and copulas on computing the Value-at-Risk: A Monte Carlo study. Computational Statistics & Data Analysis 53: 2168–88.

Fantazzini, Dean. 2019. Quantitative Finance with R and Cryptocurrencies. Chicago: Independently Publisher. ISBN-13: 978-1090685315.

Fantazzini, Dean. 2022. Crypto-coins and credit risk: Modelling and forecasting their probability of death. Journal of Risk and Financial Management 15: 304.

Fantazzini, Dean. 2023. Assessing the credit risk of crypto-assets using daily range volatility models. Information 14: 254.

Fantazzini, Dean, and Tamara Shangina. 2019. The importance of being informed: forecasting market risk measures for the Russian RTS index future using online data and implied volatility over two decades. Applied Econometrics 55: 5–31.

Fiorentini, Gabriele, Giorgio Calzolari, and Lorenzo Panattoni. 1996. Analytic derivatives and the computation of GARCH estimates. Journal of Applied Econometrics 11: 399–417.

Fiszeder, Piotr, Marcin Fałdzi´nski, and Peter Molnár. 2019. Range-based dcc models for covariance and value-at-risk forecasting. Journal of Empirical Finance 54: 58–76.

Gaillard, Pierre, Goude Yannig, Plagne Laurent, Dubois Thibaut, and Thieurmel Benoit. 2023. Opera: Online prediction by expert aggregation. Available online: https://pierre.gaillard.me/opera.html (accessed on 1 June 2024).

Garman, Mark B., and Michael J. Klass. 1980. On the estimation of security price volatilities from historical data. Journal of Business 53: 67–78.

Gibbs, Isaac, and Emmanuel Candès. 2021. Adaptive conformal inference under distribution shift. Advances in Neural Information Processing Systems 34: 1660–72.

Gibbs, Isaac, and Emmanuel Candès. 2022. Conformal inference for online prediction with arbitrary distribution shifts. arXiv. arXiv:2208.08401.

Gneiting, Tilmann. 2011. Making and evaluating point forecasts. Journal of the American Statistical Association 106: 746–62.

González-Rivera, Gloria, Tae-Hwy Lee, and Santosh Mishra. 2004. Forecasting volatility: A reality check based on option pricing, utility function, value-at-risk, and predictive likelihood. International Journal of Forecasting 20: 629–45.

Gradu, Paula, Elad Hazan, and Edgar Minasyan. 2023. Adaptive regret for control of time-varying dynamics. Paper presented at Learning for Dynamics and Control Conference, Philadelphia, PA, USA, June 15–16; pp. 560–72.

Hansen, Peter, Asger Lunde, and James Nason. 2011. The model confidence set. Econometrica 79: 453–97.

Hwang, Soosung, and Pedro Valls Pereira. 2006. Small sample properties of GARCH estimates and persistence. The European Journal of Finance 12: 473–94.

Hyndman, Rob, and Yeasmin Khandakar. 2008. Automatic time series forecasting: The forecast package for R. Journal of Statistical Software 27: 1–22.

Kath, Christopher, and Florian Ziel. 2021. Conformal prediction interval estimation and applications to day-ahead and intraday power markets. International Journal of Forecasting 37: 777–99.

Koenker, Roger, and Gilbert Bassett, Jr. 1978. Regression quantiles. Econometrica: Journal of the Econometric Society 46: 33–50.

Kratz, Marie, Yen H. Lok, and Alexander J. McNeil. 2018. Multinomial var backtests: A simple implicit approach to backtesting expected shortfall. Journal of Banking & Finance 88: 393–407.

Kupiec, Paul H. 1995. Techniques for verifying the accuracy of risk measurement models. The Journal of Derivatives 3: 73–84.

Lei, Jing, and Larry Wasserman. 2014. Distribution-free prediction bands for non-parametric regression. Journal of the Royal Statistical Society Series B: Statistical Methodology 76: 71–96.

Lei, Jing, James Robins, and Larry Wasserman. 2013. Distribution-free prediction sets. Journal of the American Statistical Association 108: 278–87.

Liu, Wei, Artur Semeyutin, Chi Keung Marco Lau, and Giray Gozgor. 2020. Forecasting value-at-risk of cryptocurrencies with riskmetrics type models. Research in International Business and Finance 54: 101259.

Lyócsa, Štefan, Peter Molnár, and Tomáš V`yrost. 2021. Stock market volatility forecasting: Do we need high-frequency data? International Journal of Forecasting 37: 1092–10.

Manokhin, Valery. 2024. Awesome Conformal Prediction. Available online: https://zenodo.org/records/6467205 (accessed on 1 June 2024).

Molnár, Peter. 2012. Properties of range-based volatility estimators. International Review of Financial Analysis 23: 20–29.

Müller, Fernanda Maria, Samuel Solgon Santos, Thalles Weber Gössling, and Marcelo Brutti Righi. 2022. Comparison of risk forecasts for cryptocurrencies: A focus on range value at risk. Finance Research Letters 48: 102916.

Nison, Steve. 1994. Beyond Candlesticks: New Japanese Charting Techniques Revealed. Hoboken: John Wiley & Sons, vol. 56.

Orabona, Francesco, and Dávid Pál. 2018. Scale-free online learning. Theoretical Computer Science 716: 50–69.

Papadopoulos, Harris. 2008. Inductive conformal prediction: Theory and application to neural networks. In Tools in Artificial Intelligence. Rijeka: IntechOpen.

Parkinson, Michael. 1980. The extreme value method for estimating the variance of the rate of return. Journal of Business 53: 61–65.

Patton, Andrew J. 2011. Volatility forecast comparison using imperfect volatility proxies. Journal of Econometrics 160: 246–56.

Podkopaev, Aleksandr and Aaditya Ramdas. 2021. Distribution-free uncertainty quantification for classification under label shift. Paper presented at Uncertainty in Artificial Intelligence, Online, July 27–30; pp. 844–53.

Romano, Yaniv, Evan Patterson, and Emmanuel J Candès. 2019. Conformalized quantile regression. Paper presented at 33rd International Conference on Neural Information Processing Systems, Vancouver, BC, Canada, December 8–14; pp. 3543–53.

Romano, Yaniv, Matteo Sesia, and Emmanuel Candes. 2020. Classification with valid and adaptive coverage. Advances in Neural Information Processing Systems 33: 3581–91.

Shafer, Glenn, and Vladimir Vovk. 2008. A tutorial on conformal prediction. Journal of Machine Learning Research 9: 371–421.

Sousa, Martim, Ana Maria Tomé, and José Moreira. 2022. A general framework for multi-step ahead adaptive conformal heteroscedastic time-series forecasting. arXiv. arXiv:2207.14219.

Stankeviciute, Kamile, Ahmed M Alaa, and Mihaela van der Schaar. 2021. Conformal time-series forecasting. Advances in Neural Information Processing Systems 34: 6216–28.

Susmann, Herbert, Antoine Chambaz, and Julie Josse. 2023. Adaptiveconformal: An r package for adaptive conformal inference. arXiv. arXiv:2312.00448.

Tibshirani, Ryan J., Rina Foygel Barber, Emmanuel J. Candès, and Aaditya Ramdas. 2019. Conformal prediction under covariate shift. Paper presented at 33rd International Conference on Neural Information Processing Systems, Vancouver, BC, Canada, December 8–14; pp. 2530–40.

Trucíos, Carlos, and James W. Taylor. 2023. A comparison of methods for forecasting value at risk and expected shortfall of cryptocurrencies. Journal of Forecasting 42: 989–1007.

Venables, William N., and Brian D. Ripley. 2002. Modern Applied Statistics with S-PLUS. Berlin: Springer Science & Business Media.

Vovk, Vladimir, Alexander Gammerman, and Glenn Shafer. 2005. Algorithmic Learning in a Random World. Berlin: Springer, vol. 29.

Vovk, Vladimir, Ilia Nouretdinov, Valery Manokhin, and Alexander Gammerman. 2018. Cross-conformal predictive distributions. Paper presented at Conformal and Probabilistic Prediction and Applications, Maastricht, The Netherlands, June 1–13; pp. 37–51

Vovk, Volodya, Alexander Gammerman, and Craig Saunders. 1999. Machine-learning applications of algorithmic randomness. Paper presented at Sixteenth International Conference on Machine Learning, Bled, Slovenia, June 27–30; pp. 444–53.

Wimmerstedt, Lisa. 2015. Backtesting Expected Shortfall: The Design and Implementation of Different Backtests. Technical Report. Stockholm: Swedish Royal Institute of Technology.

Wintenberger, Olivier. 2017. Optimal learning with bernstein online aggregation. Machine Learning 106: 119–41.

Wisniewski, Wojciech, David Lindsay, and Sian Lindsay. 2020. Application of conformal prediction interval estimations to market makers’ net positions. Paper presented at Conformal and Probabilistic Prediction and Applications, Online, September 9–11; pp. 285–301.

Xu, Chen, and Yao Xie. 2021. Conformal prediction interval for dynamic time-series. Paper presented at International Conference on Machine Learning, Virtual, July 18–24; pp. 11559–69.

Yang, Yachong, Arun Kumar Kuchibhotla, and Eric Tchetgen Tchetgen. 2022. Doubly robust calibration of prediction sets under covariate shift. arXiv. arXiv:2203.01761.

Zaffran, Margaux, Olivier Féron, Yannig Goude, Julie Josse, and Aymeric Dieuleveut. 2022. Adaptive conformal predictions for time-series. Paper presented at International Conference on Machine Learning, Baltimore, MD, USA, July 17–23; pp. 25834–66.