Model Risk Management in the Era of Generative AI: Challenges, Opportunities, and Future Directions

Joshi, Satyadhar (2025): Model Risk Management in the Era of Generative AI: Challenges, Opportunities, and Future Directions. Published in: International Journal of Scientific and Research Publications , Vol. 5, No. 15 (20 May 2025): pp. 299-309.

Abstract

The rapid adoption of generative AI in various sectors, particularly in finance, has introduced new challenges and opportunities for model risk management (MRM). This paper provides a comprehensive review of the current state of MRM in the context of generative AI, focusing on the risks, regulatory frameworks, and mitigation strategies. We explore the implications of generative AI on financial institutions, the evolving regulatory landscape, and the role of advanced MRM frameworks in ensuring compliance and mitigating risks. By synthesizing insights from 50+ recent articles, this paper aims to provide a roadmap for future research and practical applications of MRM in the generative AI era. It examines the key risks associated with these models, including bias, lack of transparency, and potential for misuse, and explores the regulatory frameworks and best practices being developed to mitigate these risks. We delve into the specific challenges faced by financial institutions in adapting their MRM strategies to encompass generative AI, and highlight the emerging tools and technologies that can support effective risk management.

This paper also discusses quantitative methods for risk quantification, such as probabilistic frameworks, Monte Carlo simulations, and adversarial risk metrics, which are essential for assessing the reliability and robustness of generative AI models. Foundational metrics, including fairness measures like demographic parity and equalized odds, are explored to address bias and ensure ethical AI deployment. Additionally, the paper presents pseudocode for key algorithms, such as risk quantification and adversarial risk calculation, to provide a practical understanding of these methods. A detailed gap analysis identifies critical shortcomings in current MRM frameworks, such as the lack of standardized validation methods and inadequate handling of adversarial robustness. Based on these gaps, the paper proposes solutions, including the development of advanced validation frameworks, integration of fairness metrics, and alignment with regulatory standards. These findings and proposals aim to guide financial institutions in adopting generative AI responsibly while addressing the unique risks it poses. This paper serves as a valuable resource for professionals and researchers seeking to understand and navigate the complexities of MRM in the age of generative AI.

• Item Type: MPRA Paper
• Language: English
• Keywords: Model Risk Management, Generative AI, Financial Institutions, Regulatory Compliance, Risk Mitigation, AI Governance.
• Subjects: C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C52 - Model Evaluation, Validation, and Selection

References

[1] “AI model governance: What it is and why it’s important Collibra.” Accessed: Mar. 18, 2025. [Online]. Available: https://www.collibra.com/blog/ai-model-governance-what-it-is-and-why-its-important

[2] “AI Use by Financial Institutions OSFI and FCAC Recommendations for Sound Risk Management McMillan LLP.” Accessed: Mar. 18, 2025. [Online]. Available: https://mcmillan.ca/insights/ai-use-by-financial-institutions-osfi-and-fcac-recommendations-for-sound-risk-management/

[3] J. C. Crisanto, C. B. Leuterio, J. Prenio, and J. Yong, “Regulating AI in the financial sector: Recent developments and main challenges,” Dec. 2024, Accessed: Mar. 18, 2025. [Online]. Available: https://www.bis.org/fsi/publ/insights63.htm

[4] Ambilio, “Generative AI for Risk Management in Financial Sector,” Ambilio. Mar. 2023. Accessed: Mar. 18, 2025. [Online]. Available: https://ambilio.com/generative-ai-for-risk-management-in-financial-sector/

[5] “Mitigating Model Risk in AI Advancing an MRM Framework for AI/ML Models at Financial Institutions Chartis Research.” Jan. 2025. Accessed: Mar. 18, 2025. [Online]. Available: https://www.chartis-research.com/custom-insights/7947296/mitigating-model-risk-in-ai-advancing-an-mrm-framework-for-aiml-models-at-financial-institutions

[6] O. of the S. of F. Institutions, “OSFI-FCAC Risk Report - AI Uses and Risks at Federally Regulated Financial Institutions.” Oct. 2024. Accessed: Mar. 18, 2025. [Online]. Available: https://www.osfi-bsif.gc.ca/en/about-osfi/reports-publications/osfi-fcac-risk-report-ai-uses-risks-federally-regulated-financial-institutions

[7] marcusevans Group, “Agenda - Model Risk, marcus evans Conferences,” Agenda - Model Risk, marcus evans Conferences. Accessed: Mar. 18, 2025. [Online]. Available: https://www.marcusevans.com/conferences/gfmimodelrisk/delegates/agenda

[8] “Generative AI for Catastrophe Risk Xceedance.” Sep. 2023. Accessed: Mar. 18, 2025. [Online]. Available: https://www.xceedance.com/hurricane-idalia-can-generative-ai-bolster-hurricane-risk-management/

[9] “AI Risk-Management Standards Profile for General-Purpose AI Systems (GPAIS) and Foundation Models,” CLTC. Accessed: Mar. 18, 2025. [Online]. Available: https://cltc.berkeley.edu/publication/ai-risk-management-standards-profile/

[10] “Uncompromising In Model Risk Management.” Feb. 2022. Accessed: Mar. 18, 2025. [Online]. Available: https://www.yields.io/, https://www.yields.io/

[11] “Fairly AI Managing AI Risk in Generative AI.” Accessed: Mar. 18, 2025. [Online]. Available: https://www.fairly.ai/blog/managing-ai-risk-in-generative-ai

[12] S. Yang, J. Chen, A. Gupta, Z. Feinstein, and W. Knottenbelt, “Generative AI and LLM in financial risk modeling and applications.”

[13] “How Generative AI Will Disrupt Credit Risk Modeling.” Accessed: Mar. 18, 2025. [Online]. Available: https://www.garp.org/risk-intelligence/credit/generative-ai-risk-090123

[14] “XFIN-702 GenAI for Financial Model Risk Management Georgetown School of Continuing Studies (SCS).” Accessed: Mar. 18, 2025. [Online]. Available: https://portal.scs.georgetown.edu/search/publicCourseSearchDetails.do?method=load&courseId=55866956

[15] “AI Model Diligence: 3 Steps for Financial Institutions to Manage Model Risk,” JD Supra. Accessed: Mar. 18, 2025. [Online]. Available: https://www.jdsupra.com/legalnews/ai-model-diligence-3-steps-for-6350445/

[16] “AI and Model Risk Governance.” Accessed: Mar. 18, 2025. [Online]. Available: https://www.jpmorgan.com/technology/news/ai-and-model-risk-governance

[17] “Artificial Intelligence and Model Risk Management.” Accessed: Mar. 18, 2025. [Online]. Available: https://kpmg.com/us/en/articles/2024/artificial-intelligence-and-model-risk-management.html

[18] B. Wire, “H2O.ai Becomes First to Bring Model Risk Management to Generative AI for Regulated Industries,” The Daily News. Mar. 2025. Accessed: Mar. 18, 2025. [Online]. Available: https://www.galvnews.com/h2o-ai-becomes-first-to-bring-model-risk-management-to-generative-ai-for-regulated-industries/article_76952b49-10d0-50a5-a7a2-ff1905dfcf25.html

[19] “Model risk management is evolving to govern generative AI.” Accessed: Mar. 18, 2025. [Online]. Available: https://informaconnect.com/model-risk-management-is-evolving-to-govern-generative-ai/

[20] “Automating Model Risk Compliance: Model Development DataRobot AI Cloud,” DataRobot. Accessed: Mar. 18, 2025. [Online]. Available: https://www.datarobot.com/blog/automating-model-risk-compliance-model-development/

[21] J. Satyadhar, “Review, tutorials and introduction to cloud platforms for agentic GenAI: A comparative studies [pre-print],” Tutorials and Introduction to Cloud Platforms for Agentic GenAI: A Comparative Studies [Pre-Print](February 15, 2025), 2025.

[22] Joshi Satyadhar, “Enhancing structured finance risk models (leland-toft and box-cox) using GenAI (VAEs GANs),” IJSRA, vol. 14, no. 1, pp. 1618–1630, 2025.

[23] Joshi, Satyadhar, “Leveraging prompt engineering to enhance financial market integrity and risk management,” World Journal of Advanced Research and Reviews WJARR, vol. 25, no. 1, pp. 1775–1785, 2025.

[24] J. Satyadhar, “The synergy of generative AI and big data for financial risk: Review of recent developments,” IJFMR-International Journal For Multidisciplinary Research, vol. 7, no. 1, 2025.

[25] Joshi Satyadhar, “Using gen AI agents with GAE and VAE to enhance resilience of US markets,” The International Journal of Computational Science, Information Technology and Control Engineering (IJCSITCE), vol. 12, no. 1, pp. 23–38, 2025.

[26] J. Satyadhar, “Advancing innovation in financial stability: A comprehensive review of ai agent frameworks, challenges and applications,” World Journal of Advanced Engineering Technology and Sciences WJAETS ISSN: 2582-8266 (Online), vol. 14, no. 2, pp. 117–126, 2025.

[27] Joshi Satyadhar, “Implementing gen AI for increasing robustness of US financial and regulatory system,” International Journal of Innovative Research in Engineering and Management, vol. 11, no. 6, pp. 175–179, 2024.

[28] J. Satyadhar, “Gen AI for market risk and credit risk learn agentically powered gen AI; gen AI agentic framework for financial risk management,” Gen AI Agentic Framework for Financial Risk Management (January 15, 2025), 2025.

[29] J. Satyadhar, “Agentic gen AI for financial risk management.” Draft2Digital, 2025.

[30] S. Joshi, “A literature review of gen AI agents in financial applications: Models and implementations,” International Journal of Science and Research (IJSR), vol. 12, no. 1, pp. 1094–1100, 2025.

[31] J. Satyadhar, “The transformative role of agentic GenAI in shaping workforce development and education in the US,” Iconic Research And Engineering Journals, vol. 8, no. 8, pp. 199–206, 2025.

[32] S. Joshi, “A comprehensive review of data pipelines and streaming for generative AI integration: Challenges, solutions, and future directions.”

[33] J. Satyadhar, “Retraining US workforce in the age of agentic gen AI: Role of prompt engineering and up-skilling initiatives,” International Journal of Advanced Research in Science, Communication and Technology (IJARSCT), vol. 5, no. 1, 2025.

[34] J. Satyadhar, “Generative AI and workforce development in the finance sector [ebook] ISBN:9798230127352,” 2025.

[35] J. Satyadhar, “Training US workforce for generative AI models and prompt engineering: ChatGPT, copilot, and gemini,” International Journal of Science, Engineering and Technology ISSN (Online): 2348-4098, vol. 13, no. 1, 2025.

[36] Joshi Satyadhar, “Quantitative foundations for integrating market, credit, and liquidity risk with generative AI,” https://www.preprints.org/, 2025.

[37] J. Satyadhar, “Introduction to vector databases for generative AI: Applications, performance, future projections, and cost considerations,” International Advanced Research Journal in Science, Engineering and Technology ISSN (O) 2393-8021, ISSN (P) 2394-1588, vol. 12, no. 2, pp. 79–93, 2025.

[38] Joshi Satyadhar, “Bridging the AI skills gap: Workforce training for financial services,” International Journal of Innovative Science and Research Technology, vol. 10, no. 2, pp. 1023–1030, 2025.

[39] J. Satyadhar, “Introduction to generative AI and DevOps: Synergies, challenges and applications,” International Journal of Advanced Research in Science Communication and Technology 2581-9429, Available: http://dx.doi.org/10.48175/IJARSCT-23634

[40] S. Joshi, “Workforce development in the finance sector, e-book, Draft2Digital, 2025,” Available: https://www.researchgate.net/publication/389091765_Generative_AI_and_Workforce_Development_in_the_Finance_Sector

[41] “5 Risks of Generative AI How to Mitigate Them in 2025,” AIMultiple. Accessed: Mar. 18, 2025. [Online]. Available: https://research.aimultiple.com/risks-of-generative-ai/

[42] R. Cruz, “Managing the Risks of Generative AI: Achieving Compliance Across Use Cases,” Smarsh. Nov. 2023. Accessed: Mar. 18, 2025. [Online]. Available: https://www.smarsh.com/blog/thought-leadership/Managing-risks-generative-AI-achieving-compliance-across-use-cases

[43] “Webinar: Model Risk Management for Financial Institutions in the Generative AI Era,” AIR. Accessed: Mar. 18, 2025. [Online]. Available: https://regulationinnovation.org/air-events/webinar-model-risk-management-for-financial-institutions-in-the-generative-ai-era/

[44] “How Generative AI in Finance Strengthening Risk & Compliance.” Accessed: Mar. 18, 2025. [Online]. Available: https://www.360factors.com/blog/generative-ai-finance-risk-compliance-management/

[45] “(27) Navigating Model Risk Management in the Age of AI LinkedIn.” Accessed: Mar. 18, 2025. [Online]. Available: https://www.linkedin.com/pulse/navigating-model-risk-management-age-ai-sri-krishnamurthy-cfa-cap-nqvbe/

[46] “Validating GenAI Models: Three Tips for AI Risk Management.” Dec. 2024. Accessed: Mar. 18, 2025. [Online]. Available: https://validmind.com/blog/validating-generative-ai-genai-models-tips-for-success/

[47] “Risks of Generative AI.” Accessed: Mar. 18, 2025. [Online]. Available: https://guidehouse.com/insights/advanced-solutions/2023/quantifying-the-risks-of-generative-ai

[48] M. Peter, “Generative AI and model risk management: New potential for the financial sector - KPMG in Germany,” KPMG. Jan. 2025. Accessed: Mar. 18, 2025. [Online]. Available: https://kpmg.com/de/en/home/insights/2024/12/generative-ai-and-model-risk-management.html

[49] A. K. Srivastava, “Model Risk in the Generative AI World: Meritorious or Detrimental?” Medium. Feb. 2025. Accessed: Mar. 18, 2025. [Online]. Available: https://medium.com/@abhayakant/model-risk-in-the-generative-ai-world-meritorious-or-detrimental-e757ea364872

[50] “The Impact of GenAI in Model Risk Management (MRM) - ValidMind.” Jun. 2024. Accessed: Mar. 18, 2025. [Online]. Available: https://validmind.com/blog/genai-and-model-risk-management-best-practices/

[51] B. Peterson, “Four Ways Banks Are Harnessing AI to Manage Model Risk,” Available: https://www.treliant.com/knowledge-center/brr-ft-four-ways-banks-are-harnessing-ai-to-manage-model-risk/

[52] EY. A. F. S. A. Partnerauthorurl:https://www.ey.com/en_us/people/susan-raffel, “Model risk management for AI and machine learning.” Accessed: Mar. 18, 2025. [Online]. Available: https://www.ey.com/en_us/insights/banking-capital-markets/understand-model-risk-management-for-ai-and-machine-learning

[53] “Emerton Data — Model Risk in the Age of Artificial Intelligence and Machine Learning.” Accessed: Mar. 18, 2025. [Online]. Available: https://www.emerton-data.com/insights/model-risk

[54] “Top 5 Ways Risk Management Teams Are Using Generative AI.” Accessed: Mar. 18, 2025. [Online]. Available: https://www.lexisnexis.com/community/insights/professional/b/industry-insights/posts/generative-ai-uses-risk-management

[55] “Model Risk Management, a true accelerator to corporate AI,” Databricks. May 2023. Accessed: Mar. 18, 2025. [Online]. Available: https://www.databricks.com/blog/model-risk-management-true-accelerator-corporate-ai

[56] “Transitioning from model risk management to AI risk management.” Accessed: Mar. 18, 2025. [Online]. Available: https://informaconnect.com/transitioning-from-model-risk-management-to-ai-risk-management/

[57] “The future of generative AI in banking McKinsey.” Accessed: Mar. 18, 2025. [Online]. Available: https://www.mckinsey.com/capabilities/risk-and-resilience/our-insights/how-generative-ai-can-help-banks-manage-risk-and-compliance

[58] A. Turner, “ERM Model Risk and AI,” ABA Banking Journal. Apr. 2024. Accessed: Mar. 18, 2025. [Online]. Available: https://bankingjournal.aba.com/2024/04/erm-model-risk-and-ai/

[59] “Adapting model risk management in the gen AI era,” Google Cloud Blog. Accessed: Mar. 18, 2025. [Online]. Available: https://cloud.google.com/blog/topics/financial-services/adapting-model-risk-management-in-the-gen-ai-era

[60] “What is model risk management? Domino Data Lab.” Accessed: Mar. 18, 2025. [Online]. Available: https://domino.ai/blog/what-is-model-risk-management-and-how-is-it-supported-by-enterprise-mlops

[61] “Generative AI Global Governance and the risk-based approach,” CERRE. Accessed: Mar. 18, 2025. [Online]. Available: https://cerre.eu/publications/generative-ai-global-governance-and-the-risk-based-approach/

[62] “Model Risk Management in an AI-Driven World [SS1/23].” Accessed: Mar. 18, 2025. [Online]. Available: https://risk.jaywing.com/news-views/model-risk-management-in-an-ai-driven-world-ss1-23/

[63] dwillis, “Harnessing AI to accelerate model risk management in FinTech,” FinTech Global. Dec. 2024. Accessed: Mar. 18, 2025. [Online]. Available: https://fintech.global/2024/12/19/harnessing-ai-to-accelerate-model-risk-management-in-fintech/

[64] H. Evans, “Generative AI Risks and Regulatory Issues,” Velvetech. Feb. 2024. Accessed: Mar. 18, 2025. [Online]. Available: https://www.velvetech.com/blog/generative-ai-risks-regulations/

[65] A. Markle, “The Future of AI Model Risk Management in Financial Institutions,” Empowered GRC Platform Streamline Governance, Risk & Compliance. Feb. 2025. Accessed: Mar. 18, 2025. [Online]. Available: https://empoweredsystems.com/blog/the-future-of-ai-model-risk-management-in-financial-institutions/

[66] “Financial services: 6 ways to support a generative AI risk management strategy.” Accessed: Mar. 18, 2025. [Online]. Available: https://deloitte.wsj.com/cfo/financial-services-6-ways-to-support-a-generative-ai-risk-management-strategy-95996453

[67] “Generative AI Masterclass Model Risk Management,” Available: https://h2o.ai/content/dam/h2o/en/marketing/documents/2024/

[68] “Generative AI for Risk Management.” Accessed: Mar. 18, 2025. [Online]. Available: https://www.xenonstack.com/blog/generative-ai-risk-management

[69] “Https://www.modelop.com/blog/five-ways-mitigate-risk-ai-models.” Accessed: Mar. 18, 2025. [Online]. Available: https://www.modelop.com/blog/five-ways-mitigate-risk-ai-models

[70] “(27) Generative AI Model Risk Management For Organizations LinkedIn.” Accessed: Mar. 18, 2025. [Online]. Available: https://www.linkedin.com/pulse/generative-ai-model-risk-management-organizations-govindhtech-gay5c/