BoE Paper Examines Explainability of Machine Learning in Finance
BoE published a working paper that examines the explainability of machine learning in finance through an application to default risk analysis. Machine-learning-based predictive techniques are seeing increased adoption in a number of domains, including finance. However, due to their complexity, their predictions are often difficult to explain and validate. This is sometimes referred to as machine learning’s "black box" problem. The paper studies a machine learning model to predict mortgage defaults and proposes a framework for addressing the "black box" problem present in some machine learning applications.
The paper addresses the explainability problem of artificial intelligence by studying the inputs and the outputs of a machine learning model.The approach is implemented by using the Quantitative Input Influence (QII) method of Datta et al (2016) in a real‑world example: a machine learning model to predict mortgage defaults. This method investigates the inputs and outputs of the model, but not its inner workings. It measures feature influences by intervening on inputs and estimating their Shapley values, representing the features’ average marginal contributions over all possible feature combinations. This method estimates key drivers of mortgage defaults such as the loan‑to‑value ratio and current interest rate, which are in line with the findings of the economics and finance literature. However, given the non‑linearity of machine learning model, explanations vary significantly for different groups of loans. The study uses clustering methods to arrive at groups of explanations for different areas of the input space. Finally, the authors conduct simulations on data that the model has not been trained or tested on. The main contribution is to develop a systematic analytical framework that could be used for approaching explainability questions in real world financial applications. The study concludes that notable model uncertainties do remain and stakeholders ought to be aware of these uncertainties.
The paper highlights regulators as one category of stakeholders that might be interested in the workings of machine learning model of mortgage defaults of a bank to assess the riskiness of its loan book. A regulator could usefully consider an influence-based explainability approach implemented by the bank. The paper also highlights that, in such situations, it is still difficult to estimate how a complex model would behave out of sample, for instance, in stress-test scenarios where inputs are deliberately stretched. The paper shows that explainable artificial intelligence tools are an important addition to the data science toolkit, as they allow for better quality assurance of black box machine learning models. These tools can usefully complement other aspects of quality assurance, including various ways of model performance testing, understanding the properties of the data set and domain knowledge.
Related Link: Working Paper
Keywords: Europe, UK, Banking, Credit Risk, Machine Learning, Artificial Intelligence, Regtech, Model Explainability, Mortgage Default, BoE
Previous Article
NBB Launches the 2019 Stress Testing Exercise for InsurersRelated Articles
BIS and Central Banks Experiment with GenAI to Assess Climate Risks
A recent report from the Bank for International Settlements (BIS) Innovation Hub details Project Gaia, a collaboration between the BIS Innovation Hub Eurosystem Center and certain central banks in Europe
Nearly 25% G-SIBs Commit to Adopting TNFD Nature-Related Disclosures
Nature-related risks are increasing in severity and frequency, affecting businesses, capital providers, financial systems, and economies.
Singapore to Mandate Climate Disclosures from FY2025
Singapore recently took a significant step toward turning climate ambition into action, with the introduction of mandatory climate-related disclosures for listed and large non-listed companies
SEC Finalizes Climate-Related Disclosures Rule
The U.S. Securities and Exchange Commission (SEC) has finalized the long-awaited rule that mandates climate-related disclosures for domestic and foreign publicly listed companies in the U.S.
EBA Proposes Standards Related to Standardized Credit Risk Approach
The European Banking Authority (EBA) has been taking significant steps toward implementing the Basel III framework and strengthening the regulatory framework for credit institutions in the EU
US Regulators Release Stress Test Scenarios for Banks
The U.S. regulators recently released baseline and severely adverse scenarios, along with other details, for stress testing the banks in 2024. The relevant U.S. banking regulators are the Federal Reserve Bank (FED), the Federal Deposit Insurance Corporation (FDIC), and the Office of the Comptroller of the Currency (OCC).
Asian Governments Aim for Interoperability in AI Governance Frameworks
The regulatory landscape for artificial intelligence (AI), including the generative kind, is evolving rapidly, with governments and regulators aiming to address the challenges and opportunities presented by this transformative technology.
EBA Proposes Operational Risk Standards Under Final Basel III Package
The European Union (EU) has been working on the final elements of Basel III standards, with endorsement of the Banking Package and the publication of the European Banking Authority (EBA) roadmap on Basel III implementation in December 2023.
EFRAG Proposes XBRL Taxonomy and Standard for Listed SMEs Under ESRS
The European Financial Reporting Advisory Group (EFRAG), which plays a crucial role in shaping corporate reporting standards in European Union (EU), is seeking comments, until May 21, 2024, on the Exposure Draft ESRS for listed SMEs.
ECB to Expand Climate Change Work in 2024-2025
Banking regulators worldwide are increasingly focusing on addressing, monitoring, and supervising the institutions' exposure to climate and environmental risks.