Featured Product

    BoE Paper Examines Explainability of Machine Learning in Finance

    August 09, 2019

    BoE published a working paper that examines the explainability of machine learning in finance through an application to default risk analysis. Machine-learning-based predictive techniques are seeing increased adoption in a number of domains, including finance. However, due to their complexity, their predictions are often difficult to explain and validate. This is sometimes referred to as machine learning’s "black box" problem. The paper studies a machine learning model to predict mortgage defaults and proposes a framework for addressing the "black box" problem present in some machine learning applications.

    The paper addresses the explainability problem of artificial intelligence by studying the inputs and the outputs of a machine learning model.The approach is implemented by using the Quantitative Input Influence (QII) method of Datta et al (2016) in a real‑world example: a machine learning model to predict mortgage defaults. This method investigates the inputs and outputs of the model, but not its inner workings. It measures feature influences by intervening on inputs and estimating their Shapley values, representing the features’ average marginal contributions over all possible feature combinations. This method estimates key drivers of mortgage defaults such as the loan‑to‑value ratio and current interest rate, which are in line with the findings of the economics and finance literature. However, given the non‑linearity of machine learning model, explanations vary significantly for different groups of loans. The study uses clustering methods to arrive at groups of explanations for different areas of the input space. Finally, the authors conduct simulations on data that the model has not been trained or tested on. The main contribution is to develop a systematic analytical framework that could be used for approaching explainability questions in real world financial applications. The study concludes that notable model uncertainties do remain and stakeholders ought to be aware of these uncertainties.

    The paper highlights regulators as one category of stakeholders that might be interested in the workings of machine learning model of mortgage defaults of a bank to assess the riskiness of its loan book. A regulator could usefully consider an influence-based explainability approach implemented by the bank. The paper also highlights that, in such situations, it is still difficult to estimate how a complex model would behave out of sample, for instance, in stress-test scenarios where inputs are deliberately stretched. The paper shows that explainable artificial intelligence tools are an important addition to the data science toolkit, as they allow for better quality assurance of black box machine learning models. These tools can usefully complement other aspects of quality assurance, including various ways of model performance testing, understanding the properties of the data set and domain knowledge.


    Related Link: Working Paper

    Keywords: Europe, UK, Banking, Credit Risk, Machine Learning, Artificial Intelligence, Regtech, Model Explainability, Mortgage Default, BoE

    Related Articles

    FINMA Approves Merger of Credit Suisse and UBS

    The Swiss Financial Market Supervisory Authority (FINMA) has approved the takeover of Credit Suisse by UBS.

    March 21, 2023 WebPage Regulatory News

    BOE Sets Out Its Thinking on Regulatory Capital and Climate Risks

    The Bank of England (BOE) published a working paper that aims to understand the climate-related disclosures of UK financial institutions.

    March 13, 2023 WebPage Regulatory News

    OSFI Finalizes on Climate Risk Guideline, Issues Other Updates

    The Office of the Superintendent of Financial Institutions (OSFI) is seeking comments, until May 31, 2023, on the draft guideline on culture and behavior risk, with final guideline expected by the end of 2023.

    March 12, 2023 WebPage Regulatory News

    APRA Assesses Macro-Prudential Policy Settings, Issues Other Updates

    The Australian Prudential Regulation Authority (APRA) published an information paper that assesses its macro-prudential policy settings aimed at promoting stability at a systemic level.

    March 07, 2023 WebPage Regulatory News

    BIS Paper Examines Impact of Greenhouse Gas Emissions on Lending

    BIS issued a paper that investigates the effect of the greenhouse gas, or GHG, emissions of firms on bank loans using bank–firm matched data of Japanese listed firms from 2006 to 2018.

    March 03, 2023 WebPage Regulatory News

    HMT Mulls Alignment of Ring-Fencing and Resolution Regimes for Banks

    The HM Treasury (HMT) is seeking evidence, until May 07, 2023, on practicalities of aligning the ring-fencing and the banking resolution regimes for banks.

    March 02, 2023 WebPage Regulatory News

    MFSA Sets Out Supervisory Priorities, Issues Reporting Updates

    The Malta Financial Services Authority (MFSA) outlined its supervisory priorities for 2023

    March 02, 2023 WebPage Regulatory News

    German Regulators Issue Multiple Reporting Updates for Banks

    Deutsche Bundesbank published the nationally deactivated validation rules for the German Commercial Code (HGB) users on the taxonomy 3.2, which became valid from December 31, 2022

    March 02, 2023 WebPage Regulatory News

    BCBS Report Examines Impact of Basel III Framework for Banks

    The Basel Committee on Banking Supervision (BCBS) published results of the Basel III monitoring exercise based on the June 30, 2022 data.

    February 28, 2023 WebPage Regulatory News

    PRA Consults on Prudential Rules for "Simpler-Regime" Firms

    Among the recent regulatory updates from UK authorities, a key development is the first-phase consultation, from the Prudential Regulation Authority (PRA), on simplifications to the prudential framework that would apply to the simpler-regime firms.

    February 28, 2023 WebPage Regulatory News
    RESULTS 1 - 10 OF 8806