Featured Product

    BoE Paper Examines Explainability of Machine Learning in Finance

    August 09, 2019

    BoE published a working paper that examines the explainability of machine learning in finance through an application to default risk analysis. Machine-learning-based predictive techniques are seeing increased adoption in a number of domains, including finance. However, due to their complexity, their predictions are often difficult to explain and validate. This is sometimes referred to as machine learning’s "black box" problem. The paper studies a machine learning model to predict mortgage defaults and proposes a framework for addressing the "black box" problem present in some machine learning applications.

    The paper addresses the explainability problem of artificial intelligence by studying the inputs and the outputs of a machine learning model.The approach is implemented by using the Quantitative Input Influence (QII) method of Datta et al (2016) in a real‑world example: a machine learning model to predict mortgage defaults. This method investigates the inputs and outputs of the model, but not its inner workings. It measures feature influences by intervening on inputs and estimating their Shapley values, representing the features’ average marginal contributions over all possible feature combinations. This method estimates key drivers of mortgage defaults such as the loan‑to‑value ratio and current interest rate, which are in line with the findings of the economics and finance literature. However, given the non‑linearity of machine learning model, explanations vary significantly for different groups of loans. The study uses clustering methods to arrive at groups of explanations for different areas of the input space. Finally, the authors conduct simulations on data that the model has not been trained or tested on. The main contribution is to develop a systematic analytical framework that could be used for approaching explainability questions in real world financial applications. The study concludes that notable model uncertainties do remain and stakeholders ought to be aware of these uncertainties.

    The paper highlights regulators as one category of stakeholders that might be interested in the workings of machine learning model of mortgage defaults of a bank to assess the riskiness of its loan book. A regulator could usefully consider an influence-based explainability approach implemented by the bank. The paper also highlights that, in such situations, it is still difficult to estimate how a complex model would behave out of sample, for instance, in stress-test scenarios where inputs are deliberately stretched. The paper shows that explainable artificial intelligence tools are an important addition to the data science toolkit, as they allow for better quality assurance of black box machine learning models. These tools can usefully complement other aspects of quality assurance, including various ways of model performance testing, understanding the properties of the data set and domain knowledge.

     

    Related Link: Working Paper

    Keywords: Europe, UK, Banking, Credit Risk, Machine Learning, Artificial Intelligence, Regtech, Model Explainability, Mortgage Default, BoE

    Related Articles
    News

    BIS Examines Use of Big Data and Machine Learning at Central Banks

    BIS published a paper that provides an overview on the use of big data and machine learning in the central bank community.

    March 04, 2021 WebPage Regulatory News
    News

    APRA Finalizes Reporting Standard for Operational Risk Requirements

    APRA finalized the reporting standard ARS 115.0 on capital adequacy with respect to the standardized measurement approach to operational risk for authorized deposit-taking institutions in Australia.

    March 03, 2021 WebPage Regulatory News
    News

    ECB Publishes Guide for Determining Penalties for Regulatory Breaches

    ECB published a guide that outlines the principles and methods for calculating the penalties for regulatory breaches of prudential requirements by banks.

    March 02, 2021 WebPage Regulatory News
    News

    MAS Sets Out Good Practices to Manage Operational Risks Amid COVID

    MAS and The Association of Banks in Singapore (ABS) jointly issued a paper that sets out good practices for the management of operational and other risks stemming from new work arrangements adopted by financial institutions amid the COVID-19 pandemic.

    March 02, 2021 WebPage Regulatory News
    News

    ACPR Announces New Data Collection Application for Banks and Insurers

    ACPR announced that a new data collection application, called DLPP (Datalake for Prudential), for collecting banking and insurance prudential data will go into production on April 12, 2021.

    March 02, 2021 WebPage Regulatory News
    News

    BCB Maintains CCyB at 0%, Initiates First Cycle of Regulatory Sandbox

    BCB announced that the Financial Stability Committee decided to maintain the countercyclical capital buffer (CCyB) for Brazil at 0%, at least until the end of 2021.

    March 02, 2021 WebPage Regulatory News
    News

    EIOPA Launches Study on Non-Life Underwriting Risk in Internal Models

    EIOPA has launched a European-wide comparative study on non-life underwriting risk in internal models, also kicking-off of the data collection phase.

    March 01, 2021 WebPage Regulatory News
    News

    SRB Publishes Overview of Resolution Tools Available in Banking Union

    SRB published an overview of the resolution tools available in the Banking Union and their impact on a bank’s ability to maintain continuity of access to financial market infrastructure services in resolution.

    March 01, 2021 WebPage Regulatory News
    News

    EBA Consults on Pillar 3 Disclosure Standards for ESG Risks Under CRR

    EBA is consulting on the implementing technical standards for Pillar 3 disclosures on environmental, social, and governance (ESG) risks, as set out in requirements under Article 449a of the Capital Requirements Regulation (CRR).

    March 01, 2021 WebPage Regulatory News
    News

    ESAs Issue Advice on KPIs on Sustainability for Nonfinancial Reporting

    ESAs Issue Advice on KPIs on Sustainability for Nonfinancial Reporting

    March 01, 2021 WebPage Regulatory News
    RESULTS 1 - 10 OF 6655