Featured Product

    BoE Paper Examines Explainability of Machine Learning in Finance

    August 09, 2019

    BoE published a working paper that examines the explainability of machine learning in finance through an application to default risk analysis. Machine-learning-based predictive techniques are seeing increased adoption in a number of domains, including finance. However, due to their complexity, their predictions are often difficult to explain and validate. This is sometimes referred to as machine learning’s "black box" problem. The paper studies a machine learning model to predict mortgage defaults and proposes a framework for addressing the "black box" problem present in some machine learning applications.

    The paper addresses the explainability problem of artificial intelligence by studying the inputs and the outputs of a machine learning model.The approach is implemented by using the Quantitative Input Influence (QII) method of Datta et al (2016) in a real‑world example: a machine learning model to predict mortgage defaults. This method investigates the inputs and outputs of the model, but not its inner workings. It measures feature influences by intervening on inputs and estimating their Shapley values, representing the features’ average marginal contributions over all possible feature combinations. This method estimates key drivers of mortgage defaults such as the loan‑to‑value ratio and current interest rate, which are in line with the findings of the economics and finance literature. However, given the non‑linearity of machine learning model, explanations vary significantly for different groups of loans. The study uses clustering methods to arrive at groups of explanations for different areas of the input space. Finally, the authors conduct simulations on data that the model has not been trained or tested on. The main contribution is to develop a systematic analytical framework that could be used for approaching explainability questions in real world financial applications. The study concludes that notable model uncertainties do remain and stakeholders ought to be aware of these uncertainties.

    The paper highlights regulators as one category of stakeholders that might be interested in the workings of machine learning model of mortgage defaults of a bank to assess the riskiness of its loan book. A regulator could usefully consider an influence-based explainability approach implemented by the bank. The paper also highlights that, in such situations, it is still difficult to estimate how a complex model would behave out of sample, for instance, in stress-test scenarios where inputs are deliberately stretched. The paper shows that explainable artificial intelligence tools are an important addition to the data science toolkit, as they allow for better quality assurance of black box machine learning models. These tools can usefully complement other aspects of quality assurance, including various ways of model performance testing, understanding the properties of the data set and domain knowledge.

     

    Related Link: Working Paper

    Keywords: Europe, UK, Banking, Credit Risk, Machine Learning, Artificial Intelligence, Regtech, Model Explainability, Mortgage Default, BoE

    Related Articles
    News

    EC Adopts Financial Reporting Changes Arising from Benchmark Reforms

    EC published Regulation 2021/25 that addresses amendments related to the financial reporting consequences of replacement of the existing interest rate benchmarks with alternative reference rates.

    January 14, 2021 WebPage Regulatory News
    News

    BIS Bulletin Examines Key Elements of Policy Response to Cyber Risk

    BIS published a bulletin, or a note, that examines the cyber threat landscape in the context of the pandemic and discusses policies to reduce risks to financial stability.

    January 14, 2021 WebPage Regulatory News
    News

    HMT Updates List of Post-Brexit Equivalence Decisions in UK

    HM Treasury, also known as HMT, has updated the table containing the list of the equivalence decisions that came into effect in UK at the end of the transition period of its withdrawal from EU.

    January 14, 2021 WebPage Regulatory News
    News

    EBA Issues Erratum for Technical Package on Reporting Framework 3.0

    EBA published an erratum for technical package on phase 1 of the reporting framework 3.0.

    January 14, 2021 WebPage Regulatory News
    News

    APRA Publishes FAQ on Measurement of Credit Risk Weighted Assets

    APRA updated a frequently asked question (FAQ), for authorized deposit-taking institutions, on the measurement of credit risk weighted assets.

    January 14, 2021 WebPage Regulatory News
    News

    EBA Publishes Risk Dashboard for Third Quarter of 2020

    EBA published the quarterly risk dashboard, along with the results of the Risk Assessment Questionnaire survey among 60 banks and 15 market analysts.

    January 13, 2021 WebPage Regulatory News
    News

    ECB Analysis Shows Privacy as Biggest Concern in Use of Digital Euro

    ECB concluded the public consultation on the introduction of a digital euro in EU.

    January 13, 2021 WebPage Regulatory News
    News

    ECB Finalizes Guide on Supervisory Approach to Bank Consolidation

    ECB published a guide that sets out the supervisory approach to consolidation in the banking sector.

    January 12, 2021 WebPage Regulatory News
    News

    SRB Chair Outlines Work Priorities for 2021

    The SRB Chair Elke König published an article setting out work priorities for 2021.

    January 11, 2021 WebPage Regulatory News
    News

    FDIC Selects Companies to Compete in Final Phase of Tech Sprint

    FDIC has selected 11 technology companies—including BearingPoint, Fed Reporter, Inc, and S&P Global Market Intelligence, LLC—for inclusion in the third and final phase of the rapid prototyping competition.

    January 11, 2021 WebPage Regulatory News
    RESULTS 1 - 10 OF 6417