Featured Product

    BoE Paper Examines Explainability of Machine Learning in Finance

    August 09, 2019

    BoE published a working paper that examines the explainability of machine learning in finance through an application to default risk analysis. Machine-learning-based predictive techniques are seeing increased adoption in a number of domains, including finance. However, due to their complexity, their predictions are often difficult to explain and validate. This is sometimes referred to as machine learning’s "black box" problem. The paper studies a machine learning model to predict mortgage defaults and proposes a framework for addressing the "black box" problem present in some machine learning applications.

    The paper addresses the explainability problem of artificial intelligence by studying the inputs and the outputs of a machine learning model.The approach is implemented by using the Quantitative Input Influence (QII) method of Datta et al (2016) in a real‑world example: a machine learning model to predict mortgage defaults. This method investigates the inputs and outputs of the model, but not its inner workings. It measures feature influences by intervening on inputs and estimating their Shapley values, representing the features’ average marginal contributions over all possible feature combinations. This method estimates key drivers of mortgage defaults such as the loan‑to‑value ratio and current interest rate, which are in line with the findings of the economics and finance literature. However, given the non‑linearity of machine learning model, explanations vary significantly for different groups of loans. The study uses clustering methods to arrive at groups of explanations for different areas of the input space. Finally, the authors conduct simulations on data that the model has not been trained or tested on. The main contribution is to develop a systematic analytical framework that could be used for approaching explainability questions in real world financial applications. The study concludes that notable model uncertainties do remain and stakeholders ought to be aware of these uncertainties.

    The paper highlights regulators as one category of stakeholders that might be interested in the workings of machine learning model of mortgage defaults of a bank to assess the riskiness of its loan book. A regulator could usefully consider an influence-based explainability approach implemented by the bank. The paper also highlights that, in such situations, it is still difficult to estimate how a complex model would behave out of sample, for instance, in stress-test scenarios where inputs are deliberately stretched. The paper shows that explainable artificial intelligence tools are an important addition to the data science toolkit, as they allow for better quality assurance of black box machine learning models. These tools can usefully complement other aspects of quality assurance, including various ways of model performance testing, understanding the properties of the data set and domain knowledge.

     

    Related Link: Working Paper

    Keywords: Europe, UK, Banking, Credit Risk, Machine Learning, Artificial Intelligence, Regtech, Model Explainability, Mortgage Default, BoE

    Related Articles
    News

    FASB Proposes Taxonomy Changes Related to Topics 848 and 470

    FASB proposed taxonomy improvements for the proposed Accounting Standards Update on topic 848 on facilitation of effects of reference rate reform on financial reporting.

    September 16, 2019 WebPage Regulatory News
    News

    BoE Statement on Recalculating Transitional Measures Under Solvency II

    BoE notified that it will be willing to accept applications from firms to recalculate transitional measure on technical provisions (TMTP) as at September 30, 2019.

    September 16, 2019 WebPage Regulatory News
    News

    BoE Paper on Market-Implied Systemic Risk and Shadow Capital Adequacy

    BoE published a working paper that presents a forward-looking approach to measure systemic solvency risk.

    September 13, 2019 WebPage Regulatory News
    News

    HKMA Consults on Policy Module on Pillar 2 Supervisory Review Process

    HKMA is consulting on the revised Supervisory Policy Manual module CA-G-5 that sets out the HKMA approach to conducting the supervisory review process under Pillar 2.

    September 13, 2019 WebPage Regulatory News
    News

    PRA Publishes Waiver by Consent of Continuity of Access Rules

    PRA published a new waiver by consent to waive the Continuity of Access requirements contained in the Depositor Protection Part of the PRA Rulebook (DPP).

    September 13, 2019 WebPage Regulatory News
    News

    EBA Single Rulebook Q&A: Second Update for September 2019

    EBA updated the Single Rulebook question and answer (Q&A) tool with answers to three questions.

    September 13, 2019 WebPage Regulatory News
    News

    PRA Revises Branch Return and Updates Guidance for Regulatory Reports

    PRA published the policy statement PS17/19, which contains the final policy related to changes in the format and content of the Branch Return Form and reporting guidance.

    September 12, 2019 WebPage Regulatory News
    News

    ISDA Guide on Collateral Management Under Smart Derivatives Contracts

    ISDA published the third in a series of legal guidelines for smart derivatives contracts.

    September 12, 2019 WebPage Regulatory News
    News

    ESA Report Highlights Risks of No-Deal Brexit in EU Financial System

    ESAs published a Joint Committee report on risks and vulnerabilities in the EU financial system.

    September 12, 2019 WebPage Regulatory News
    News

    ECB Modifies New Targeted Longer-Term Refinancing Operations

    The Governing Council of ECB decided to modify some of the key parameters of the third series of targeted longer-term refinancing operations (TLTRO III) to preserve favorable bank lending conditions (Decision (EU) 2019/1558).

    September 12, 2019 WebPage Regulatory News
    RESULTS 1 - 10 OF 3819