Featured Product

    BoE Paper Examines Explainability of Machine Learning in Finance

    August 09, 2019

    BoE published a working paper that examines the explainability of machine learning in finance through an application to default risk analysis. Machine-learning-based predictive techniques are seeing increased adoption in a number of domains, including finance. However, due to their complexity, their predictions are often difficult to explain and validate. This is sometimes referred to as machine learning’s "black box" problem. The paper studies a machine learning model to predict mortgage defaults and proposes a framework for addressing the "black box" problem present in some machine learning applications.

    The paper addresses the explainability problem of artificial intelligence by studying the inputs and the outputs of a machine learning model.The approach is implemented by using the Quantitative Input Influence (QII) method of Datta et al (2016) in a real‑world example: a machine learning model to predict mortgage defaults. This method investigates the inputs and outputs of the model, but not its inner workings. It measures feature influences by intervening on inputs and estimating their Shapley values, representing the features’ average marginal contributions over all possible feature combinations. This method estimates key drivers of mortgage defaults such as the loan‑to‑value ratio and current interest rate, which are in line with the findings of the economics and finance literature. However, given the non‑linearity of machine learning model, explanations vary significantly for different groups of loans. The study uses clustering methods to arrive at groups of explanations for different areas of the input space. Finally, the authors conduct simulations on data that the model has not been trained or tested on. The main contribution is to develop a systematic analytical framework that could be used for approaching explainability questions in real world financial applications. The study concludes that notable model uncertainties do remain and stakeholders ought to be aware of these uncertainties.

    The paper highlights regulators as one category of stakeholders that might be interested in the workings of machine learning model of mortgage defaults of a bank to assess the riskiness of its loan book. A regulator could usefully consider an influence-based explainability approach implemented by the bank. The paper also highlights that, in such situations, it is still difficult to estimate how a complex model would behave out of sample, for instance, in stress-test scenarios where inputs are deliberately stretched. The paper shows that explainable artificial intelligence tools are an important addition to the data science toolkit, as they allow for better quality assurance of black box machine learning models. These tools can usefully complement other aspects of quality assurance, including various ways of model performance testing, understanding the properties of the data set and domain knowledge.

     

    Related Link: Working Paper

    Keywords: Europe, UK, Banking, Credit Risk, Machine Learning, Artificial Intelligence, Regtech, Model Explainability, Mortgage Default, BoE

    Related Articles
    News

    PRA Finalizes Approach to Supervision of International Banks

    In a recent Market Notice, the Bank of England (BoE) confirmed that green gilts will have equivalent eligibility to existing gilts in its market operations.

    July 26, 2021 WebPage Regulatory News
    News

    FCA Issues PS21/9 on Implementation of Investment Firms Regime

    The Financial Conduct Authority (FCA) published the policy statement PS21/9 on implementation of the Investment Firms Prudential Regime.

    July 26, 2021 WebPage Regulatory News
    News

    EBA Proposes Regulatory Standards to Identify Shadow Banking Entities

    The European Banking Authority (EBA) proposed regulatory technical standards that set out criteria for identifying shadow banking entities for the purpose of reporting large exposures.

    July 26, 2021 WebPage Regulatory News
    News

    IOSCO Proposes Recommendations on ESG Ratings and Data Providers

    The Board of the International Organization of Securities Commissions (IOSCO) proposed a set of recommendations on the environmental, social, and governance (ESG) ratings and data providers.

    July 26, 2021 WebPage Regulatory News
    News

    EC to Defer Application of SFDR Standards Till July 2022

    The European Commission (EC) announced plans to defer the application of 13 regulatory technical standards under the Sustainable Finance Disclosure Regulation (2019/2088) by six months, from January 01, 2022 to July 01, 2022.

    July 23, 2021 WebPage Regulatory News
    News

    EIOPA Consults on Reporting and Disclosures Under Solvency II

    The European Insurance and Occupational Pensions Authority (EIOPA) proposed to amend the supervisory statement on supervision of run-off undertakings that are subject to Solvency II regulation.

    July 23, 2021 WebPage Regulatory News
    News

    BoE Consults on Approach to Setting MREL, Publishes Bail-In Guidance

    The Bank of England (BoE) published a consultation paper on approach to setting minimum requirement for own funds and eligible liabilities (MREL), an operational guide on executing bail-in, and a statement from the Deputy Governor Dave Ramsden.

    July 22, 2021 WebPage Regulatory News
    News

    EBA Seeks Views on Proportionality Assessment Methodology

    The European Banking Authority (EBA) is seeking preliminary input on standardization of the proportionality assessment methodology for credit institutions and investment firms.

    July 22, 2021 WebPage Regulatory News
    News

    US Agencies Propose Changes to Call Reports and Instructions

    Certain regulatory authorities in the US are extending period for completion of the review of certain residential mortgage provisions and for publication of notice disclosing the determination of this review until December 20, 2021.

    July 22, 2021 WebPage Regulatory News
    News

    PRA Finalizes Rulebook Definition of Higher Paid Material Risk-Taker

    The Prudential Regulation Authority (PRA) published the policy statement PS18/21, which introduces an amendment in the definition of "higher paid material risk taker" in the Remuneration Part of the PRA Rulebook.

    July 21, 2021 WebPage Regulatory News
    RESULTS 1 - 10 OF 7293