Featured Product

    BoE Paper Examines Explainability of Machine Learning in Finance

    August 09, 2019

    BoE published a working paper that examines the explainability of machine learning in finance through an application to default risk analysis. Machine-learning-based predictive techniques are seeing increased adoption in a number of domains, including finance. However, due to their complexity, their predictions are often difficult to explain and validate. This is sometimes referred to as machine learning’s "black box" problem. The paper studies a machine learning model to predict mortgage defaults and proposes a framework for addressing the "black box" problem present in some machine learning applications.

    The paper addresses the explainability problem of artificial intelligence by studying the inputs and the outputs of a machine learning model.The approach is implemented by using the Quantitative Input Influence (QII) method of Datta et al (2016) in a real‑world example: a machine learning model to predict mortgage defaults. This method investigates the inputs and outputs of the model, but not its inner workings. It measures feature influences by intervening on inputs and estimating their Shapley values, representing the features’ average marginal contributions over all possible feature combinations. This method estimates key drivers of mortgage defaults such as the loan‑to‑value ratio and current interest rate, which are in line with the findings of the economics and finance literature. However, given the non‑linearity of machine learning model, explanations vary significantly for different groups of loans. The study uses clustering methods to arrive at groups of explanations for different areas of the input space. Finally, the authors conduct simulations on data that the model has not been trained or tested on. The main contribution is to develop a systematic analytical framework that could be used for approaching explainability questions in real world financial applications. The study concludes that notable model uncertainties do remain and stakeholders ought to be aware of these uncertainties.

    The paper highlights regulators as one category of stakeholders that might be interested in the workings of machine learning model of mortgage defaults of a bank to assess the riskiness of its loan book. A regulator could usefully consider an influence-based explainability approach implemented by the bank. The paper also highlights that, in such situations, it is still difficult to estimate how a complex model would behave out of sample, for instance, in stress-test scenarios where inputs are deliberately stretched. The paper shows that explainable artificial intelligence tools are an important addition to the data science toolkit, as they allow for better quality assurance of black box machine learning models. These tools can usefully complement other aspects of quality assurance, including various ways of model performance testing, understanding the properties of the data set and domain knowledge.

     

    Related Link: Working Paper

    Keywords: Europe, UK, Banking, Credit Risk, Machine Learning, Artificial Intelligence, Regtech, Model Explainability, Mortgage Default, BoE

    Related Articles
    News

    FSB Sets Out Effective Practices for Cyber Incident Recovery

    FSB finalized the toolkit of effective practices to assist financial institutions in their cyber incident response and recovery activities.

    October 19, 2020 WebPage Regulatory News
    News

    HKMA Urges Early Action for Adherence to IBOR Fallbacks Protocol

    HKMA urged authorized institutions to take early action to adhere to the IBOR Fallbacks Protocol, which ISDA is expected to publish soon.

    October 16, 2020 WebPage Regulatory News
    News

    FSB Sets Out Roadmap for Transition to Alternative Reference Rates

    FSB published a global transition roadmap for London Inter-bank Offered Rate (LIBOR).

    October 16, 2020 WebPage Regulatory News
    News

    HM Treasury Publishes Response to Proposal on BRRD2 Transposition

    HM Treasury published a document that summarizes the responses received from a consultation on the approach of UK to transposition of the revised Bank Resolution and Recovery Directive (BRRD2).

    October 15, 2020 WebPage Regulatory News
    News

    HM Treasury Publishes Response to Proposal on CRD5 Transposition

    HM Treasury published the government response to the feedback received on the consultation for updating the prudential regime of UK before the end of the Brexit transition period.

    October 15, 2020 WebPage Regulatory News
    News

    BoE Publishes Reporting Schedule for Statistical Returns

    In a recent statistical notice, BoE announced publication of the reporting schedule for statistical returns for 2021.

    October 15, 2020 WebPage Regulatory News
    News

    EC Welcomes Declaration by Member States on EU Cloud Federation

    EC welcomed the joint declaration by 25 EU member states on building the next generation of cloud in Europe.

    October 15, 2020 WebPage Regulatory News
    News

    MAS Amends Notice on Issuance of Covered Bonds by Banks in Singapore

    MAS published amendments to Notice 648 on the issuance of covered bonds by banks incorporated in Singapore.

    October 15, 2020 WebPage Regulatory News
    News

    FDIC Selects Technology Companies for Rapid Prototyping Competition

    FDIC has selected 14 technology companies—including Accenture Federal Services, LLC, Fed Reporter, Inc, and S&P Global Market Intelligence, LLC—for inclusion in the next phase of the rapid prototyping competition.

    October 15, 2020 WebPage Regulatory News
    News

    GLEIF Defines New Validation Agent Role for Financial Institutions

    GLEIF announced that financial institutions worldwide can realize a variety of cost, efficiency, and customer experience benefits by assuming a new “validation agent” role within the Global Legal Entity Identifier (LEI) System.

    October 15, 2020 WebPage Regulatory News
    RESULTS 1 - 10 OF 5979