Featured Product

    BoE Paper Examines Explainability of Machine Learning in Finance

    August 09, 2019

    BoE published a working paper that examines the explainability of machine learning in finance through an application to default risk analysis. Machine-learning-based predictive techniques are seeing increased adoption in a number of domains, including finance. However, due to their complexity, their predictions are often difficult to explain and validate. This is sometimes referred to as machine learning’s "black box" problem. The paper studies a machine learning model to predict mortgage defaults and proposes a framework for addressing the "black box" problem present in some machine learning applications.

    The paper addresses the explainability problem of artificial intelligence by studying the inputs and the outputs of a machine learning model.The approach is implemented by using the Quantitative Input Influence (QII) method of Datta et al (2016) in a real‑world example: a machine learning model to predict mortgage defaults. This method investigates the inputs and outputs of the model, but not its inner workings. It measures feature influences by intervening on inputs and estimating their Shapley values, representing the features’ average marginal contributions over all possible feature combinations. This method estimates key drivers of mortgage defaults such as the loan‑to‑value ratio and current interest rate, which are in line with the findings of the economics and finance literature. However, given the non‑linearity of machine learning model, explanations vary significantly for different groups of loans. The study uses clustering methods to arrive at groups of explanations for different areas of the input space. Finally, the authors conduct simulations on data that the model has not been trained or tested on. The main contribution is to develop a systematic analytical framework that could be used for approaching explainability questions in real world financial applications. The study concludes that notable model uncertainties do remain and stakeholders ought to be aware of these uncertainties.

    The paper highlights regulators as one category of stakeholders that might be interested in the workings of machine learning model of mortgage defaults of a bank to assess the riskiness of its loan book. A regulator could usefully consider an influence-based explainability approach implemented by the bank. The paper also highlights that, in such situations, it is still difficult to estimate how a complex model would behave out of sample, for instance, in stress-test scenarios where inputs are deliberately stretched. The paper shows that explainable artificial intelligence tools are an important addition to the data science toolkit, as they allow for better quality assurance of black box machine learning models. These tools can usefully complement other aspects of quality assurance, including various ways of model performance testing, understanding the properties of the data set and domain knowledge.

     

    Related Link: Working Paper

    Keywords: Europe, UK, Banking, Credit Risk, Machine Learning, Artificial Intelligence, Regtech, Model Explainability, Mortgage Default, BoE

    Related Articles
    News

    PRA Amends Pillar 2 Capital Framework for Banks

    PRA published the policy statement PS2/20 that contains the final amendments to the Pillar 2 framework and provides feedback to responses to the consultation paper CP5/19 on updates related to Pillar 2 capital framework.

    January 23, 2020 WebPage Regulatory News
    News

    FED Proposes to Revise Information Collection Under Market Risk Rule

    FED proposed to revise and extend, for three years, FR 4201, which is the information collection under the market risk capital rule.

    January 22, 2020 WebPage Regulatory News
    News

    MAS Amends Notices on Minimum Liquid Asset Requirements for Banks

    MAS published amendments to Notices 1015, 613, and 649 related to the minimum liquid assets (MLA) requirements.

    January 21, 2020 WebPage Regulatory News
    News

    APRA Publishes Submission on Fintech and Regtech

    APRA published its submission, to the Senate Select Committee, on financial technology and regulatory technology.

    January 21, 2020 WebPage Regulatory News
    News

    ECB Consults on Guideline on Threshold for Credit Obligations Past Due

    ECB published a draft guideline, along with the frequently asked questions (FAQs), on the definition of the materiality threshold for credit obligations past due for less significant institutions.

    January 20, 2020 WebPage Regulatory News
    News

    BIS Discusses Role of Central Banks in Addressing Climate Change Risks

    BIS published a book that reviews ways of addressing the climate change risks within the financial stability mandate of central banks.

    January 19, 2020 WebPage Regulatory News
    News

    FSB Report Examines Global Nonbank Financial Intermediation Activity

    FSB published the ninth annual report examining the global non-bank financial intermediation activity.

    January 19, 2020 WebPage Regulatory News
    News

    OSFI Publishes Instruction Guide on Solvency Information Return

    OSFI published an instruction guide to assist administrators of pension plans in completing the Solvency Information Return that is required to be filed with OSFI.

    January 17, 2020 WebPage Regulatory News
    News

    EU Amends IFRS 9 Rule, Changes Concern Interest Rate Benchmark Reforms

    EU published Regulation 2020/34 regarding the International Accounting Standard (IAS) 39 and International Financial Reporting Standards (IFRS) 7 and 9.

    January 16, 2020 WebPage Regulatory News
    News

    FDIC and OCC Issue Statement on Heightened Cyber Security Risk

    In response to the heightened cyber-security risk facing the financial services industry and other critical business sectors, FDIC and OCC issued an interagency statement on heightened cyber-security risk.

    January 16, 2020 WebPage Regulatory News
    RESULTS 1 - 10 OF 4515