Featured Product

    BoE Paper Examines Explainability of Machine Learning in Finance

    August 09, 2019

    BoE published a working paper that examines the explainability of machine learning in finance through an application to default risk analysis. Machine-learning-based predictive techniques are seeing increased adoption in a number of domains, including finance. However, due to their complexity, their predictions are often difficult to explain and validate. This is sometimes referred to as machine learning’s "black box" problem. The paper studies a machine learning model to predict mortgage defaults and proposes a framework for addressing the "black box" problem present in some machine learning applications.

    The paper addresses the explainability problem of artificial intelligence by studying the inputs and the outputs of a machine learning model.The approach is implemented by using the Quantitative Input Influence (QII) method of Datta et al (2016) in a real‑world example: a machine learning model to predict mortgage defaults. This method investigates the inputs and outputs of the model, but not its inner workings. It measures feature influences by intervening on inputs and estimating their Shapley values, representing the features’ average marginal contributions over all possible feature combinations. This method estimates key drivers of mortgage defaults such as the loan‑to‑value ratio and current interest rate, which are in line with the findings of the economics and finance literature. However, given the non‑linearity of machine learning model, explanations vary significantly for different groups of loans. The study uses clustering methods to arrive at groups of explanations for different areas of the input space. Finally, the authors conduct simulations on data that the model has not been trained or tested on. The main contribution is to develop a systematic analytical framework that could be used for approaching explainability questions in real world financial applications. The study concludes that notable model uncertainties do remain and stakeholders ought to be aware of these uncertainties.

    The paper highlights regulators as one category of stakeholders that might be interested in the workings of machine learning model of mortgage defaults of a bank to assess the riskiness of its loan book. A regulator could usefully consider an influence-based explainability approach implemented by the bank. The paper also highlights that, in such situations, it is still difficult to estimate how a complex model would behave out of sample, for instance, in stress-test scenarios where inputs are deliberately stretched. The paper shows that explainable artificial intelligence tools are an important addition to the data science toolkit, as they allow for better quality assurance of black box machine learning models. These tools can usefully complement other aspects of quality assurance, including various ways of model performance testing, understanding the properties of the data set and domain knowledge.

     

    Related Link: Working Paper

    Keywords: Europe, UK, Banking, Credit Risk, Machine Learning, Artificial Intelligence, Regtech, Model Explainability, Mortgage Default, BoE

    Related Articles
    News

    PRA Finalizes Policy on Prudent Person Principle Under Solvency II

    PRA published the policy statement PS14/20, which contains the supervisory statement SS1/20 and the feedback to responses to the consultation paper CP22/19 on expectations for investment by firms in accordance with the Prudent Person Principle, or PPP, as set out in the Investments Part of the PRA Rulebook.

    May 27, 2020 WebPage Regulatory News
    News

    EBA on Extending Large Exposure Limits for French Systemic Banks

    EBA published an opinion following the notification by the French macro-prudential authority, the Haut Conseil de Stabilité Financière (HCSF), of its intention to extend a measure introduced in 2018 on the use of Article 458(9) of the Capital Requirements Regulation (CRR).

    May 27, 2020 WebPage Regulatory News
    News

    ECB Highlights NPL Resolution as Key Policy Issue in Post-COVID Europe

    As part of a Research Bulletin on the recent policy-relevant work, ECB published an article that examines the lessons learned from past crises for nonperforming loan resolution in the post COVID-19 period.

    May 27, 2020 WebPage Regulatory News
    News

    RBNZ Publishes Financial Stability Report for May 2020

    RBNZ published the financial stability report for May 2020. This review of the financial system in the country highlights that the economic disruption associated with COVID-19 will present challenges to the financial system.

    May 27, 2020 WebPage Regulatory News
    News

    ECB Updates Guidance on Reporting of Securities Holdings Statistics

    ECB updated the guidance notes for reporting related to the statistics on holdings of securities by reporting banking groups (SHSG).

    May 26, 2020 WebPage Regulatory News
    News

    ECB Publishes Results of Financial Stability Review in May 2020

    ECB published results of the financial stability review in May 2020. Among other issues, the financial stability review assesses operations of the financial system so far during the COVID-19 pandemic.

    May 26, 2020 WebPage Regulatory News
    News

    Regulators and Private Sector Meet to Discuss COVID Policy Responses

    Financial policymakers and international standard-setters met virtually with private-sector executives to discuss international policy responses to COVID-19 pandemic.

    May 26, 2020 WebPage Regulatory News
    News

    ESMA Responds to IASB Consultation on Interest Rate Benchmark Reform

    ESMA published a letter responding to IASB on the exposure draft on the phase 2 of the interest rate benchmark reform.

    May 25, 2020 WebPage Regulatory News
    News

    HKMA Consults on Supervisory Policy for OTC Derivatives Transactions

    HKMA is consulting on revisions to the Supervisory Policy Manual module CR-G-14 on margin and other risk mitigation standards for non-centrally cleared over-the-counter (OTC) derivatives transactions.

    May 25, 2020 WebPage Regulatory News
    News

    EBA Examines Impact of COVID Crisis on Banking Sector in EU

    EBA published thematic note presenting a preliminary assessment of the impact of COVID-19 outbreak on the banking sector in EU.

    May 25, 2020 WebPage Regulatory News
    RESULTS 1 - 10 OF 5221