Featured Product

    BoE Paper Examines Explainability of Machine Learning in Finance

    August 09, 2019

    BoE published a working paper that examines the explainability of machine learning in finance through an application to default risk analysis. Machine-learning-based predictive techniques are seeing increased adoption in a number of domains, including finance. However, due to their complexity, their predictions are often difficult to explain and validate. This is sometimes referred to as machine learning’s "black box" problem. The paper studies a machine learning model to predict mortgage defaults and proposes a framework for addressing the "black box" problem present in some machine learning applications.

    The paper addresses the explainability problem of artificial intelligence by studying the inputs and the outputs of a machine learning model.The approach is implemented by using the Quantitative Input Influence (QII) method of Datta et al (2016) in a real‑world example: a machine learning model to predict mortgage defaults. This method investigates the inputs and outputs of the model, but not its inner workings. It measures feature influences by intervening on inputs and estimating their Shapley values, representing the features’ average marginal contributions over all possible feature combinations. This method estimates key drivers of mortgage defaults such as the loan‑to‑value ratio and current interest rate, which are in line with the findings of the economics and finance literature. However, given the non‑linearity of machine learning model, explanations vary significantly for different groups of loans. The study uses clustering methods to arrive at groups of explanations for different areas of the input space. Finally, the authors conduct simulations on data that the model has not been trained or tested on. The main contribution is to develop a systematic analytical framework that could be used for approaching explainability questions in real world financial applications. The study concludes that notable model uncertainties do remain and stakeholders ought to be aware of these uncertainties.

    The paper highlights regulators as one category of stakeholders that might be interested in the workings of machine learning model of mortgage defaults of a bank to assess the riskiness of its loan book. A regulator could usefully consider an influence-based explainability approach implemented by the bank. The paper also highlights that, in such situations, it is still difficult to estimate how a complex model would behave out of sample, for instance, in stress-test scenarios where inputs are deliberately stretched. The paper shows that explainable artificial intelligence tools are an important addition to the data science toolkit, as they allow for better quality assurance of black box machine learning models. These tools can usefully complement other aspects of quality assurance, including various ways of model performance testing, understanding the properties of the data set and domain knowledge.

     

    Related Link: Working Paper

    Keywords: Europe, UK, Banking, Credit Risk, Machine Learning, Artificial Intelligence, Regtech, Model Explainability, Mortgage Default, BoE

    Related Articles
    News

    PRA Consults on Implementation of Certain Provisions of CRD5

    PRA, via the consultation paper CP12/20, proposed changes to its rules, supervisory statements, and statements of policy to implement certain elements of the Capital Requirements Directive (CRD5).

    July 31, 2020 WebPage Regulatory News
    News

    EIOPA Report Identifies Key Financial Stability Risks for Insurers

    EIOPA published the financial stability report that provides detailed quantitative and qualitative assessment of the key risks identified for the insurance and occupational pensions sectors in the European Economic Area.

    July 30, 2020 WebPage Regulatory News
    News

    EBA Publishes Risk Dashboard for First Quarter of 2020

    EBA published its risk dashboard for the first quarter of 2020 together with the results of the risk assessment questionnaire.

    July 30, 2020 WebPage Regulatory News
    News

    EBA Issues Updates on Stress Test Exercise for Banks in EU

    EBA announced that the next stress testing exercise is expected to be launched at the end of January 2021 and its results are to be published at the end of July 2021.

    July 30, 2020 WebPage Regulatory News
    News

    PRA Proposes Guidance Related to Matching Adjustment under Solvency II

    PRA published the consultation paper CP11/20 that sets out its expectations and guidance related to auditors’ work on the matching adjustment under Solvency II.

    July 30, 2020 WebPage Regulatory News
    News

    MAS Issues Guidance on Dividend Distributions by Banks

    MAS published a statement guidance on dividend distribution by banks.

    July 30, 2020 WebPage Regulatory News
    News

    APRA Updates Guidance on Capital Management for Banks

    APRA updated its capital management guidance for banks, particularly easing restrictions around paying dividends as institutions continue to manage the disruption caused by COVID-19 pandemic.

    July 29, 2020 WebPage Regulatory News
    News

    FSB Report Reviews Macro-Prudential Framework and Tools in Germany

    FSB published a report that reviews the progress on data collection for macro-prudential analysis and the availability and use of macro-prudential tools in Germany.

    July 29, 2020 WebPage Regulatory News
    News

    EBA Urges Firms to Finalize Preparations for End of Brexit Transition

    EBA issued a statement reminding financial institutions that the transition period between EU and UK will expire on December 31, 2020; this will end the possibility for the UK-based financial institutions to offer financial services to EU customers on a cross-border basis via passporting.

    July 29, 2020 WebPage Regulatory News
    News

    SRB on Operational Continuity in Resolution and FMI Contingency Plans

    SRB published guidance on operational continuity in resolution and financial market infrastructure (FMI) contingency plans.

    July 29, 2020 WebPage Regulatory News
    RESULTS 1 - 10 OF 5606