Featured Product

    BoE Paper Examines Explainability of Machine Learning in Finance

    August 09, 2019

    BoE published a working paper that examines the explainability of machine learning in finance through an application to default risk analysis. Machine-learning-based predictive techniques are seeing increased adoption in a number of domains, including finance. However, due to their complexity, their predictions are often difficult to explain and validate. This is sometimes referred to as machine learning’s "black box" problem. The paper studies a machine learning model to predict mortgage defaults and proposes a framework for addressing the "black box" problem present in some machine learning applications.

    The paper addresses the explainability problem of artificial intelligence by studying the inputs and the outputs of a machine learning model.The approach is implemented by using the Quantitative Input Influence (QII) method of Datta et al (2016) in a real‑world example: a machine learning model to predict mortgage defaults. This method investigates the inputs and outputs of the model, but not its inner workings. It measures feature influences by intervening on inputs and estimating their Shapley values, representing the features’ average marginal contributions over all possible feature combinations. This method estimates key drivers of mortgage defaults such as the loan‑to‑value ratio and current interest rate, which are in line with the findings of the economics and finance literature. However, given the non‑linearity of machine learning model, explanations vary significantly for different groups of loans. The study uses clustering methods to arrive at groups of explanations for different areas of the input space. Finally, the authors conduct simulations on data that the model has not been trained or tested on. The main contribution is to develop a systematic analytical framework that could be used for approaching explainability questions in real world financial applications. The study concludes that notable model uncertainties do remain and stakeholders ought to be aware of these uncertainties.

    The paper highlights regulators as one category of stakeholders that might be interested in the workings of machine learning model of mortgage defaults of a bank to assess the riskiness of its loan book. A regulator could usefully consider an influence-based explainability approach implemented by the bank. The paper also highlights that, in such situations, it is still difficult to estimate how a complex model would behave out of sample, for instance, in stress-test scenarios where inputs are deliberately stretched. The paper shows that explainable artificial intelligence tools are an important addition to the data science toolkit, as they allow for better quality assurance of black box machine learning models. These tools can usefully complement other aspects of quality assurance, including various ways of model performance testing, understanding the properties of the data set and domain knowledge.

     

    Related Link: Working Paper

    Keywords: Europe, UK, Banking, Credit Risk, Machine Learning, Artificial Intelligence, Regtech, Model Explainability, Mortgage Default, BoE

    Related Articles
    News

    APRA Consults to Standardize Submission Date for Quarterly Reporting

    APRA proposed to standardize quarterly reporting due dates for authorized deposit-taking institutions. The proposed standardized due date is 35 calendar days after the last day of the reference quarter, which will create a 14-calendar-day extension for credit unions and building societies.

    November 08, 2019 WebPage Regulatory News
    News

    EBA Publishes Methodology and Draft Templates for Stress Tests in 2020

    EBA published a package for the 2020 EU-wide stress test exercise for banks.

    November 07, 2019 WebPage Regulatory News
    News

    FSB Publishes Summary of Plenary Meeting in Paris

    At the meeting, the Plenary reviewed vulnerabilities in the global financial system, fintech developments (including developments in the crypto-asset markets), ongoing work of FSB, and the work program for 2020.

    November 07, 2019 WebPage Regulatory News
    News

    EC Publishes Results of Fitness Check of Reporting Requirements in EU

    EC published results of the fitness check of supervisory reporting requirements in financial services legislation in EU.

    November 07, 2019 WebPage Regulatory News
    News

    BCBS Assesses NSFR and LE Rules in Argentina and China as Compliant

    BCBS published reports that assess the implementation of net stable funding ratio (NSFR) and large exposures, or LE, framework in Argentina and China.

    November 07, 2019 WebPage Regulatory News
    News

    HKMA Highlights Technology Initiatives at the Hong Kong FinTech Week

    HKMA co-organized, with InvestHK, the Hong Kong FinTech Week 2019, which was a five-day flagship fintech event that attracted thousands of attendees worldwide.

    November 06, 2019 WebPage Regulatory News
    News

    ECB Report on Fallback Provisions in Contracts Referencing EURIBOR

    ECB published a report, by private sector working group on euro risk-free rates, presenting recommendations for fallback provisions in contracts for cash products and derivative transactions referencing EURIBOR.

    November 06, 2019 WebPage Regulatory News
    News

    EC Regulation on Homogeneity of Underlying Exposures in Securitization

    EC published the Delegated Regulation 2019/1851 regarding the regulatory technical standards on the homogeneity of the underlying exposures in securitization.

    November 06, 2019 WebPage Regulatory News
    News

    FSI Examines Regulatory Approaches on Climate Risk Assessment

    The Financial Stability Institute (FSI) of BIS published a paper that examines the regulatory approaches being used for climate risk assessment in the insurance sector, in particular through enterprise risk management (ERM) frameworks.

    November 06, 2019 WebPage Regulatory News
    News

    SRB Launches Q&A Process for Resolution Reporting

    SRB has setup a dedicated solution for raising questions on the resolution reporting.

    November 06, 2019 WebPage Regulatory News
    RESULTS 1 - 10 OF 4112