Featured Product

    BoE Paper Examines Explainability of Machine Learning in Finance

    August 09, 2019

    BoE published a working paper that examines the explainability of machine learning in finance through an application to default risk analysis. Machine-learning-based predictive techniques are seeing increased adoption in a number of domains, including finance. However, due to their complexity, their predictions are often difficult to explain and validate. This is sometimes referred to as machine learning’s "black box" problem. The paper studies a machine learning model to predict mortgage defaults and proposes a framework for addressing the "black box" problem present in some machine learning applications.

    The paper addresses the explainability problem of artificial intelligence by studying the inputs and the outputs of a machine learning model.The approach is implemented by using the Quantitative Input Influence (QII) method of Datta et al (2016) in a real‑world example: a machine learning model to predict mortgage defaults. This method investigates the inputs and outputs of the model, but not its inner workings. It measures feature influences by intervening on inputs and estimating their Shapley values, representing the features’ average marginal contributions over all possible feature combinations. This method estimates key drivers of mortgage defaults such as the loan‑to‑value ratio and current interest rate, which are in line with the findings of the economics and finance literature. However, given the non‑linearity of machine learning model, explanations vary significantly for different groups of loans. The study uses clustering methods to arrive at groups of explanations for different areas of the input space. Finally, the authors conduct simulations on data that the model has not been trained or tested on. The main contribution is to develop a systematic analytical framework that could be used for approaching explainability questions in real world financial applications. The study concludes that notable model uncertainties do remain and stakeholders ought to be aware of these uncertainties.

    The paper highlights regulators as one category of stakeholders that might be interested in the workings of machine learning model of mortgage defaults of a bank to assess the riskiness of its loan book. A regulator could usefully consider an influence-based explainability approach implemented by the bank. The paper also highlights that, in such situations, it is still difficult to estimate how a complex model would behave out of sample, for instance, in stress-test scenarios where inputs are deliberately stretched. The paper shows that explainable artificial intelligence tools are an important addition to the data science toolkit, as they allow for better quality assurance of black box machine learning models. These tools can usefully complement other aspects of quality assurance, including various ways of model performance testing, understanding the properties of the data set and domain knowledge.

     

    Related Link: Working Paper

    Keywords: Europe, UK, Banking, Credit Risk, Machine Learning, Artificial Intelligence, Regtech, Model Explainability, Mortgage Default, BoE

    Related Articles
    News

    PRA Finalizes Supervisory Approach for Non-Systemic Banks in UK

    PRA published the policy statement PS8/21, which contains the final supervisory statement SS3/21 on the PRA approach to supervision of the new and growing non-systemic banks in UK.

    April 15, 2021 WebPage Regulatory News
    News

    EBA Finalizes Standards on Methods of Prudential Consolidation

    EBA published a report that sets out the final draft regulatory technical standards specifying the conditions according to which consolidation shall be carried out in line with Article 18 of the Capital Requirements Regulation (CRR).

    April 15, 2021 WebPage Regulatory News
    News

    EBA Updates List of Other Systemically Important Institutions in EU

    EBA updated the list of other systemically important institutions (O-SIIs) in EU.

    April 15, 2021 WebPage Regulatory News
    News

    BCBS Report Concludes Basel Risk Categories Can Capture Climate Risks

    BCBS published two reports that discuss transmission channels of climate-related risks to the banking system and the measurement methodologies of climate-related financial risks.

    April 14, 2021 WebPage Regulatory News
    News

    UK Authorities Welcome FSB Review of their Remuneration Regime

    UK Authorities (FCA and PRA) welcomed the findings of FSB peer review on the implementation of financial sector remuneration reforms in the UK.

    April 14, 2021 WebPage Regulatory News
    News

    PRA and FCA Letter on Addressing Risks from Use of Deposit Aggregators

    PRA and FCA jointly issued a letter that highlights risks associated with the increasing volumes of deposits that are placed with banks and building societies via deposit aggregators and how to mitigate these risks.

    April 14, 2021 WebPage Regulatory News
    News

    MFSA to Amend Banking Act and Rules in Coming Months to Transpose CRD5

    MFSA announced that amendments to the Banking Act, Subsidiary Legislation, and Banking Rules will be issued in the coming months, to transpose the Capital Requirements Directive (CRD5) into the national regulatory framework.

    April 14, 2021 WebPage Regulatory News
    News

    EC Delegated Regulation on Specialized Lending Exposures Under CRR

    EC finalized the Delegated Regulation 2021/598 that supplements the Capital Requirements Regulation (CRR or 575/2013) and lays out the regulatory technical standards for assigning risk-weights to specialized lending exposures.

    April 14, 2021 WebPage Regulatory News
    News

    OSFI Proposes to Enhance Assurance Expectations for Basel Returns

    OSFI launched a consultation to explore ways to enhance the OSFI assurance over capital, leverage, and liquidity returns for banks and insurers, given the increasing complexity arising from the evolving regulatory reporting framework due to IFRS 17 (Insurance Contracts) standard and Basel III reforms.

    April 13, 2021 WebPage Regulatory News
    News

    ECB Issues Results of Benchmarking Analysis of Recovery Plans of Banks

    ECB published results of the benchmarking analysis of the recovery plan cycle for 2019.

    April 13, 2021 WebPage Regulatory News
    RESULTS 1 - 10 OF 6858