How can banks measure the success of their stress testing efforts? This article explores where banks can look for the “alpha” in stress testing – that is, how they can measure the performance of their stress testing programs, identify weaknesses, and make the process more efficient and effective.
In sports, a team’s success is measured by winning percentages, and an individual player’s, by statistics such as batting average, yards gained, or points scored. In business, success is measured by profitability. For banks, that specifically means net profit from fees and interest and return on investments.
Portfolio managers use the concept of “alpha” to gauge their performance, which is a measure of how much better their return is versus a benchmark index. For example, if a benchmark index increases five percent, but a mutual fund generates a return of seven percent over the same period, that fund would have an alpha of two percent, meaning that it performed that much better than the underlying index.
But how can we measure the performance of a bank’s stress testing efforts? Although it’s not as simple as measuring a batting average, there are some best practices that banks can adopt to help them find quantifiable “alpha factors.”
For US banks subject to the Dodd-Frank Act Stress Test (DFAST), the results are pass/fail for each bank; that is, for the banks that pass the test there is no merit to being first, second, third, or last, for that matter.
If stress testing were like a ski competition, each skier (in this case, each bank) would race down their own individual course, there would be no clock, and each would be declared a winner if they just made it safely to the bottom of the course.
To assess a bank’s stress testing performance, we need to measure how well the bank’s stress test program succeeded beyond simply passing the test. Rather than compare one bank’s performance to another’s, it makes more sense to compare the bank’s own performance from one year to the next.
To quantify how a bank’s program is improving (taking for granted that it passed the DFAST test), we need to find the “alpha” in stress testing.
In terms of an organizational framework, a typical stress testing process involves:
- Gathering data from across the firm – not just finance, risk, and treasury, but all lines of business
- Preparing an initial balance sheet using the jump-off data
- Forecasting what the balance sheet will look like in the future in a variety of economic scenarios using models for projections of losses, net income, pre-provision net revenue, cash flows, and other elements of the balance sheet
- Incorporating proposed capital plans, overlays, and expert judgment
- Preparing reports and supporting documentation to fully explain how the forecasts were derived
Developing and running a stress testing process is a daunting challenge for most banks. Multiple regulatory reforms with complex oversight and compliance guidelines have added to the already difficult challenges of risk management for financial institutions.
Government urgency to ensure that financial systems are safe and stable has prompted continued enhancements to the relatively new regulations even as they widen in scope. This heightened regulatory expectation and intense scrutiny come at a time when organizations are already under pressure to improve their profitability and establish a competitive edge in a low-return market environment.
As a result, stress testing budgets at most financial institutions have soared. These regulations have one characteristic in common: They require that financial institutions set up processes and systems to manage an ever-growing amount of data and oversee an enterprise-wide exercise. Organizations must access, validate, and reconcile data from across the enterprise, including all geographies, portfolios, and instruments, irrespective of the origin of the data.
On top of these data aggregation challenges, firms need to widen the scope and improve the accuracy and governance of their models and estimation processes for generating the forecasts needed for the stress tests. These challenges are straining firm resources even further.
To put the required effort in perspective: We believe that the largest US banks are running their stress testing programs year round, with upwards of four or five hundred full-time-equivalent resources engaged in the program for much of that time.1
Thwarting the success of these efforts is the data itself. Data exists in many formats, such as paper records, desktop files, and other suboptimal data storage solutions, which adds to the difficulty of efficiently meeting reporting obligations. Organizations often need to build additional environments to pull data from different locations for reporting purposes, as shown in Figure 2.
Primitive data systems, which are characterized by inconsistent standards, data duplication, and missing and conflicting data, lead financial institutions to make major and potentially erroneous assumptions about how to reconcile data. Firms must also contend with the challenge of ensuring consistency across reporting dates and between different reports.
One of the principles put forth by the Basel Committee on Banking Supervision (BCBS) states that organizations should strive to determine a single authoritative source of risk data for each type of risk. A firm must create a structure and processes that can aggregate risk data in a way that is accurate, complete, and transparent for its senior management, board of directors, and regulators – enabling these stakeholders to make informed decisions.
A centralized datamart that connects different pieces of information is key to mitigating the challenges of data management and can clear a path for banks to determine quantifiable stress testing performance measures.
The first step to passing regulatory requirements is to compile the numbers. The next is to ensure the transparency of the process, so that the bank can explain easily and clearly where the numbers came from and how the forecasts were derived. The final major step is to make sure the process is auditable: The bank needs to be able to make sure the process can be audited to show how everything came together, with clear and detailed documentation showing what estimates and models were used, what parameters and defaults were included, how they were validated, how overlays and expert judgment were applied, and how all decisions were made throughout the process.
The tactical approaches many organizations use to meet such increasingly complex regulatory and accounting requirements don’t necessarily include the strategic investments critical to delivering a more sustainable and cost-efficient business model. Compliance strategies that address only current needs could lead to unintended downstream consequences and additional costs. As we’ve shown, this process isn’t straightforward; it is iterative and complicated, and requires substantial time and labor, all of which makes it hard for banks to see beyond the primary goal of satisfying regulatory requirements and thus to realize secondary benefits or find alpha in the process.
Making the process repeatable, in addition to streamlining it, will minimize the time and cost of the exercise. It also allows for more what-if and sensitivity analyses, which can provide more clarity into the bank’s forecasts and improve its test results.
Stress testing should be seen within the wider context of the efforts banks put into improving their risk management capabilities. Banks face a number of common challenges, including the following:
- Significantly enhancing data and systems
- Improving risk governance and board oversight
- Integrating approaches to risk management
- Enhancing stress testing methodology
- Changing stress testing processes and operating models
- Extending reporting capability
Ideally, overcoming these challenges would allow banks to derive value from their stress testing programs beyond merely responding to regulatory requirements and passing tests – but this is seldom the case. Last year, we conducted surveys of large and mid-sized banks in the US. As Figure 4 shows, one question we asked was how the banks were using their stress testing programs.
When we asked the banks what they were using their stress testing results for, they all said regulatory compliance; most also said capital planning. But the degree to which banks use stress testing results quickly drops off after that. Not only did fewer banks mention that they were using stress testing for other purposes, such as financial planning and budgeting, but the ones that did were much less emphatic about how they used stress testing in these areas compared to compliance.
Regulators have indicated they would like to see banks use stress testing for other things, like risk appetite definition, limits, and risk management in general. Unfortunately, there seems to be a long way to go before banks start incorporating stress testing into their business in these areas. Many simply don’t have the time or resources to go beyond the regulatory requirements.
If risk managers did see ways to derive insights from stress testing that could help them run the bank, this would be a form of alpha. But for the most part this isn’t the case, and any value realized beyond satisfying compliance goals would thus be hard to quantify.
Making the process more efficient – easier and less time-consuming – will mean that stress testing takes up less of a bank’s resources. So perhaps this is where banks can start to look for alpha. Saving time for key resources by making the stress testing process more business-as-usual means a bank’s key people can stop “working for the regulators” and have more time to be creative and focus on the business of the bank.
Developing automated, well-governed processes for stress testing will lead to better results with more transparency, auditability and repeatability, and in less time. Banks will be even better positioned to achieve the primary goal of compliance, with less cost and effort.
Integrating the best practices outlined in this article can help banks streamline their processes, freeing up time for key resources. That is, they may change their stress testing race course, making it easier and faster to run through, and find their stress testing alpha.
1 Jamie Dimon, Annual letter to shareholders, p12, April 9, 2015. He mentioned the cost of stress testing at JP Morgan.
Juan M. Licari, PhD, is Chief International Economist with Moody's Analytics. As the Head of Economic and Credit Research in EMEA, APAC and Latin America, Juan and his team specialize in generating alternative macroeconomic forecasts and building econometric tools to model credit risk portfolios.
Focuses on helping financial institutions improve their data management practices and capabilities for enhanced risk management, business value, and regulatory compliance.
Previous ArticleEffect of Credit Deterioration on Regulatory Capital Risk Weights for Structured Finance Securities
Most banks are able to stand up to quantitative stress testing and even prove their capital adequacy. But many organizations lack a streamlined process that allows them to run stress tests with ease and control. This article outlines a five-step process that will help banks maximize their stress testing investment.
May 2015 WebPage Greg Clemens, Mark McKenna
Moody’s Analytics surveyed our banking clients to help better understand how they currently use information technology to improve the efficiency of their stress testing processes. In this article, we present our key findings regarding the current best practices to be more efficient in the future.
November 2014 WebPage Greg Clemens, Michael Richitelli
This article discusses the Moody’s Analytics Banking Lab (MABL), which provides a virtual and immersive environment to explore the possibilities of any stress test. It is a place to see what a transparent, auditable, and integrated stress testing solution looks like.
November 2014 WebPage Greg Clemens
This article provides an update on the future of the regulatory stress tests and their industry impact, and how banks can best address the challenges to create an optimal program that generates business value.
WebPage Greg Clemens