CCAR and DFAST Stress Testing Survey Insight

In May 2014, Moody’s Analytics hosted senior representatives from 39 banks for a series of stress testing roundtables for Comprehensive Capital Analysis and Review (CCAR) banks, Dodd-Frank Act Stress Test (DFAST) banks, and Canadian banks/Foreign Banking Organizations (FBOs). We have hosted these events for the past two years. As the universe of banks subject to the stress testing regulation has grown, so has the participation in our roundtables.

Three major themes emerged from the conversations across the various sessions:

1. Management judgment versus quantitative modeling: There is uncertainty on when to build a model for a process that has traditionally been forecasted using management judgment.
2. Data, data, data: There is a pressing need to better manage the data used in the stress tests, including auditing, augmenting, and aggregating that data.
3. Getting more out of the stress testing exercise: With the amount of time, effort, and money being spent on stress testing, banks would clearly like to use the results for multiple purposes. For most banks, stress testing is still about regulatory compliance, rather than improving their business decisions and risk practices.

The larger banks all agreed that it is hard to determine what to do first, given the multiple regulations in multiple jurisdictions. Banks of all sizes are still focusing a great deal on building or updating models for loss estimation to satisfy the regulatory compliance requirements, and data quality and granularity continue to be issues across most institutions. Improving infrastructure is desired, but the way forward is not always clear, as banks often don’t have well-defined roadmaps for longer term projects when they are faced with multiple Matters Requiring Attention (MRA) that need attention before the next submission. The time pressure, combined with a necessarily short-term view on improvements, makes it difficult for them to move beyond regulatory compliance and leverage their stress testing investments to help run their businesses.

Many of the smaller banks noted that there is little scalability to stress testing and that they have to go through many of the same steps as the larger banks. I was talking to a small bank located in the south not long ago and it struck me that their experience with stress testing was somewhat similar to my experience hosting a wedding at my house. Even with a small event, we still needed a tent, caterer, someone to officiate, seats, flowers, a band, food, and drinks.

The main steps for small bank stress testing are roughly similar to those required for large banks – from data work to economic scenario generation/regionalization, credit and cash flow modeling to reporting. Of course, scale and complexity add significant time and expense for larger institutions, but the minimum effort for smaller institutions is considerable.

One of the main differences is where the banks are with their loan-level datasets and the use of granular loan-level models. Only about a third of DFAST institutions use loan-level data for their modeling. That number should grow as banks improve their data sources and data capturing processes. Unsurprisingly, small banks are also not as concerned about bespoke economic modeling, reporting, and Pre-Provision Net Revenue (PPNR) modeling at this point. Of course, many chief risk officers and others at small banks agreed that things would quickly change based on specific regulatory feedback.

Another obvious difference is that the complexity is ramping up for the large banks, with risk-weighted asset (RWA) and liquidity rules on the horizon, while the smaller banks will not have to deal with those issues in the immediate future. Large banks must validate and document many more models, so they are confronted with a bigger challenge. With the increased complexity of regulatory requirements, documentation, and bespoke modeling, large banks are also anticipating a significant increase in their spending on stress testing. They generally agreed on an estimate of $10 million per year operating cost, and a technology spend of $30-$50 million on top of that.

On the other hand, smaller banks do not have the resources – neither cash nor staff – to match the kind of expenses the larger banks have incurred. We have observed some are spending proactively on data, modeling, and infrastructure, but for many this will be a longer incremental struggle.

Regardless of the size of the bank, nearly everyone mentioned model enhancement and loss estimation as major priorities. This admission was a bit surprising given that the large banks have been performing both already for many years. Across 30 different banks, large and small, credit modeling was the top priority for improvement in our surveys.

Banks that are starting out plan to build or acquire models, while banks that have been through the stress testing process look to improve or update their models. Banks that have primary models look for challenger models to augment and validate their forecasts.

Large and small banks also similarly ranked infrastructure, data, and process/workflow as the next important items. Smaller banks will focus on migrating from top-down, statistically-based models to bottom-up models with more granular data, but they will need to clean up their data and data infrastructure as a first step. The larger banks have done much of this already, so they are instead focusing more on implementing longer-term infrastructure improvements.

One item that is becoming a top priority is documentation. Regulators are consistently asking for more details and clearer explanations. The larger banks that have gone through the submission process and received feedback are already aware of this and plan to give documentation more attention.

We will provide more details after hosting our ongoing regulatory roundtables, but from our recent interactions they have consistently discussed a few themes:

• Capital plans and model methodologies still lack detail. Documentation needs to be improved across the board.
• Models and data used for stressed loss, revenue, and expense modeling need to be “tailored to the task.” In other words, use the right tool for the task.
• The process for review, challenge, and aggregation could be improved. Generally, banks need to audit the results and determine how they arrived at their conclusions.
• Operational risk loss estimation is challenging and may lend itself more naturally to scenario analysis (and management judgment) rather than correlation modeling.
• The narrative around the stress test will be more important, as will many of the qualitative aspects of the test.

The regulators want more details in banks’ capital policies. They also want better documentation for model assumptions and adjustments, and are generally looking for the banks to provide more details and clearer explanations, from their models through their idiosyncratic scenarios and capital plans.

Looking ahead, regulators have indicated they will evaluate a number of related areas, in addition to the ongoing themes listed earlier.

First, the overall emphasis will continue to shift from the quantitative stress test results to the qualitative capital plans. This was already evident in this year’s CCAR exercise, as there was only one quantitative failure, but there were four qualitative failures. Heading into next year’s stress tests, the likelihood of a quantitative failure will continue to go down, but the likelihood of qualitative failures will go up. As mentioned previously, the narrative around the stress test – that is, the story about the key assumptions, choices, adjustments, and overlays, etc. – will continue to be more important.

Second, the regulators wish to differentiate expectations, and are looking more closely at the specifics of each bank. They are putting less emphasis on the mandated Baseline, Adverse, and Severely Adverse scenarios, and more on the idiosyncratic scenarios and sensitivity analysis around the key assumptions the banks use for their internal stress tests.

Finally, the regulators intend to heighten expectations for the largest and most complex Bank Holding Companies (BHCs). This makes sense as the biggest BHCs have the largest potential impact on the financial system. This does not mean, however, that they plan to lower expectations or make it easier for the smaller banks.

Banks will spend a significant amount of money on infrastructure over the next few years. To get more out of the stress testing exercise and use the system for more than just regulatory compliance, banks need speed and repeatability – which they will not get without revamping infrastructure. Auditability and model management functionality also need to be built into stress testing systems to answer questions from regulators and work backward from line items in the regulatory reports to source data, model assumptions, and management overlays. As current stress testing systems have been largely built in-house across multiple existing systems, there is no natural aggregation point to house models and to provide the auditability from report to database that is now required. Thus, there is a prevalent need for infrastructure overhauls.

However, banks face a challenge at this point. The stress testing process is set up as a repeatable annual/semi-annual event, where improvements and remedies for past MRAs are expected. Many banks jump from short-term fix to short-term fix and feel that they cannot embark on a multi-year overhaul. Sharing longer-term improvement plans with the regulators could be a solution to this problem, but many banks fear they will be held to a roadmap that they may miss or want to alter. Many have mentioned that they do not want to commit to long-term goals. On the other hand, regulators have expressed interest in seeing longer-term plans. Perhaps, as the processes become more routine and trust develops between regulators and banks, we will see multi-year roadmaps enter the regulatory discussion.

When we asked banks what they use their stress testing results for, they all said regulatory compliance and most also said capital planning. The degree to which banks use stress testing results quickly drops off after that. Not only did fewer banks mention that they were using stress testing for other purposes, but the ones that did were much less emphatic about how they use stress testing in other non-compliance areas. It was fairly surprising how few banks use the results for strategic purposes, such as financial planning and budgeting.

Regulators have indicated they would like to see the banks use stress testing for other purposes, like risk appetite definition, limits, and general risk management, but banks have indicated there is still a long way to go before they incorporate stress testing into these areas of their business.

Based on feedback from participants, we anticipate that investments in data, modeling, scenario design, and infrastructure will accelerate as banks seek to deliver more efficient and consistent responses to regulators and to maximize returns on their stress testing expenditures.