Since the Great Recession, stress tests have been used to assess bank’s capital sufficiency. The Current Expected Credit Losses (CECL) standard also asks for an estimation of expected losses over the life of the loans. A key aspect of these tests is to determine a reasonable estimate of losses, given a stress scenario, across different segments of the portfolio. This paper focuses on determining probability of default, given a stress scenario, which, along with LGD and EAD, comprises the major components of the expected loss calculations.
We have developed an upgraded stress testing model based on the prior 2.0 model. This model can be applied to any internal rating framework and is compatible with the RiskCalc™ U.S. 4.0 Corporate, REO, Dealership, and NFP models. In this paper, we present the updated PD model that incorporates new data and more sectors. We also expand model outputs to comply with CECL requirements. The model produces accurate in-sample estimates and reasonable loss forecasts under various stress scenarios.
This paper is a shortened version of the full methodology paper. Please contact [email protected] if interested in receiving the complete paper.
1. Overview
In 2016, FASB introduced CECL in ASU No. 2016-13, Topic 326. The new accounting standard applies to all banks, savings associations, credit unions, and financial institution holding companies (hereafter, “institutions”), regardless of size, that file regulatory reports for which the reporting requirements conform to U.S. generally accepted accounting principles (GAAP). For Public Business Entities (PBE) that file with the SEC, but NOT SRC, the fiscal year of the effective date begins after December 15, 2019. For all other Public Business Entities (PBE), the fiscal year of the effective date begins after December 15, 2022. For non-PBEs, CECL is effective from fiscal years beginning after December 15, 2022.
The current CECL interpretation assumes assessing lifetime expected loss under reasonable and supportable forecasts of major macro variables. Our existing stress testing framework already links changes in macroeconomic variables to credit risk drivers such as PD, LGD, and EAD. Therefore, this framework is a natural choice for CECL modeling. We expand our existing stress testing model to be more suitable for CECL applications and to continue to comply with CCAR.
We have developed an upgraded stress testing model based on the prior 2.0 model.1 This model is compatible with the RiskCalc U.S. 4.0 Corporate, REO, Dealership, and NFP models. This paper describes the methodology for estimating the PD component of expected losses and presents the updated version of the entire framework. We retain the model’s robust methodology, including the “PD bucket bootstrapping” from the previous model version. We update the macroeconomic variables to include Oil Price Index, and we use a more robust convexity adjustment methodology for the PD model. We also add additional features to streamline CECL calculation. We now also have built-in support for customizable, mean reversion functionality for stressed PD and LGD. For expected loss calculation, we accommodate an input for a term loan amortization schedule. In addition to term loans, we also support revocable and irrevocable revolver loan types. The main outputs include on-balance sheet expected loss, off-balance sheet expected loss, and lifetime loss rate calculations for each loan type, respectively. Validation tests show robust results for the new model.
The remainder of this paper is organized as follows: Section 2 describes the RiskCalc Private PD model,2 the macro variables, and the CRD RiskCalc EDF™ (Expected Default Frequency) measure data utilized, as well as the model’s application. Section 3 discusses model specification and estimation. Section 4 shows the in-sample and out-of-sample model performance. Section 5 discusses lifetime expected loss calculation. Section 6 concludes. The Appendix presents graphs detailing the time series of private EDF measures.
2. Modeling Framework
Our framework uses a bank’s internal rating-mapped PDs and links the time series dynamics of PD changes to macro variables. As before, we use a two-way fixed effects model for estimation and coefficients are granular by sector and by PD level. We developed this model using historical RiskCalc 4.0 EDF value data from Moody’s Analytics Credit Research Database (CRD). The main idea for PD modeling is to use the macroeconomic scenario change to predict PD changes. We follow the same statistical procedure to select the macroeconomic variables and choose five variables with high explanatory power and significant economic intuitions as in the previous version of the model. We apply the “bucket bootstrapping” methodology to obtain granular coefficients. To streamline CECL calculations, our framework supports three different loan types and provides robust and accurate calculations for on- and off-balance sheet expected loss and lifetime loss rates.
2.1 Model Overview
The main model is a two-way fixed effects multivariate regression model, with the first order autoregressive term for the dependent variable. The dependent variable is the mean log EDF change. We apply a generic macro variable selection procedure that can be considered a forward, stepwise selection process. We also employ subjective judgment based on economic intuition and model fit between steps to ensure that we select a robust yet parsimonious set of independent variables. After we select the macroeconomic variables and fix the model form, we run the rating buckets’ bootstrapping to generate the model estimators for each PD value in each sector.
Overall, our PD modeling methodology links the macroeconomic conditions with the change of the PD distribution for different sets of obligors. We want to model credit migration patterns for different groups of firms under different macroeconomic conditions. Specifically, we dynamically divide the entire dataset into portfolios with the same sector and credit risk (based upon PD) characteristics, and we model how the PD of the representative obligor of each specific portfolio changes as new macroeconomic information takes effect.
In our methodology, sector and credit risk are the two main factors for classifying the portfolios used for modeling and estimation, based upon the following motivations: 1) different sectors may have different exposures to macro variables and, thus, respond differently to the macro shocks; 2) from the EDF data, we observe a mean reversion pattern for firms’ credit risk, regardless of the macro conditions: low PD firms, on average, tend to have slightly higher PD in the next period, and high PD firms’ credit conditions tend to improve if they survive in the next period; 3) firms with different credit risk should have different sensitivities to macroeconomic shocks: we do not expect the (very) safest and riskiest firms to be very sensitive to macro shocks. By modeling the credit risk changes for the sector/PD buckets, we can accommodate the phenomena described above, thereby resulting in a model with substantial granularity.
To apply this model, clients can map their own internal ratings to certain starting PDs across different sectors. The framework projects forward PDs based upon the sector and initial PD, as well as the macro variables specified by the scenario. The procedure iteratively produces multi-quarter PD projections for any sector/rating bucket. These projections can be used in combination with models and/or assumptions to determine the expected losses of each sector and rating pair. In order to move from expected losses to provisions, we make further assumptions about the loss emergence process and prepayment rates, as well as the nature of new originations.
2.2 Macroeconomic Variables
Data include macro variables and CRD EDF measures. We combine the macro variables determined by the Federal Reserve and Moody’s Analytics alternative scenario.
As we currently focus on stressing a domestic middle market private C&I portfolio, we consider primarily domestic macroeconomic variables. Historical data for each variable covers Q1 1990–Q3 2018.
2.3 Moody’s Private EDF Data
CRD EDF values are based upon the financial statements of middle market exposures we collect as part of the Credit Research Database’s consortium of U.S. banks. These financial statements are the core of the datasets used to develop, calibrate, and validate the RiskCalc suite of models.
For the PD modeling of private firms, we use the Credit Cycle Adjusted (CCA) EDF credit measure produced by Moody’s Analytics RiskCalc U.S. 4.0. Model. The RiskCalc U.S. 4.0 Model development dataset begins in 1994, covers more than 133,000 firms, and includes more than 9,000 defaults and more than 580,000 financial statements.
The primary data used for this project are CCA EDF measures of all available firms in the CRD. The sample spans from 1993 to 2017. We assign 16 major sectors for all the firms in the CRD database. They include: Agriculture, Business Products, Business Services, Communication, Construction, Consumer Products, Dealership, Health Care, HiTech, Mining, Not-for-Profits, Real Estate Operator, Services, Trade, Transportation, and Utilities. We exclude firms with missing or “unassigned” sector information in the CRD database (as well as firms from the financial sector), from the final model estimation.
Table 2 provides the development sample’s descriptive statistics for the CCA EDF measures.
Figure 15 in the Appendix shows the time series of EDF value quantiles for the entire data and for some sectors. Generally, the patterns of the figures look similar across different sectors. Two EDF value peaks appear in the graphs: one during the Dot-com bubble crash during 2000–2003, and the other during the recent subprime mortgage crisis beginning in 2007. As we expect, the first crisis has a larger impact upon the “HiTech” sector, and the second crisis is more severe and has a larger impact upon the default risk of all sectors.
3. Model Specification and Estimation
This section provides details on dependent variable selection, model specification, macro variable selection, rating bucket bootstrapping, PD projection procedure, and estimation results.
3.1 The Dependent Variable
We use the following steps to construct the dependent variables used in our model:
Given a PD or rating bucket mapping, for each quarter, we group all observations into different sector/rating buckets based upon their sector and PD information. Bucket components vary over time, as firms shift out of or move into the given PD range. To obtain an initial rating buckets division, we divided the private EDF data into 10 rating buckets, based upon the equally-spaced 10 percentiles. Section 3.4 describes how we generalize the rating buckets definition to introduce more granularities to the model.
The dependent variable is the change of the mean log EDF value for each bucket from one quarter to the next. We have estimates of 𝑦𝑡 for each sector/rating bucket at each quarter, and, finally, we have a panel data structure. The panel data has two cross-sectional dimensions, sector and rating, and it also has a time dimension.
To illustrate the time series patterns for the model’s final dependent variable, Figures 1 and 2 show the plots for Rating=1−2 and Rating=9−10 for the Agriculture and Business Services sectors.
A few observations can be made from the graphs:
- The dependent variable is time-varying, and the pattern coincides with the economic environment. For all the plots, the peak — the highest EDF value change — occurs during the second half of 2008 and shows that the dependent variable responds to macroeconomic shocks and is appropriate for use in a stress testing context.
- For different rating buckets, the means of the dependent variable are clustered around different levels. For good rating buckets (low EDF value firms), we see more frequent large-magnitude positive EDF value changes than negative EDF value changes, i.e., the credit risk of these firms tend to deteriorate on average. For lower rating buckets (high EDF value firms), we see more frequent, large-magnitude EDF value decreases than increases. So, there is “mean-reversion” behavior in the EDF value change for different rating buckets.
- For different rating buckets, the dependent variable’s sensitivities to macroeconomic conditions seem to differ. This trait can be seen from the magnitude of dependent variable during the crisis periods. For example, for both sectors shown, we see that Rating=9 has the largest EDF value change in 2008 compared to other rating buckets.
- It is clear that the two sectors shown in the graphs, Agriculture and Business Services, have different exposures to the macroeconomic environment. Compared to the Agriculture sector, Business Services has a more drastic EDF value change around 2000–2011, the magnitude of which is comparable to that in the subprime mortgage crisis. In contrast, we see that, for the Agriculture sector, the EDF value change in 2008 stands out, when compared to all other periods.
Based upon the above observations, besides the time-series effect, the final model should take into account the differences between economic impacts across the sector and the rating dimensions.
3.2 Model Specification
For the PD model, we assume a client’s ratings are comparable and can be mapped to Moody’s Analytics PD measures directly. Additionally, we make the following assumptions:
- The impact of macro variables can be decomposed into sector effect and rating effect.
- Sector effect and rating effect are independent.
- The impacts of macroeconomic variables do not change over time.
- The autocorrelation term does not vary across sector/rating and does not change over time.
3.3 Macroeconomic Variable Selection
3.3.1 TRANSFORMATIONS
Besides the original macroeconomic variables, we also consider additional variables, constructed from the original macro variables, such as treasury term structure slope, and various transformations of initial macroeconomic variables.
3.3.2 NOMINAL VS. REAL VARIABLES
For GDP and disposable income, we have both nominal and real variables available. When the real term and nominal variables have similar explanatory power, we prefer to use the real variables, since they are not contaminated by inflationary effects and measure the pure economic activities.
3.3.3 UNIT-ROOT TESTS
It is well-documented in the econometrics literature that if a time series variable has unit root (or is “non-stationary”), the inference, based upon OLS regression, on this variable could be spurious and cannot be trusted. To avoid this issue, for each macro variable, we use the augmented Dickey-Fuller test (ADF) to test unit-root behavior and exclude all the variables that cannot pass the stationarity check. The variables finally excluded are the raw variables (levels); their transformations, including change or growth, mostly pass the stationarity checks.
3.3.4 MACROECONOMIC VARIABLES SELECTION
In general, we can group all macroeconomic variables into the following categories:
- Market variables: these include variables such as the Dow Jones variables, CBOE VIX, BBB yield and credit spread;
- Economic variables: these include GDP, DPI, Inflation, Unemployment Rate, Oil Price Index, etc.
- Interest rate-related variables: 3-month Treasury yield, 10-year Treasury yield, and Mortgage Rate;
- Housing price: Housing Price Index and CRE Price Index.
Our goal is to choose a set of macroeconomic variables from different categories so that the model is robust and can work well in different stress scenarios.
After applying transformations and unit-root tests on the macroeconomic variables, we use a forward, step-wise selection process, with the help of subjective judgment and economic intuition, and review model fit between steps. We repeat the procedure until the model fit cannot be improved.
3.4 PD Bucket Bootstrapping Methodology
If we build models based upon given, fixed PD bucket definitions, we may encounter the issue of a low firm-count for certain Sector/PD buckets.
To achieve the dual-goal of having granularity and obtaining accurate model estimations, we utilize a “rating/PD buckets bootstrapping” methodology. We use different PD cutoffs to define the PD (or “rating”) buckets so that we have different ways to construct the portfolio, based upon sector and PD characteristics. We then compute the dependent variable for each iteration of portfolio construction and estimate the model, obtaining estimators for different PD ranges. The bootstrapping methodology also provides more granular and accurate model coefficients.
3.5 Final Model and Results
Univariate Analysis
After applying transformations and unit-root tests on the macroeconomic variables, as the first step, we place each of the macroeconomic variables into the full model, estimate their coefficients for all buckets, and study their statistical fit. Variables are ranked based upon their adjusted R-squared in the full model. From the initial univariate check, we see that there are good candidates from all categories. We also see that market variables have high explanatory power. These good candidates have intuitive coefficient signs too.
Final Model Specification
Table 4 shows the final variables used in the model. The regression with 16 sectors and 10 rating buckets has approximately 10,000 observations. We transform those variables and make sure the signs are intuitive.
During the selection process, we restrict the variables entering the final selection to correlation within the range of ±0.5. The largest correlation (in terms of magnitude) is correlation of 0.36 between the Unemployment_QoQ_change_lag1 and Credit_Spread_YoY_ratio. These variables do not appear to be highly correlated.
After fixing the set of macroeconomic variables, we compare different reduced form models with the full model. The tests confirm that sector, rating, intercept should all be included.
PD Buckets Bootstrapping
After we finalize the macroeconomic variables based upon the 16 sectors and initial 10 rating segments, we conduct the rating bucket bootstrapping to span the coefficients estimations for the full PD range.
Section 3.4 describes the bootstrapping methodology steps. For each sector, we pool all the coefficients and the mean PDs of each rating bucket together, as shown in Figure 3 using the Trade Sector as an example.
From the plots (for all sectors), we observe that the coefficients generated are consistent with intuition. The magnitude varies with PD range, but the sign does not change overall. The intercepts and the coefficients mostly have a U-shaped pattern. Firms in the middle PD ranges have higher sensitivities to macroeconomic variables than firms with very low or very high PDs. This finding is also consistent with intuition: we do not expect extremely safe or risky firms to respond much to the market condition, i.e., they are either very safe or already in trouble. The firms with PDs in the middle range have more room to either move up to go down.
We also fit the adjustment against PD value for each sector. After the local regression and curve fitting on coefficients, the coefficient curves against PD are smooth lines.
Since we do not require the autoregressive terms to vary across different sectors/ratings, we have one estimation for each loop of the bootstrapping. In the end, we take the mean of all N estimations of the autoregressive term 𝜌 from the bootstrapping as the final estimation.
4. Model Performance
This section presents results of various validation tests. We investigate both the model’s in-sample and out-of-sample fit to demonstrate performance robustness. In addition, to demonstrate the model’s applicability for stress testing, we conduct multi-quarter forward projections by applying the model recursively to show that the model performs quite well even for long-horizon projections, in which the prediction errors supposedly accumulate over the projection horizons. We present both in- and out-of-sample results and the multi-quarter ahead analysis at aggregate, as well as sector and rating bucket levels.
4.1 In-Sample Fit
For each iteration of the rating buckets’ bootstrapping, we form portfolios differently, and we have a different set of estimates and model fit statistics. As a robustness check, Figure 4 shows the adjusted R-squared from all the iterations of the bootstrapping. We can see that the adjusted R-squared lies within a relatively narrow range, from about 57.5%−61.0%, which shows that our model is quite robust for in-sample fitting and is not very sensitive to the different definitions of rating buckets.
The estimations of the autoregressive term are within the range of 0.29−0.30, and, overall, the histogram shows close to a normal distribution. We use the mean of all estimations as the final estimate.
Figure 5 shows the model-predicted value and the actual value of the dependent variable. This set of results is based upon the original 16 sectors and 10 rating buckets divided by sample deciles.
We show the graphs for the lowest and highest PD ranges (Rating=1, 2, 9, and 10) for illustration purposes. We observe the following from the graphs:
- Generally, the predicted log-EDF change follows the time-series pattern of the actual log-EDF change, although the predicted log-EDF change is much smoother than the actual log-EDF change (as expected).
- We see that the predicted value in the recent crisis (around the second half of 2008) matches very well with the actual value. Besides the peak in the recent crisis, the predicted log-EDF change also increases notably in the first recession period (around 2001–2003).
- The predicted actually carries less noise of the actual data, and the time series pattern is more in-line with economic cycles.
4.2 Robustness
4.2.1 MULTI-HORIZON PROJECTION
We perform additional robustness checks on long-horizon projections. The main reason for performing this analysis is because we build our model upon short-term, one-quarter EDF measures and macro variable dynamics, while the goal of this model is to produce up to twenty-quarter projections of PDs and losses. It is crucial to demonstrate that the model performs well even for long-horizon projections.
For each quarter and for each sector, we form 10 rating portfolios using the PD deciles. We compute the mean PD for each portfolio and use it as the starting point to compute the four-quarter ahead projected PD using the model. We then compare the four-quarter ahead projected PD with the actual mean PD of the same portfolio. Figure 6 shows the comparison for the Trade sector. For a given quarter and rating bucket, say Q1 2002 and Trade sector Rating=1 cohort, we plot the projected and realized mean Q1 2003 EDF value of the same cohort.
It is clear that the model performs well in predicting the mean EDF value at the aggregate level. To see how model projections align with the actuals by each individual sector, we conduct 20Q projections at the sector level for the crisis period (Q1 2008–Q1 2013). Overall, we see strong predictive power across different sectors. Figure 11 displays results for some representative sectors.
Besides the backtesting results on the historical macro scenarios, we have also applied the PD model to project the quarterly PDs under CCAR and Moody’s Analytics alternative scenarios. We take a snapshot of our CRD portfolio at Q4 2018 to form the initial PDs at Q0. We then stress the PDs under different macro scenarios. Figures 12 and 13 show the average PDs of the entire CRD sample under CCAR 2019 and Moody’s Analytics alternative scenarios. We choose four different scenarios: Baseline, Consensus, S1 (Stronger Near-Term Growth Scenario) and S3 (Moderate Recession Scenario) from Moody’s Analytics alternative scenarios. We also mean-revert the stressed PD to its forward 5-year FSO EDF (1.62% on average) in Figure 14. The average PD at Q0 (Q4 2018) is 1.69%.
5. CECL Calculation
To comply with the Current Expected Credit Loss (CECL) requirement, we provide a detailed methodology for calculating lifetime expected loss. This section presents brief descriptions on how to combine Probability of Default (PD), Loss Given Default (LGD) and Exposure at Default (EAD) models together to fully capture the expected losses.
For LGD calculation, we follow the same methodology as described in “Stressed LGD Model.”3
We also provide the mean reversion function for both stressed PD and LGD.
We divide loans into three loan types: term loan, revocable revolver loan, and irrevocable revolver loan. For term loans, we treat the usage for both defaulters and non-defaulters as constant 100% through the lifetime. For revocable revolver loans, we treat the usage for both defaulters and non-defaulters as constant (Balance at Q0 / Commitment at Q0) through the lifetime, because the loan issuer can take the loan back at any time. For irrevocable revolver loans, we model the usage, defined as the ratio of current balance over current commitment, directly as an AR(1) process, controlling for the current PD level. We model defaulters, which includes observations within three years prior to their default dates, separately in order to project EAD and then the non-performing balance.
The calculation projects the expected loss based upon the stressed PD, LGD, EAD, taking the amortization schedule as an input. The lifetime loss rate is given by sum of the loss per quarter divided by the initial total commitment. Other quarterly outputs include commitment, performing balance, non-performing balance, usage for defaulted and non-defaulted portion, on-balance sheet expected loss, off-balance sheet expected loss, NCOs, ALLL, and provisions. Please refer to Sections 3.2−3.3 in the methodology paper “Calculating Scenario-Dependent ALLL and Provisions” for detailed calculations.
6. Conclusion
Our benchmark PD was developed originally from the Federal Reserve’s framework to mimic their stress PD model used for the Comprehensive Capital Analysis and Review (“CCAR”) exercises. The model methodology lends itself well as a basis for a new model that can also satisfy additional CECL requirements. In addition to stressed PD, we also calculate fields such as lifetime expected loss, on-balance sheet expected loss, and off-balance sheet expected loss, which make the model fit for CECL purposes. We also make a number of additional data-related and econometric refinements. In particular, we build our private PD model based upon Moody’s Analytics CRD RiskCalc EDF data to better reflect the dynamics of the probability of default for private, middle market borrowers. The model uses a two-way fixed effects panel regression with autoregressive term specification to capture the rating and sector effects. We develop a generic methodology for selecting macro variables to ensure the model is intuitive, parsimonious and has good statistical fit. We also apply the “PD bucket bootstrapping” method to simulate a full spectrum of PD bucket definitions to use the EDF data to the largest extent and achieve the maximum granularity. Extensive in-sample and out-of-sample validation tests show the model is robust and works well under different economic scenarios.
The methodology can be applied to any internal rating framework calibrated to produce PDs. Managers who use fundamental analysis to determine ratings or who have a portfolio for which financial statements are not readily available (or both) can also apply our approach to their C&I portfolios for CCAR and CECL purposes.
Figure 15 shows the time series of private EDF values (RiskCalc) for different sector quantiles.
We thank Douglas Dwyer, Zhuang Zhong, and Janet Zhao for their contributions to some of the related CCAR stress testing projects. We also thank Christopher Crossen for his editorial assistance.
1 Chen, Nan, Jian Du, Dwyer, Douglas, Jing Zhang, and Zhong Zhuang, “Stress Testing Probability of Default for C&I Portfolios: A Granular Approach,” Moody’s Analytics, April 2014.
2 Other components, such as public PD, LGD, and EAD models are discussed in separate research methodology papers.
3 Dwyer et al. “Stressed LGD Model,” 2014.
Chen, Nan, Jian Du, Dwyer, Douglas, “Calculating Scenario-Dependent ALLL and Provisions,” Moody’s Analytics, June 2014. Chen, Nan, Jian Du, Dwyer,
Douglas, Jing Zhang, and Zhong Zhuang, “Stress Testing Probability of Default for C&I Portfolios: A Granular Approach,” Moody’s Analytics, April 2014.
Crosbie, Peter, and Jeff Bohn, “Modeling Default Risk,” Moody’s Analytics White Paper, 2003.
Dwyer, Douglas, “RiskCalc: New Research and Model Validation Results,” Moody’s Analytics White Paper, 2011.
Dwyer, Douglas, I. Korablev, U. Makarov, J. Wang, A. Zhang, and J. Zhao, “Moody’s Analytics RiskCalc 4.0 U.S.,” Moody’s Analytics White Paper, 2012.
Dwyer, Douglas, Sanjay Rathore, and Heather Russell, ”Stressed LGD Model,” Moody’s Analytics, January 2014.
Federal Reserve System, “Comprehensive Capital Analysis and Review: Summary Instructions and Guidance,” 2011.
Federal Reserve System, “Comprehensive Capital Analysis and Review 2012: Methodology and Results for Stress Scenario Projection,” 2012.
Federal Reserve System, “Dodd-Frank Act Stress Test 2013: Supervisory Stress Test Methodology and Results,” 2013.
Federal Reserve System, “Frequently Asked Questions on the New Accounting Standard on Financial Instruments--Credit Losses,” 2019.
Federal Reserve System, “Supervisory Scenarios for Annual Stress Tests Required under the Dodd-Frank Act Stress Testing Rules and the Capital Plan Rule,” 2014.
Greene, William, Econometric Analysis. Sixth Edition, Pearson Prentice Hall, 2008.
Korablev, Irina, and Shisheng Qu. “Validating the Public EDF™ Model Performance During the Credit Crisis,” Moody’s Analytics White Paper, 2009.
Zhao, Janet, Dwyer, Douglas, and Jing Zhang, “Usage and Exposures at Default of Corporate Credit Lines: An Empirical Study.” Moody’s Analytics, December 2011.
