Combining Financial and Behavioral Information to Predict Defaults for Small and Medium-Sized Enterprises: A Dynamic Weighting Approach

Currently, the banking industry uses two primary types of information—financial information and behavioral information—independently, to assess risk. Financial information includes Income Statement, Balance Sheet, Cash Flow, and Financial Ratios. Behavioral information includes spending and payment patterns, among others. Both types of information provide unique insights, but, to date, they have not been combined to generate one comprehensive risk metric for commercial use.

This note presents the first tool that assesses borrowers’ credit risk using a scientific method that leverages both financial and behavioral information. The tool enables lenders to utilize both types of information when determining a borrower’s risk. We use a discrete-choice model framework that allows size-dependent weighting on the two types of information. Further, the tool places an increased emphasis on the importance of financial statements for larger firms, while relying more on behavioral information for smaller firms.

We examine the model’s ability to separate high-risk companies from low-risk companies, and show that the combined model produces more realistic default rate levels compared to financial information or behavioral information alone. This study uses Italian small and medium-sized enterprises (SMEs) as an example. We are currently implementing the model on other countries as well.

This note is an abbreviated version of the full-length paper. For more details, please contact us.

1. Introduction
Banks face exposures to credit risk from their borrowers. Corporates face exposures to credit risk from their customers. A well-designed origination process allows lenders to make loans faster, increase market share, and lower operational costs. A good risk model lowers charge-offs and provisions, especially during business downturns. To remain competitive, lenders must be able to do both well. Lenders assess borrower credit risk using different types of information. Does the borrower have a history of repaying in a timely fashion? Does the borrower bring in enough cash to meet their commitments? Does the value of the borrower's assets exceed their liabilities? The information utilized depends on borrower size as well as the size and duration of the loan. In general, financial information is viewed as more accessible and more reliable for larger firms than for smaller firms. When making a sizable, long-term loan, a bank wants to thoroughly examine the borrowers’ history and finances. In contrast, a business may be willing to deliver goods on short-term credit in the context of an ongoing business relationship without a thorough examination of the borrowers’ finances.

With advances in technology, more information is available to more lenders. One of the challenges facing lenders is how to combine the different types of information into metrics that can support good business decisions: Does the bank make the loan? Does the company deliver the goods on credit? Two types of information are widely used independently of one another to help with these decisions—financial information and behavioral information. Financial information includes Income Statement, Balance Sheet, Cash Flow, and Financial Ratios. Behavioral information includes a company’s spending and payment patterns. Historically, lenders have relied upon financial information for medium-sized enterprises and behavioral information for small
businesses. Small and medium-sized enterprises consist of enterprises that employ fewer than 250 people and that have either an annual turnover not exceeding €50 million or an annual balance sheet total not exceeding €43 million.¹

Further, some lenders may make the decision based on intuition, while others may have invested in a model that employs scientific method to help make the decision. If lenders can combine both of these information types into one score, they can make better decisions than lenders who do not. Using both types of information requires collecting data for each as well as utilizing a model that enables combining them into a useful, summary risk measure.

This note presents the first model and methodology that combines financial and behavioral information into a unique risk measure, helping lenders to better discern risk and to remain ahead of the competition by utilizing a more comprehensive measure. Moody’s Credit Research Database (CRD) provides data for large private firm financial information and CDR’s Credit Behavioral Database (CBD) provides high frequency, behavioral information database, focused on Italian firms. The database also updates firms subsequently unable to pay their debts. Using this unique, combined dataset, we estimate a model that determines probability of default, based on both the financial information as well as the behavioral information. The model utilizes both types
of information to determine the risk assessment of a borrower. Further, the tool places an increased emphasis on the importance of financial statements for larger firms.

Section 2 describes model development data. Section 3 introduces our methodology, Section 4 discusses results, and Section 5 concludes. 

2. Data
This section discusses the dataset used to estimate the model. With the advent of Basel II, banks chose to develop quantitative approaches to assess the default risk of exposures in their portfolio. Such models typically involve a mixture of financial statement information and behavioral factors. For large and medium-sized enterprises, the financial statement information plays a large role in risk assessment. For small firms, behavioral information is more important.

Concurrently, Basel II leads to a harmonization of the definition of default around concepts of 90 days past due and unlikely to pay. Under Basel, if a firm has a term loan, a line of credit, and makes payments against the term loan while drawing down the line of credit, the borrower is not in default. In contrast, if the borrower does not have an explicit line of credit, but uses the term loan as an implicit line of credit by not making the required payments, Basel considers this instance a default.

Our work utilizes two databases. The first, Moody’s Analytics Credit Research Database, contains financial statement data since 1993. It uses an insolvency-based default definition. This default definition intends to be consistent with the Basel definition of default to the best extent possible, while, at the same time, it can be uniformly applied over the 20-year period. The second database, CDR’s Credit Behavioral Database, contains behavioral information since 2010. It uses a definition of default consistent with the modern treatment of 90 days past due.

2.1 Default Definition
Moody’s CRD database contains liquidation events. These types of insolvency defaults represent the most severe default cases. The definition of insolvency defaults remains consistent over time. Nevertheless, it is a subset of all Basel-compliant default types. Therefore, we scale up the RiskCalc™ model outputs based on the CRD database to account for all default types. In the Credit Behavioral Database, a default occurs if a company is 90 days past due for at least 90 days with material loss. We use this default definition in the combined model, which combines the financial statement information with behavioral information.

2.2 Model Development Data
We use the intersection of data from the CRD and CBD to estimate the model. For each observation in the dataset, we can observe both the financial statement items and the loan behavioral items.

Table 1 summarizes the sample. The combined sample covers 2010–2016, with 19,000+ monthly observations on 5,000+ companies. There are 1,600+ defaults in the sample. The average sample default rate is 7.3%. We use 2010–2015 data for model development. We use 2014–2016 data for the out-of-sample test.

Figure 1 shows the distribution of observations and defaults across years. We merge year-end financials with corresponding behavioral information. The default year in Figure 1 reflects the year when a defaulted company first entered into real default state. We mark companies entering real default state between next year April 1st and March 31st of the year afterward as default when modeling. 

3. Methodology
There are two ways to use financial information and behavioral information in the model. We can include the individual financial ratios and payment-related variables, or we can use summary scores.

Our approach uses RiskCalc Italy 4.0 Model EDF values as the financial score and CDB 2.0 scores as the behavioral score. There are several aspects as to why we choose to use summary scores. Parsimony is one concern. We can include the financial ratios used in the RiskCalc model, but the prediction power does not differ from using EDF values directly. Using summary scores also provides a clearer picture of the relative importance of different types of information. In cases where one information type is not available, we can use RiskCalc EDF values or CDB 2.0 scores independently, as we test both measures of power.

We use a probit model framework to combine behavioral score and financial score. The interaction terms between the size variable and the two summary scores allow the importance on financial information and behavioral information to change dynamically with firm size.

The combined model produces a one-year probability of default (PD). To calculate the PD for longer horizons, we apply the RiskCalc term structure to the one-year PD.

4. Results
This section discusses combined model performance. We first examine the model’s ability to separate high-risk companies from low-risk companies. We then discuss the different roles of financial and behavioral information for companies of different sizes. Finally, we show that the combined model also produces more realistic default rate levels compared to financial information or behavioral information alone. We create a two-dimensional “triangle graph” to help visualize the comparisons between the combined model PD, the realized PD, and the individual model PD.

Accuracy Ratio (AR) is a widely-used metric that measures a model’s rank-ordering ability. Both RiskCalc EDF values and CDB 2.0 scores demonstrate good predictive power, indicated by high AR values.

As discussed in Section 2, the RiskCalc 4.0 Italy Model predicts insolvency type defaults, while the CDB 2.0 Model predicts delinquency type defaults. The combined model adopts delinquency type defaults, which covers the more severe, insolvency defaults.

The first column in Table 2 shows the company size groups based on total assets. Following number of observations and defaults, column four through column six show the AR of the combined CDB 2.0 and RiskCalc outputs. Finally, we calculate the increase in ARs by adding behavioral information on top of financial information, and the increase in AR by adding financial information to behavioral information. Taking the entire sample as an example, the increase in AR by including behavioral information is 30.4% (AR of the combined model output, 74.9% minus the AR of the RiskCalc EDF 44.5%).

We find that behavioral information adds 20–30% AR improvement on top of financial information alone, while financial information adds 5–10% AR improvement to behavioral information alone. From top to bottom, as company size increases, financial information adds more power, as shown in the last column of Table 2, especially for companies with total assets greater than €6MM. Our hypothesis proposed financial information impacts larger companies’ ARs more, and. these observations demonstrate our hypothesis empirically. 

Finally, other than identifying high-risk companies, another purpose of PD modeling is to predict PD level. Predicted PD is often used as input used for calculating portfolio losses, and regulatory/economic capital. To verify the PD level, we compare across predicted PDs from different models and also the realized PD. One challenge is visualizing pairwise comparisons, especially if deeper analysis is required for different cuts. The triangle plot in Figure 2 solves this problem. In the two-dimensional space, it lays three PD values for comparison by two cuts. To read the triangle plot, the left upper triangle shows the observed default rate, the right triangle shows the CDB 2.0 score, and the lower triangle shows the combined PD. Along the vertical axis, the poorer the financial score, the worse the EDF value. Along the horizontal axis, the CDB 2.0 score increases (worsens) from left to right. Darker color indicates higher default rate. 

Taking the uppermost right square as an example, we see three PD values, taken from the three models for companies with the worst 20th percentile, financial-implied PD and the worst 20th percentile, behavioral-implied PD. The upper left triangle shows a realized PD of 27.07%. The right triangle shows the CDB 2.0 model-predicted PD,11.76%. The lower triangle shows the combined model-predicted PD, 28.83%. The combined model increases the predicted PD to a more realistic level for companies with poor financials that cannot pay their bills on time.

Figure 2 shows that, in general, the combined PD matches observed default rates well and performs better than the CDB 2.0 score. The CDB 2.0 score under-predicts default rate when financial or behavioral score deteriorates. The under-prediction becomes more severe when both scores are becoming worse, which are in blocks toward the upper right corner. We also see that the combined score performs better than EDF value if we use the EDF value instead of the CDB 2.0 score. 

5. Summary
More and more credit analysts realize the importance of considering both financial information and behavioral information when assessing the borrower’s credibility, but there is not yet an industry standard for consolidating these different information types. Our approach formalizes this process. We use a dynamic weighting to combine financial-implied credit scores and behavioral information-implied scores. The combined measure provides better rank ordering and level prediction than individual scores. The framework also agrees with economic intuition that the weight of financial information should be positively correlated with company size.