ESG Score Predictor: Applying a Quantitative Approach for Expanding Company Coverage

Abstract

Assessing Environmental, Social, Governance (ESG) and climate risk is often subject to data constraints, including limited company coverage. This paper provides an overview of Moody’s ESG Score Predictor, an analytical framework that can expand coverage gaps by generating a wide array of ESG and climate risk metrics. Our comparable and standardized predicted metrics include large-, mid-, and small-cap firms, spanning a wide variety of industries across both developed and emerging markets. We demonstrate our approach’s effectiveness using two illustrative portfolios comprised of a large number of firms. The resulting portfolio profiling and vulnerability analysis allow us to achieve full coverage and to generate heatmaps identifying company performance disparities across sectors, industries, and regions.

1. Introduction

Environmental, Social, Governance (ESG) issues and climate risk have become critical considerations for organizations such as banks and asset managers to identify risks and opportunities within their portfolios. Under increasing regulatory and market pressure, assessing firm-related ESG and climate risk management practices requires consistent and comparable metrics. Despite rapid growth in available underlying data, limitations continue to impact company coverage, reported data quality, assessment consistency among major ESG score providers, and infrequent updates. Additional limitations arise when historical ESG scores are retroactively adjusted, as they can invalidate insights obtained during earlier analyses.¹

Analyzing how well a firm manages ESG issues and climate risk is typically performed using company-level quantitative and qualitative information. This process involves directly engaging with companies, reviewing publicly disclosed information, and sometimes supplementing with alternative data to measure and assign attribute weights during the assessment scope. However, such a full assessment is not always feasible, as data gaps persist, and the number of firms may be prohibitively large for direct engagement. Coverage is particularly patchy for smaller companies, less regulated industries, and emerging markets. The number of companies covered by major ESG score providers typically ranges between 1,000−10,000, representing a major challenge for organizations with many more firms in their portfolios.

The ESG Score Predictor is a set of models designed to provide more than 50 comparable and standardized metrics, including granular ESG scores, an energy transition score, a physical risk management score, and carbon emissions footprints. These predicted metrics allow us to compare companies across industrial sector, any market cap size segment, and location, while accounting for economic, social, natural, and human capital development indicators in the location(s) where a company operates. This paper discusses an analytical solution for closing this coverage gap using Moody’s ESG Score Predictor to assess “unscored” firms and, hence, achieve full portfolio coverage.

Leveraging consistent historical data from Moody’s ESG Solutions² 2004 through present, we construct and calibrate the models on a dataset containing more than 100,000 firms to predict metrics for 600+ industries and 12,000 sub-national locations in 220 countries and territories. The prediction model for each metric consists of individual regressions and alternative machine learning (ML) models, with a variety of drivers combined into one using ensemble methods. The models are then applied to the “unscored” firms to produce interpretable, predicted metrics for expanding coverage in terms of size, location, and industry. The coverage universe for these predicted metrics is many times the size of the covered universe. As long as we have data on a firm’s size, location, and industry, we can use these three factors as inputs to generate predicted metrics using the SP models. Our coverage is “infinite,” limited only by access to these three inputs, and as long as industries are included in the NACE 4 list and locations are within the 12,000 subnational locations.

To demonstrate the effectiveness of our methodology, we perform profiling and vulnerability analysis for two illustrative portfolios — one with 17,000+ firms and the other with 65,000+ firms. The portfolios differ in terms of underlying companies’ location, size, and industry. We show how our analytics achieve full portfolio coverage and identify specific groups of firms exposed to increased risk associated with various ESG and climate factors. We create heatmaps to highlight sector and location risk. Using this approach, organizations can assess portfolio exposure and the appropriateness of mitigating strategies implemented by firms within the portfolio. For firms associated with higher risk, a more detailed, full assessment should be performed, which may include using available disclosures, data from alternative sources, as well as direct engagement with management to enrich assessment based on these predicted metrics.

We organize the remainder of this paper as follows: Section 2 provides an overview of the ESG Score Predictor methodology, including data, models, and calibration, and discusses model performance and interpretability. Section 3 demonstrates the methodology in action for two illustrative portfolios and analyzes results. Section 4 concludes. The Appendix provides an overview of the model drivers.

2. Introducing ESG Score Predictor

Moody’s ESG Score Predictor provides ESG and carbon emissions footprint estimates, as well as transition and physical risk management scores, for any size company. We combine the scoring methodology and data from our ESG Assessment universe with robust environmental and socioeconomic measures. This innovative approach delivers an unparalleled set of globally comparable and standardized scores based on company size, industry, and location.

The ESG Score Predictor models are designed to provide the best approximation of full ESG and climate risk assessment metrics listed in Table 1. The assessment contains multiple layers of granular ESG scores for each domain, following the V.E Equitics^© methodology, while climate metrics are represented by physical risk management scores, energy transition scores, and Scope 1 and Scope 2 carbon emissions. The predicted metrics using the ESG Score Predictor models serve to expand company coverage and complement the full-assessment metrics for the existing and growing universe of companies covered in our ESG Assessment universe.

Within our framework, each metric represents a predictable target variable for “unscored” companies, using models with a variety of drivers or features. Scores ranging between 0−100 provide a forward-looking view of a firm’s trajectory, including how well they are managing ESG risks and opportunities, tackling the transition to a low-carbon economy, as well as anticipating, preventing, and managing the physical risks of climate change. Carbon emissions metrics include direct Scope 1 and indirect Scope 2, represented in tons of CO₂ equivalent. Carbon emissions provide a point-in-time view, reflecting a firm’s actual climate impact and behavior.

This granular set of metrics and their combinations enable a multifaceted and holistic sustainability assessment for portfolio analysis. For example, the energy transition score, which measures a strategic approach to reduce emissions, may indicate that companies are already moving in a favorable direction when it comes to undertaking climate action, while the actual carbon footprint grade may be still problematic.

Data

We obtain the data to build the prediction models from various sources. Firm-level target variables are classified into three key groups. First, the firm-level corporate disclosures including company size, location, and industry obtained from Moody’s Implied Ratings (MIR), Moody’s Analytics CreditEdge™ (CE), and Moody’s Default and Recovery Databases (DRD). Second, we source country-level climate and physical risk metrics.³ Third, we obtain firms’ participation in the UN Global Compact, as well as country and sub-national macroeconomic indicators, and sustainability, development, and freedom measures from many different private and government data providers aggregated in the Data Buffet database.⁴

Datasets are merged carefully and cleaned extensively to ensure the data used to build the models are of very high quality. The goal of the merged dataset construction is to populate firm-level data for as many companies as possible, using various mappings and assumptions. The Standard Industry Classification (SIC) and North American Industry Classification System (NAICS) are mapped into industry-standard Nomenclature of Economic Activities (NACE) codes.⁵ To measure company size, we include a firm’s total assets, number of employees, and turnover and then fill in remaining missing values using sector and country averages. We use the country of a firm’s operations rather than the country of incorporation for mapping the regional data when merging all datasets. For parent companies with strong influence on their subsidiaries, we utilize the same target metric values, unless subsidiaries are scored in our ESG Assessment universe.

The resulting merged dataset used for model building and subsequent calibrations contains the corporate disclosures for more than 100,000 companies. Meanwhile, the data for target metrics cover 19,000+ firms globally. Depending on the data availability for each target metric and corresponding drivers, the modeling datasets for each metric contain between 28,985−323,051 firm- year observations from 2004−2020, for 96 countries.

Models

The ESG Score Predictor consists of individual models working in unison to provide the best approximation of each target metric, using a variety of drivers. We combine the individual models using ensemble techniques to make predictions that are more flexible, stable, and less data-sensitive than standalone models. Figure 1 illustrates the process used to build the prediction models for each metric.

We start by defining the list of potential drivers considered for the construction of individual models for each target metric. Driver choice is based on comparability, impact, data availability, and relevance for influencing the target metric. We also consider documented findings on the empirical determinants of the metrics in the literature. For example, firm size is often associated with “better” ESG scores, together with the economic and social development of the country where the firm operates.⁶

Firm-level drivers are company size (measured by total assets, the number of employees or turnover); industrial sector NACE 3; firm country location; and the firm’s participation in the UN Global Compact.⁷ To assess a company’s efforts to adhere to sustainability, we also incorporate regional-level drivers, such as economic, social, natural, and human capital indicators in the location where a company operates. The economic capital indicators include economic activity, unemployment, and foreign dependency. Social capital includes corruption, the rule of law, and governance. Natural capital includes climate hazard risks, atmospheric pollution, and the share of renewable energy. Human capital relates to knowledge access, health, labor, and other sustainable development indices. The Appendix provides a complete list of drivers.

To fit the relationship between the drivers and each target metric, we construct individual regression models and individual alternative ML models. The regression models for the scores include linear regression, regression with logistic transformation of the target variable, and fractional response regression. For carbon emission metrics, we only use linear regression. The ML individual models for all metrics include gradient boosted and random forest decision trees to capture non-linear dependences in a non- parametric way and to boost model performance.⁸

The next stage of model selection relies on interpretability measures, including the Accumulated Local Effect (ALE) and variable importance plots, to further screen drivers with negligible or counterintuitive impact on the target metrics. This step allows us to eliminate drivers that do not comply with prior expectations about their contribution and impact. Figures 2 and 3 display an example for the Overall Social Score. The homicide rate is excluded due to its counterintuitive positive impact on the target metric, in contrast to the Human Development Index.

Figure 4 presents an example of the importance of top drivers in generating the Overall ESG Score model. Company location, size, and industry are the top three drivers, followed by sustainability, freedom, and development indicators. We observe that company size is the strongest driver of a company’s ability and willingness to implement sustainable business practices, with diminishing impact for the largest companies. For social and governance aspects, larger companies tend to have more complex structures, requiring procedures such as audits, while they are more scrutinized for corporate social responsibility undertakings and labor rights. In contrast, many small- and mid-cap companies are still at the beginning of adjusting their business models to be more ESG-focused. Nevertheless, smaller companies are more agile and can improve quicker to lock in market financing, which means the relationship between size and ESG scores may change in the future.

Industry and location effects, together with economic environment and development indicators, are also noticeable. The level of scrutiny, norms, regulations, and societal development within a region influence the expectations for companies that operate there. Meanwhile, the socioeconomic indicators ascertain development pathways and capacity, which promote adaptation and foster ESG policies. Figures 5 and 6 are selected examples demonstrating that a country’s stronger socioeconomic development measured by income index and mean schooling years is associated with higher overall ESG scores for the firms operating within these countries.

Finally, we combine the individual ML models and the regressions into ensemble models using Elastic Net, Lasso, and Ridge, depending on the metrics. This process provides a common ground between the traditional and ML techniques and performs better than individual models, as we minimize the individual model errors. When training the ensemble model over the prediction of the base learners, the regularization inherent in the Elastic Net and Lasso drops the output of the underperforming base learners.⁹

We test the accuracy of the final models using measures including R-squared, Root Mean Square Error (RSME), and Mean Absolute Error (MAE). To mitigate a sample-dependency bias for evaluating model performance, we generate, train, and test subsets using random splitting, making sure that their distributions in terms of size, location, and industry remain in line with the full sample. The 30% testing sample is used to demonstrate the robust model performance once we train the models. Meanwhile, the distribution of residuals confirms that model choice is appropriate. In addition, we look at the distribution of the model output, breaking down by country and industry, to evaluate how close the predicted metrics are to the actual target metrics in each subgroup.

Calibrations

The next step applies the models to the “unscored” firms to produce the predicted metrics. It represents a challenge for certain companies in locations, sizes, or industries that have limited or no coverage in our modeling dataset, as the predicted metrics may suffer statistical biases. We thus use calibrations to ensure metrics robustness when expanding the country coverage, including micro- and small-cap firms, increasing granularity of industries, and producing predicted metrics at the sub-national level.

We apply an iterative k-means clustering algorithm to expand the country coverage from 96 countries in the modeling dataset to 220 countries and territories. We base the country and territory clusters on sociopolitical factors, macroeconomic indicators, environment and energy indicators such as energy intensity index, share of renewable energy, human development indices such as life expectancy, and sovereign ESG scores. We then develop cluster-specific models to derive a calibration overlay based on empirical relationships between the country and the cluster metrics.

To further increase location granularity from 220 countries and territories to 12,000 sub-national locations, we apply an adjustment to differentiate each sub-national region from a country as a whole. Up to three distinct levels of sub-national granularity are addressed¹⁰ by leveraging established empirical relationships between the target metrics, economic indicators, and development indices¹¹ using sub-national data. We treat advanced economies and emerging countries separately to account for fundamental differences in their economic systems.

Similarly, we increase the granularity of model-driven predicted metrics from 272 NACE 3 industry groups to 615 NACE 4 industries. Under the assumption that companies’ financial performances contribute to discriminating among the NACE 4 categories in terms of target metrics, we use a best-fitted regression model on industry-specific financial ratios¹² to quantify corrections to the predicted scores.

Finally, to capture the specific features of small-, medium-, and micro-enterprises, the output scores for the small-size company segment are calibrated using a formulaic alteration, inferred from the relation connecting the target metrics and company size. We capture this trend across the corporate data by means of a fractional response regression between the target metrics and the logarithm of asset size.

3. Applying ESG Score Predictor for Vulnerability Analysis

This section shows how to use the predicted metrics for portfolio profiling and vulnerability analysis, which can reveal how well portfolio companies manage ESG and climate risk issues. We look at the set of metrics listed in Table 1 jointly for a multifaceted view of the portfolio’s risk assessment across peers.

We leverage the ESG Score Predictor models and data for drivers described above to obtain the predicted metrics for all firms in two relatively large and diverse portfolios to achieve full coverage. Figure 9 describes the key characteristics of the two illustrative portfolios, Portfolio A and Portfolio B. While some companies in these portfolios have available ESG and climate risk data, and a full assessment can be performed, this is not the case for the remaining firms. To illustrate analysis results, we build portfolio heatmaps to identify risk exposures across industry, location, and company size.

We obtain firm-level data for Portfolio A from Moody’s MIR database. This portfolio is composed of more than 62,000 companies, located in 107 countries, with the majority having operations in the U.S., Japan, China, Canada, the UK, and India. The companies belong to 20 industries, with the top three representing manufacturing, financial and insurance services, information, and communication. Almost 80% of Portfolio A are firms with annual turnover between 1 million and 1 billion USD, 19% have annual turnover above 1 billion USD, while only 2% have turnover less than 1 million USD.

Portfolio B differs significantly from Portfolio A. We obtain Portfolio B‘s data from the European Data Warehouse (EDW), which publishes monthly performance data of European securitizations. It contains 17,000+ companies (annual turnover below 1 billion USD), located in Italy and Spain. This portfolio covers 18 industries, with 50% of the sample trade and manufacturing companies.

Figure 10 displays the resulting heatmap by industry for predicted metrics for both portfolios sorted by Portfolio A exposure. For brevity, we focus on seven selected metrics and present the results for NACE 1 industries represented in the portfolios. We construct these heatmaps relative to each portfolio’s average score: green indicates better performance compared to the averages, while red denotes poorer performance for each metric. Grey means the portfolio does not cover companies in that industry. The heatmap provides a quantitative overview of how relevant individual ESG and climate metrics are to different sectors, based on the location and size of the companies within the portfolios under analysis.

Eight industries from both portfolios, which represent more than 20%, have above average Overall ESG Scores. Industries with relatively high scores in these portfolios include financial and insurance firms, as well as electricity, gas steam, and AC supply. Financial and insurance companies are typically highly scrutinized by regulators, investors, and customers, adding pressure to introduce ESG criteria in underwriting policies and portfolio diversification. Their business relies heavily on reputation, creating additional pressures to adopt broader ESG-focused goals into their business models and internal controls, compared to other sectors.

Meanwhile, electricity, gas steam, and AC supply companies are among the largest carbon emitters and one of the most affected by social, regulatory, and economic developments related to carbon emissions. Our heatmaps reflect this trait, showing this sector boasts the worst carbon emissions metrics. For other metrics, this sector displays relatively high scores in response to the increasing social and regulatory pressure to transform their operating business into becoming more sustainable through the adoption of technologies to decarbonize their operations, provide greater worker safety, and increase health and community involvement, as well as better report transparency, inclusiveness, and remuneration.

Industries with relatively low scores include manufacturing and wholesale and retail trade. Almost all major firms in these two industries are tightening their ESG standards. However, depending on size and location, companies face more rigorous regulation and have more capacity to adopt greener technologies and adjust to changing consumer behavior and demands. The heatmap indicates that, while these industries have promoted ESG initiatives, their average performance still shows room for improvement. For instance, such improvements may include the selection of manufacturers closer to end-markets and supply chains, more sustainable products, and superior labor rights for employees.

Company location also determines specific standards and regulations and influences implementation speed. To analyze the heterogeneity of scores across countries, we calculate the deviation of the Overall ESG Score from the Portfolio A average for selected countries (Figure 11). While developed countries are usually more active in ESG regulation and have well-defined laws that ensure higher transparency and reporting, emerging nations usually have less stringent regulations and lag in implementing ESG standards. For Portfolio A, European companies are among the best performers, due to increasing regulation and newly implemented obligations for ESG disclosures. We also observe strong performers in locations such as South Africa, India, Brazil, and Thailand, mainly driven by larger companies in the manufacturing and electrical and gas sectors. In our country sample, the poorest performers reside in South Korea and Indonesia, where regulation does not yet oblige companies to disclose ESG information.

We also observe heterogeneity of scores across regions within a country, where regional operating environment, industry, and company size play important roles. Figure 12 displays the deviation of the Overall ESG Score from the portfolio average for companies operating in Italy in Portfolio B. The northern and central regions have a high concentration of companies with the largest turnover, which translates into above average, Overall ESG Scores, while the southern and insular regions, which have the smallest companies in our sample, have below-average scores.

To provide more insights regarding companies’ and industries’ performance, we look at a combination of ESG and climate risk metrics. Carbon footprint data provide a picture of a company’s emissions level at a given point in time, but they do not reflect any efforts taken to decarbonize and adapt operations to transition to a low-carbon economy. Overall Environment Score and Transition Score assess a company’s capacity to reduce its carbon footprint, providing forward-looking capabilities. Companies with lower scores are typically characterized by the absence of commitment to reducing environmental impact or lack of environmental performance measures.

Figure 13 displays the portfolio carbon emissions and the deviation points from the portfolio average for Portfolio A’s Overall Environment Score. It is apparent that high emissions do not necessarily translate into low scores. For instance, electricity, gas steam, and AC supply companies have the second-highest emissions, given the carbon-intensive nature of their activities. Nevertheless, their average score signals the industry’s commitment to decarbonization through technological innovation for reliable energy supply, better environmental policies, and disclosure of commitments and targets concerning climate protection. Manufacturing, as well as mining and quarrying companies, are among the largest emitters. However, their scores fall below the portfolio average, reflecting the challenges they face in closing the gap in ESG efforts and disclosures.

Figure 14 shows that wholesale and retail trade, as well as transportation and storage companies, are among the largest carbon emitters in Portfolio B. For industries such as transportation, characterized by unavoidable environmental impact, investors and clients expect them to take significant steps to minimize environmental impact. A relatively high Overall Environment Score for transportation reflects its environmental commitment, such as partnerships between airlines and rail companies to encourage passengers to replace short-haul flights with train travel. Meanwhile, a lower score for wholesale and retail trade companies may indicate a lag in promoting and disclosing ESG initiatives, together with increasing and changing customer demands that require more complex logistics and product rebranding. Some trade companies cannot meet these new challenges, and this issue is reflected in the below portfolio average Overall Environment Score.

4. Summary

Moody’s s ESG Score Predictor provides an analytical solution for generating a wide range of comparable and standardized metrics for assessing ESG and climate risk in portfolios where a full assessment for each underlying company is not possible. Using only company size, location, and industry as inputs, our models generate predicted metrics for each firm to ensure full portfolio coverage. Assessment applies to various company types, filling in missing metrics using estimates, especially useful for smaller companies, less-regulated industries, and emerging markets. A combination of predicted metrics yields a more accurate portfolio assessment, as it helps to identify performance disparities and common patterns across industries and regions from point-in-time and forward-looking vantages, enabling an integrated view.

Appendix

References

Berg, F., Fabisik, K. and Sautner, Z., “Rewriting History II: The (Un)Predictable Past of ESG Ratings.” European Corporate Governance Institute – Finance Working Paper 708/2020: https://ssrn.com/abstract=3722087, 2021.

Berg, F., Kölbel, J. and Rigobon, R., “Aggregate Confusion: The Divergence of ESG Ratings.” MIT Sloan Research Paper No. 5822-19, August 20, 2019.

Chen, T. and Guestrin, C., “XGBoost: A Scalable Tree Boosting System.” KDD '16: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining and arXiv:1603.02754: https://dl.acm.org/doi/10.1145/2939672.2939785, 2016.

Crespi, F. and Migliavacca, M., "The Determinants of ESG Rating in the Financial Industry: The Same Old Story or a Different Tale?" Sustainability, MDPI, Open Access Journal, Vol. 12(16), pages 1-20, August: https://ideas.repec.org/a/gam/jsusta/v12y2020i16p6398-d396416.html, 2020.

Drempetic, S., Klein, C., and Zwergel, B., “The Influence of Firm Size on the ESG Score: Corporate Sustainability Ratings Under Review.” Journal of Business Ethics, 167, 333–360: https://doi.org/10.1007/s10551-019-04164-1, 2020.

Gareth, J., Witten, D., Hastie, T., and Tibshirani, R., An Introduction to Statistical Learning. Springer, 2013.

Quéré, B.P., Nouyrigat, G. and Baker, C.R., “A Bi-Directional Examination of the Relationship Between Corporate Social Responsibility Ratings and Company Financial Performance in the European Context.” Journal of Business Ethics, 148, 527– 544:https://doi.org/10.1007/s10551-015-2998-1 , 2018.

Reverte, C., “Determinants of Corporate Social Responsibility Disclosure Ratings by Spanish Listed Firms.” Journal of Business Ethics, 88 (2):351-366. URL: https://philpapers.org/rec/REVDOC-2, 2009.

Yongjun T., D., Yan, J. and Yaqiong Yao, C., “The Determinants of ESG Ratings: Rater Ownership Matters.” Northern Finance Association: https://portal.northernfinanceassociation.org/viewp.php?n=2240017640, 2020.

Zhou, X., Caldecott, B., Harnett, E., and Schumacher, K., “The Effect of Firm-level ESG Practices on Macroeconomic Performance.” Oxford Sustainable Finance Programme, Smith School of Enterprise and the Environment, University of Oxford: https://www.greenfinanceplatform.org/research/effect-firm-level-esg-practices-macroeconomic-performance, 2020.