Rating the Raters Yet Again: Six Challenges for ESG Ratings

This is a continuation of the previously posted Rating the Raters Yet Again: Increasing ESG Scrutiny Makes Current Rate the Raters Study Even More Crucial. If you are eligible and interested in participating in the Rate the Rater’s 2022/23 research, the window is currently open: corporate issuer respondents should respond to the corporate survey, and investor respondents should respond to the investor survey.

In June of this year, the United Kingdom Financial Conduct Authority (FCA) declared its support for efforts to include ESG rating agencies under its regulatory authority. At an industry event soon after, FCA’s ESG director Sacha Sadan said:

We should try to regulate ESG ratings to try to get much more commonality because they’re very important—they’re being used a lot—but there’s also so much confusion. And we’ve seen that debate, where certain oil and gas stocks are in a rating, and certain stocks like Tesla are out. Not saying one is right or wrong, but the methodology is being confused.

Sadan’s emphasis that all parts of the investment chain have obligations of transparency and accuracy is just one of many indicators that 2022 marks a sea change for discourse on ESG ratings. Rate the Raters 2022 / 2023 is plunging into those waters.

Challenge 1: ESG Ratings Are Expensive and Bothersome

 Demand for sustainable investing options—from the largest asset owners to the smallest retail investors—means that the financial sector uses ESG ratings voraciously. This has a cost, with a recent ERM research study finding that 33 institutional investors spent an average of $487,000 per year on external ESG ratings, data, and consultants. Those higher operating costs generally get passed on to sustainable investment product customers in the form of higher management fees, which may impact investor returns rather than the bottom lines of asset managers.

Corporate sustainability professionals at large companies often find themselves caught up in endless cycles of responding, engaging, and correcting ESG ratings. Because ratings can easily become a focus for investor or board assessments of ESG performance, sustainability teams are reluctant to ignore data requests or questionnaires from major ESG raters.

Although ESG ratings are often criticized as too simplistic, constructing more complex rating methodologies or incorporating more data into the rating process increases the rater’s costs. As investors rely more on ESG ratings, the expense and bother are likely to ratchet higher too. The best approach for raters might not be to try to keep prices down, but instead to demonstrate that their ratings are worth what they cost.

“If the history of financial reporting is any indication, companies should not hold their breath for ESG data requests to somehow consolidate or narrow.”

Todd Cort, Senior Lecturer at the Yale School of Management and Faculty Co-Director at the Yale Center for Business and Environment

Challenge 2: ESG Ratings Contradict Each Other

Each ESG rater uses its own methodologies to collect and analyze data and to produce scores – methodologies which are often quite secretive. There is little impetus for ESG rating firms—who compete against one another for market share—to try to produce more interchangeable scores or to adopt a unified approach to ratings methodologies.

As a result, different firms’ ESG ratings of a company can diverge significantly from one another, a phenomenon frequently noted by both the investors who use the ratings and the corporations being rated. This variation blurs investors’ understanding of sustainability risk and performance at both equity and portfolio level. It also tempts corporations to focus on flattering ESG ratings and ignore indicators of trouble that critical ratings might signal. 

The level of divergence between ratings is striking. MIT researchers found an average correlation of only 61 percent between scores from six major ESG ratings agencies, compared to a 99 percent correlation between credit rating scores. According to one 2022 survey of over 1,000 global investors, lack of ESG scoring consistency is the number one challenge in implementing ESG investment strategies.

“Until we have an agreed upon definition of what to capture, divergence in ESG ratings is likely to continue and people might lose faith in them. We need more transparency and a better understanding of what it is that everybody is capturing.”

Dane Christensen, Associate Professor of Accounting, Lundquist College of Business, University of Oregon

Challenge 3: ESG Rating Divergence Is Often Intentional

The divergence described in Challenge 2 above comes about both unintentionally and intentionally. Unintentional divergence is mostly at the level of specific factors or datapoints. If different raters assessing a given company encounter variations in data access or interpretation that lead to contradictory verdicts for one or more sustainability factor, unintentional divergence of ratings is the result. Intentional divergence is mostly at the composite score level and is a product of the sum of a company’s sustainability data—this divergence is the “secret sauce” that distinguishes raters from one another.  

Investors and regulators often comment that ESG rating divergence demonstrate a clear need for more consistent disclosure, oversight, and data standardization. Corporations, on the other hand, might actually suffer if there were more consistency across ratings, since greater consistency would result in fewer third-party points of view that they could use—or ignore—as assessments of their sustainability performance.

ESG raters will continue to use proprietary scoring methodologies and unique metrics to differentiate themselves from competitors, causing intentional ratings divergence. Indeed, ESG raters each lean on the unique attributes of their methodological approach and responsiveness to stakeholder needs. Although more consistent disclosure could increase the predictability of each rater’s ratings, the intentional divergence from rater to rater is unlikely to ever dissipate.

“While we don't view rating divergence to be an issue, it is important for companies to determine the main ESG data and ratings providers as well as spend time to understand their methodologies and the underlying datasets to ensure that they are capturing as much as possible the money that can be re-directed to the company through passive investment strategies.”

Carmen Ng, Director, SquareWell Partners

Challenge 4: ESG Ratings Are Not Evenly Distributed

ESG ratings are not equally accessible to investors or to corporations. Nonpublic companies are often left out of ESG ratings entirely. Companies that have recently gone public are unlikely to be rated during their first year of listing. Companies trading on major exchanges in North America and Europe are far more likely to get properly rated than those trading elsewhere, especially in emerging markets. And even within the ranks of stocks listed on major exchanges, companies with a higher market cap or free float get covered by more raters and have their ratings refreshed more frequently than other companies.

This uneven distribution of rater attention has detrimental consequences for the entire ESG ecosystem. Institutional investors make do with spotty coverage of ESG ratings for newly listed stocks or for certain sectors and markets. Independent investors and smaller asset managers have limited access to ESG ratings since—as discussed in Challenge 1—ratings are often expensive to access and understand. Private companies may have no concept of how their sustainability performance stacks up against peers, hindering efforts to improve. Some supply chain partners may be poorly covered by ESG raters, meaning corporate sustainability teams are deprived of the decision-making insight that the ratings would provide.

Challenge 5: ESG Ratings Are Not Predictive

Many investors have hoped that ESG ratings would come to predict a company’s ESG risk with the same accuracy that credit ratings predict credit risk. Indeed, most of the major ESG raters have christened at least one of their rating products a “risk rating,” with implied or explicit promises of predicting a company’s probability of future ESG risk events. Recent studies have highlighted investors’ criticism of ESG ratings including claims that they are not reliably predictive of future sustainability success and that they do not correlate with ESG risk management capability.  

Quickly evolving sustainability expectations may also limit the utility of historical data to predict ESG risks or opportunities. The complexity of ESG-related data and metrics may mean that triangulating on risk predictions is best achieved by using multiple ESG ratings, accompanied by bespoke research, to construct robust predictions of risk.

ESG risk is not as straightforward as credit risk. Even the definition of success regarding the determination of ESG risk may be in the eye of the beholder. Counterintuitively, improving predictiveness may depend both on more accurate and more divergent ratings, allowing investors to compile a risk assessment that is a composite of differing perspectives and informed analyses.

“We want to see companies start looking forward. We don’t care about where they were, we want to know where they’re going. And these ratings agencies should prioritize a company’s forward trajectory over past performance.”

Michael Krzus, Audit and Public Policy Partner, Grant Thornton LLP (Retired) and Senior Advisor to ERM

Challenge 6: ESG Ratings Are Unregulated

The last challenge on our list is one that might be short-lived: ESG ratings, despite their importance to institutional investors and other stakeholders, are largely unregulated.

As discussed in our earlier piece in this blog series, ESG raters can expect increased scrutiny of their methodologies as a likely ripple effect due to regulatory developments such as the U.S. Securities and Exchange Commission’s upcoming rules to improve transparency on ESG investment practices, the inclusion of ESG rating rules in the European Commission’s Sustainable Finance Strategy, and new rules governing corporate ESG disclosure from both regulators and key standards setters.

While improved disclosure will likely not eliminate intentional divergence, the accidental ESG rating divergence outlined in Challenge 3 could become a thing of the past if regulatory guidance is strong / deep enough. However, the other challenges explored here are not easily regulated away—and ESG raters, sustainable investors, and the corporate sustainability community may still be grappling with some of them for years to come.

Rate the Raters 2022 / 2023

The challenges faced by ESG ratings agencies and their stakeholders are sometimes as daunting as they are important. As demands and expectations for ESG ratings evolve, we should expect to see a range of innovative solutions from raters to ensure investors and companies recognize them as comprehensive, useful, and accurate. In this blog post and the previous one in this series, the Rate the Raters team has examined the current state of the ESG ratings ecosystem and the challenges raters face. As a key part of our RtR 2022 / 2023 research, the issuer and investors surveys we have in the field now and which are linked below aim to understand how sustainability practitioners evaluate the ESG landscape and what they see as the most appealing next steps and solutions.

Taking the pulse of sustainability-minded investors and corporate sustainability professionals about the impact and value of ESG ratings is at the heart of each Rate the Raters effort —and we’ve never been more interested to find out what survey respondents will say. The response window for the current surveys is open now: corporate issuer respondents should respond to the corporate survey, and investor respondents should respond to the investor survey.