Methodology: Developing an investible universe of climate adaptation and resilience companies

Turbold Baatarchuluu, Data Scientist, Global Adaption and Resilience Investment Working Group

Umar Ashfaq, Research Director, U.S. | MSCI Sustainability Institute

Society is increasingly confronting a need to withstand and adapt to the effects of a changing climate. That creates growing demand for products and services that improve the ability of businesses and people to prevent, prepare for, and adapt to climate hazards, as well as to rebuild better after climate-related damages, together with a need for investment that makes such innovation possible.

Recognizing the significance of this task, the Global Adaptation and Resilience Investment (GARI) working group and the MSCI Sustainability Institute, with support from the Bezos Earth Fund and ClimateWorks Foundation, embarked on a project to identify publicly listed companies that offer products and services that address climate-change adaptation and resilience. The project aimed to create an investible universe of adaptation and resilience companies.

A novel approach to identifying companies that contribute to climate adaptation and resilience

Identifying products and services that contribute to adaptation and resilience comes with challenges, as evidenced by the gap between investment (and action) to reduce and eliminate greenhouse gas emissions and investment in technologies for adaptation.^[1]

Both financial regulators and industry organizations have adopted or are in the process of adopting sustainable finance taxonomies.^[2] Examples include the Climate Bonds Initiative’s Climate Bonds Taxonomy, the EU’s Sustainable Finance Taxonomy, and the UN Environment Program’s Taxonomy of Climate Change.^[3]

These taxonomies provide a framework for understanding and defining adaptation and resilience. According to recent studies conducted by the WWF and the German sustainable finance think tank Climate & Co., most of these taxonomies mainly focus on climate mitigation, which is relatively easier to address, than on adaptation and resilience.^[4] The few that encompass adaptation and resilience either focus regionally or vary significantly based on countries’ unique economic development and climate challenges. While such approaches contribute to localized solutions, they limit opportunities for investment by global investors.

To address this investment gap and promote the flow of capital for adaptation and resilience globally, the GARI working group developed the Climate Resilience Investment Solutions Principles (CRISP) framework, which is designed to provide investors with a baseline for identifying companies that offer technologies, products and services necessary to prevent, prepare for, respond to, and recover from climate-related events.

To encourage widespread adoption of this framework, GARI and the MSCI Sustainability Institute collaborated to develop a sector-agnostic approach for investment in adaptation and resilience products and services. The approach is designed for ease of implementation and use across sectors and industries.

Implementing the new approach: A look at the process

Researchers at GARI and the MSCI Sustainability Institute used MSCI’s established thematic relevance score methodology as a foundation for the analysis.^[5] The methodology uses natural language processing (NLP) techniques that enable the detection of relevant keywords associated with specific investment themes. Initially, we considered several different NLP methodologies to identify the universe of adaptation solutions. These approaches can be categorized into traditional machine learning (ML)-based NLP and transformer-based language models (LLMs).

As a base case for comparison, we conducted the search using MSCI’s NLP engine, which relies on traditional NLP techniques such as cosine similarity between a set of relevant words and a company’s summary description. This method, however, provided a narrow universe of companies, although it provided high certainty in identifying firms providing adaptation and resilience. The process of identifying keywords for the search criteria runs into the same problem as following a singular taxonomy: Following a strict set of keywords can result in narrow search criteria susceptible to bias based on locality and analyst.

For the final research approach, we used LLMs to find relevant companies. These models, which are a form of artificial intelligence, were trained on a wide range of internet text and applied using a question-answer approach. The map below illustrates the process (Figure 1).

The analysis began with abridged business descriptions of the constituents of the MSCI ACWI IMI Index, as of December 2021. This index comprises approximately 9,000 companies in developed- and emerging-market countries, covering 99% of the global investable universe and providing a broad and diverse sample for the study.[6]

We employed various LLM application methods to define economic activities related to adaptation and resilience. These methods included question answering, sentiment analysis, LLM-response evaluations, and information extraction. To handle the complexity of this task, we used an ensemble of these methods, using an emerging technique called LLM chaining, which breaks down the task into subtasks and creates a chain of prompt operations for analysis.^[7]

To start, we developed a list of closed-ended criteria questions that addressed specific needs and risks posed by climate change. We designed the questions to be comprehensive, covering various potential impacts and responses. These questions served as the basis for our LLMs to assess a company’s products and services for alignment with adaptation and resiliency needs. To elicit answers with explanations, we employed prompt engineering techniques such as chain of thought and tree of thought.^[8] We validated the results manually by selecting ideal answers from an initial test run, which helped us instruct the LLM with examples of good answers using a few-shot prompting approach.^[9]

Despite the promise of these models, we encountered several challenges, including problems with hallucinations and consistent answer quality.^[10] LLM hallucinations refer to instances in which an LLM generates false or misleading information. To mitigate these issues, we developed an LLM evaluation methodology. This methodology used an LLM to score answers based on three evaluation criteria: relevancy, specificity, and clarity. This step ensured that the LLM’s responses were accurate, relevant, specific to the question asked, and articulated clearly.^[11]

We conducted sentiment analysis on the answers provided to determine which questions were answered affirmatively, negatively, or indirectly (Table 1). The analysis allowed us to gauge the company’s alignment with climate adaptation and resilience based on their responses.

Table 1: Examples of sentiment analysis

Subsequently, we used the LLM to extract the products and activities mentioned in the affirmative answers. This step allowed us to identify which specific product or services are used to explain the affirmative answer, providing us with a clear picture of how the company contributes to climate adaptation and resilience. This was a crucial step that allowed the final set of companies to be compared to existing taxonomies for alignment assessments.

Analysts from  MSCI ESG Research reviewed and validated the companies that the analysis identified. Their review deepened insight into the failure mechanisms of the LLMs and allowed us to adjust the prompts accordingly.

Results

Our approach led to some promising results. By employing LLM chaining, sentiment analysis, and product/service extraction, we were able to identify approximately 817 companies with 3,917 products and services associated with climate adaptation and mitigation. We observed marked differences the GICS sector makeup, with a higher representation from the industrials and materials sectors, which constituted approximately 58% of the identified set of companies compared with around 27% of companies in the MSCI ACWI IMI.

Further analysis of the specific products and services revealed that this resulted from most of the adaptation and resilience companies in the industrial sector being well-diversified with a wide variety of products and services. As a case study, we compared our findings with the Taxonomy of Climate Change developed by the United Nations Environment Programme to assess alignment between our findings and the UNEP taxonomy.^[12] The comparison revealed that 745 out of the 3,917 identified products and services were covered in the UNEP taxonomy. Hence, only 309 companies from our first-cut universe of 817 companies would be considered providers of adaptation and resilience products and services that align with the UNEP taxonomy.

Challenges and opportunities

The resulting dataset of adaptation and resilience companies, central to our analysis, presents areas ripe for refinement. Notably, a comprehensive validation of the dataset remains outstanding, a reflection of the project’s proof-of-concept stage. This limitation underscores the exploratory nature of our work and the necessity for further verification of our findings.

There is a specific challenge related to the LLM’s handling of modal verbs when it comes to justifying potential adaptation and resilience activities (Table 2). In some cases, the model predicts a company’s involvement in such activities based on its abbreviated business description. But the ambiguity surrounding these predictions creates a dilemma.

We have to decide whether to exclude these companies conservatively from our dataset or to seek additional information for validation, such as analyzing the full annual filings of each company. This decision depends on balancing the risk of including false positives against the potential of ignoring genuine contributors to climate adaptation and resilience.

Table 2: Example of modal verbs

Suggestions for future exploration

Our research provides a foundation for exploration. To build upon the groundwork established by our initial investigation, future research can enhance the reliability and accuracy of our discoveries in multiple ways:

• Refining the LLM chaining methodology: Developing sophisticated techniques that reduce the LLM’s dependence on modal verbs and minimize instances of hallucination. This refinement aims to improve the precision of identifying pertinent activities.
• Diversifying data sources: To enrich the dataset, it is crucial to incorporate a broader range of data sources, such as corporate TCFD reports, comprehensive annual filings, and direct business descriptions from company websites. This expansion necessitates significant enhancements to the data ingestion pipeline, potentially requiring retrieval augmented generation and vector databasing to manage the increased data complexity.
• Exploring different taxonomies: Conducting case studies across various taxonomies can provide insight into their robustness and applicability. This comparative analysis would offer a novel approach to evaluate and refine current frameworks, highlighting their strengths and weaknesses.
• Incorporating a materiality component: Our analysis does not account for the significance of identified products and services in relation to a company’s entire product offering. Future research could quantify their impact by mapping these offerings to specific Standard Industrial Classification (SIC) codes for each company. This approach, similar to MSCI’s thematic indexing methodology, would enable an assessment of relative revenue generated from adaptation and resilience activities, providing a more nuanced understanding of materiality to each company’s business.

Conclusion

The collaboration between the GARI working group and the MSCI Sustainability Institute has produced an innovative approach to identifying companies key to climate adaptation and resilience. Central to this achievement is the novel use of artificial intelligence, especially LLMs, which offered a groundbreaking method for analyzing vast amounts of textual data automatically. This technological advancement, paired with the critical insights of MSCI sector analysts, ensured the precision and relevance of our findings.

The integration of LLMs into our research highlights a promising future where AI significantly enhances analysts’ capabilities in sustainable investment research. By combining AI’s computational efficiency with human expertise, we’ve taken a significant step towards creating more adaptable and resilient economies. This project not only showcases the potential of AI in the field but also sets a precedent for its application in aiding analysts to navigate the complexities of climate change adaptation and resilience efforts more effectively.