The Future of ESG: Technology, Data, and Regulatory Compliance

Financial and ESG insights begin with big data coupled with data science.

At SESAMm, our artificial intelligence (AI) and natural language processing (NLP) platform analyzes text in billions of web-based articles and messages. It generates investment insights and ESG analysis used in systematic trading, fundamental research, risk management, and sustainability analysis.

This technology enables a more quantitative approach to leveraging the value of web data that is less prone to human bias. It addresses a growing need in public and private investment sectors for robust, timely, and granular sentiment and environment, social, and governance (ESG) data.
This article will outline how the data is derived and illustrate its effectiveness and predictive value.

Content coverage and ESG data collection

The genesis of SESAMm’s process is the high-quality content that comprises its data lake, the source from which it draws its insights. SESAMm scans over four million data sources rigorously selected and curated to maximize coverage of both public and private companies. Three guiding criteria—quality, quantity, and frequency—ensure a consistently high input value.

Every day the system adds millions of articles to the 16 billion already in the data lake, going back to 2008. The coverage is global, with 40% of the sources in English (the U.S. and international) and 60% in multiple languages. The data lake, expanding every month, comprises over 4 million sources, including professional news sites, blogs, social media, and discussion forums.

The following tables illustrate SESAMm’s data lake distribution (Q1 2022):

Respect for personal privacy figures highly in the data gathering process. We don’t capture personal data, like personally identifiable information (PII), and respect all website terms of service and global data handling and privacy laws. SESAMm’s data also doesn’t contain any material non-public information (MNPI).

Deriving financial signals and ESG performance indicators

SESAMm’s new TextReveal® Streams platform applies NLP and AI expertise to process the premium quality content gathered in its data lake. This complex process involves named entity recognition (NER) and disambiguation (NED)—the process of identifying entities and distinguishing like-named entities using contextual analysis—and mapping the complex interrelationships between tens of thousands of public and private entities, connecting companies, products, and brands by supply chain, location, or competitive relationship.

Using SESAMm’s TextReveal Streams, this wealth of information is filtered to focus on four crucial contexts for systematic data processing, risk management, and alpha discovery:

• Sentiment covering major global indices: world equities (and Small Caps, Emerging), U.S. 3000, Europe 600, KOSPI 50, Japan 500, Japan 225
• Sentiment covering all assets and derivatives traded on the Euronext exchange
• Private company sentiment on more than 25,000 private companies
• ESG risks covering 90 major environmental, social, and governance risk categories for the entire company universe, which includes more than 10,000 public and more than 25,000 private companies with worldwide coverage

TextReveal Streams data sets and assessments are used by financial institutions, rating agencies, and the financial services sector, such as hedge funds (quantitative and fundamental) and asset managers, to optimize trade timing and identify new sustainable investment opportunities. Private equity deal and credit teams also use the data for deal sourcing and due diligence. Private equity ESG teams use it to manage initiatives like portfolio company environmental, social, and governance risk and reporting.

Methodology and technology for processing unstructured data

NLP workflow, from data extraction to granular insight aggregation

Data is continually extracted from an expanding universe of over four million sources daily. As it enters the system, it is time-stamped, tagged, indexed, and stored in our data lake to update a point-in-time history extending from 2008 to the present.
The source material is then transformed from raw, unstructured text data into conformed, interconnected, machine-readable data with a precise topic.

Mapping relationships between entities with the Knowledge Graph

At the heart of the text analytics process is SESAMm’s proprietary Knowledge Graph, a vast map connecting and integrating over 70 million related entities and their keywords. It’s essentially a cross-referenced dictionary of keywords, relating each organization to its brands, products, associated executives, names, nicknames, and their exchange identifiers in the case of public companies.

Entities within the Knowledge Graph are updated weekly and tagged to ensure changes are correctly tracked. The CEO of a company today, for example, may not be the CEO tomorrow, and brands may be bought and sold, changing the parent company with each sale. Weekly updates within the Knowledge Graph ensure the system is aware of these changes.

Named entity disambiguation (named entity recognition plus entity linking) is one of the NLP techniques used to identify named entities in text sources using the entities mapped within the Knowledge Graph universe.

At SESAMm, NED identifies named entities based on their context and usage. Text referencing “Elon,” for example, could refer indirectly to Tesla through its CEO or to a university in North Carolina. Only the context allows us to differentiate, and NED considers that context when classifying entities. This method is superior to simple pattern matching, limiting the number of possible matches, requiring frequent manual adjustments, and cannot distinguish homophones.

SESAMm uses three other NLP tools to identify entities and create actionable insights. These are lemmatization, embeddings, and similarity. Each is explained in more detail below.

Analyzing the morphology of words with lemmatization

News articles, blog posts, and social media discussions reference organizations and associated entities in various forms and functions. Lemmatization seeks to standardize these references so the system knows they mean the same thing.

For example, “Tesla,” “his firm,” “the company,” and “it” are all noun phrases that can appear in a single article and refer to a single entity. Even where the reference is apparent, it can take different forms. For example, “Tesla” and “Teslas” both refer to the same entity but have slightly different meanings (semantics) and shapes (morphology).

The lemmatization process standardizes reference shape (morphology) to facilitate identification and aggregation. Lemmatization is a more sophisticated process than stemming, which truncates words to their stem and sometimes deletes information.

Encoding context and meaning with word embedding

In NLP, embedding is a numerical representation of a word that enables its manifold contextual meanings to be calculated relationally. Embeddings are typically real-valued vectors with hundreds of dimensions that encode the contexts in which words appear and, thus, also encode their meanings.
Because they are vectors in a predefined vector space, they can be compared, scaled, added, and subtracted. An example of how this works is that the vector representations of king and queen bear the same relation to each other as the representations of man and woman once you subtract the vector that represents royal.

Using embedding is key to analyzing how words change meaning depending on context and understanding the subtle differences between words that refer to the same concept: synonyms. For example, the words business, company, enterprise, and firm can all refer to the same thing if the context is “organizations.” But they represent different things and even different parts of speech if the context changes.

In the phrase, “[Tesla] will be by far the largest firm by market value ever to join the S&P,” for example, one could replace the word firm with company or enterprise without affecting the meaning significantly. Contrast that with “a firm handshake,” where a similar substitution would render the phrase meaningless.

Also, words referring to the same concept can emphasize slightly different aspects of the concept or imply specific qualities. For example, an enterprise might be assumed to be larger or to have more components than a firm. Embeddings enable machines to make these subtle distinctions.

One advantage of using embedding is that it’s practical because it’s empirically testable. In other words, we can look at actual usage to determine what a word means.

Another advantage is that embeddings are computationally tractable. This understanding of a word’s definition allows us to transform words into computation objects to programmatically examine the contexts in which they appear and, thus, derive their meaning.

As lemmatization is an improvement on stemming, embeddings improve techniques such as one-hot encoding, which is close to the common conception of a definition as a single entry in a dictionary.

SESAMm uses the global vectors for word representation (GloVe) algorithm to generate embeddings. It’s an unsupervised learning algorithm that begins by examining how frequently each word in a text corpus co-occurs with other words in the same corpus. The result is an embedding that encapsulates the word and its context together, allowing SESAMm to identify specific words in a list and different forms of the listed words and unlisted synonyms.

GloVe is an extension of recent approaches to vector representation, combining the global statistics of matrix factorization techniques like latent semantic analysis (LSA) with the local context-based learning of word2vec. The result is an unsupervised algorithm that performs well at capturing meaning and demonstrating it on tasks like calculating analogies and identifying synonyms.

BERT is another algorithm used by SESAMm to generate embeddings. BERT produces word representations that are dynamically informed by the words around them. Google developed the technique, and it’s what’s known as a transformer-based machine learning technique, which means it doesn’t process an input sequence token by token but instead takes the entire sequence as input in one go. This technique is a significant improvement over sequential recurrent neural network (RNN) based models because it can be accelerated by graphics processing units (GPUs).

SESAMm uses BERT for multilingual NLP of its extensive foreign language text because it has been retained using an extensive library of unlabeled data extracted from Wikipedia in over 102 languages. BERT model was trained to predict words from context and next sentence prediction where it was trained to predict if a chosen following sentence was probable or not given the first sentence. As a result of this training process, BERT learned contextual embeddings for words. Due to this comprehensive pre-training, BERT can be finetuned with fewer resources on smaller datasets to optimize its performance on specific tasks.

Linking words, sentences, and topics with cosine similarity

Cosine similarity with centered means it’s identical to the correlation coefficient, which highlights another element of the computational tractability of the embeddings approach. It makes it easy to compare words and contexts for similarity.

Converting words to vector representations means we can quickly and easily compare word similarity by comparing the angle between two vectors. This angle is a function of the projection of one vector onto another. It can identify similar, opposite, or wholly unrelated vectors, which allows us to compute the similarity of the underlying word that the vector represents.

Two vectors aligned in the same orientation will have a similarity measurement of 1, while two orthogonal vectors have a similarity of 0. If two vectors are diametrically opposed, the similarity measurement is -1. In practice, negative similarities are rare, so we clip negative values to 0.

Cosine similarity measures whether two words, sentences, or corpora are close to one another in vector space or “about” the same thing in semantic space. To answer the question, “Is this sentence referencing company X?” we embed the sentence using the process described above and compute the cosine similarity between the sentence and the embedded company profile. Analogously, we compute similarities between sentences and the ESG topics SESAMm monitors by taking the maximum similarity between a sentence and each embedded keyword associated with an ESG topic.

These similarities allow us to identify whether a sentence references fraud, tax avoidance, pollution, or any other ESG risk topic among the more than 90 that SESAMm tracks across the web.

Similarities within ESG topics combine with word counts to resolve the recall and precision problem. Word counts are precise because if a word is identified within a context, then that context, by construction, references the topic.

The virtue of using these NLP techniques is that even if a given keyword list does not include every possible combination of words that a person might use to discuss a topic, relevant entities missed by the word-count process will be identified through vector similarity.

This is the power of SESAMm’s NLP expertise. We can scan many lifetimes’ worth of data in seconds to find the concepts you explicitly ask for and the concepts relevant to your search but that you did not think of yourself.

Sentiment analysis with deep learning and neural networks

Once we’ve identified the concepts and contexts of interest in all the forms they appear, we analyze the context to determine the speakers’ attitudes.

We use sentiment classification models to score a sentence with three possible outcomes: negative, neutral, or positive. The current classification models are based on deep learning AI technologies. Specifically, we stack convolutional neural networks with word embeddings and bayesian optimized hyperparameters—parameters not learned during training. This architecture improves the accuracy and enables fast shipping of production-ready models for a given language. We also produce state-of-the-art frameworks with architecture variations enabling multilingual capabilities, such as transformers and universal sentence encoders.

Condensing information and extracting insights with daily aggregation

Similarities, embedded word counts, and sentiment are state-of-the-art tools for processing unstructured text data. The same tools are effective cross-linguistically.

Once the information has been extracted from millions of data points, it’s aggregated and condensed into actionable insights.

All entities are referenced directly or indirectly within an article. Then, sentence-level references are aggregated to obtain an article-level perspective, and finally, all relevant articles are aggregated to gain an entity-level view of that day.

In this way, reams of data are compressed into several metrics to provide a daily aggregate view for each entity, highlighting trends at a sentence, article, and entity-level comparable over a multi-year history.

ESG analysis use cases

SESAMm’s TextReveal Streams is used in various investment domains, from asset selection to alpha generation and risk management. Systematic hedge funds track retail interest in real time to identify investment opportunities and protect their existing positions. In the Private Equity industry, equity and credit-deal teams use the data in various ways, from monitoring consumer perspectives via forums and customer reviews for evaluating deal prospects to estimating due diligence risks, all to help make investment decisions. Dedicated teams use our data for monitoring portfolio companies for ESG red flags that conventional ESG reporting might miss.

Below are two examples of how aggregated TextReveal Streams data can be used to help identify investment risk and opportunity.

LFIS CapitalL: ESG signals for equity trading

ESG controversies can significantly impact asset prices in the short term, and it’s now estimated that intangible assets, including a company’s ESG rating, account for 90% of its market value.

Working in partnership with LFIS Capital (LFIS), a quantitative asset manager and structured investment solutions provider, SESAMm developed machine learning and NLP algorithms that could analyze ESG keywords in articles, blogs, and social media, to generate a daily ESG score specific to each stock, which is part of the TextReveal Streams’ platform’s core functionality.

The results were promising when these scores were incorporated into a simulated strategy for trading stocks in the Stoxx600 ESG-X index.

A simulated long-only strategy running between 2015 and 2020, using the signals, delivered a 7.9% annualized return, 2.9% higher than the benchmark for similar annualized volatility (17.3% vs. 17.1%). The information ratio of the strategy was greater than 1, with a tracking error of 2.8%. Results for the previous three years were compelling, reflecting the growing interest and news flow around ESG themes.

Researchers also backtested a hypothetical long-short strategy for all stocks in the Stoxx600 ESG-X index with a market cap of over $7.5bn. This investment strategy delivered a Sharpe ratio of approximately 1 with annualized returns and volatility of 6.1% and 5.9%, respectively, between 2015 and 2020. Like the long-only strategy, returns were particularly robust over the three years up to 2020: +6.0% in 2018, +7.3% in 2019, and +11.3% in 2020.

Finally, a simulated “130/30” ESG strategy that combined 100% of the long-only ESG strategy and 30% of the long-short ESG strategy delivered a 10.8% annualized return, 5.8% higher than that of the Stoxx600 ESG-X index. Annualized volatility was similar at 16.9% vs. 17.1%. The strategy experienced a tracking error of 3.8% and an information ratio of over 1.5, with a consistent outperformance each year.

Disclaimer: Past performance is not an indicator of future results. Theoretical calculations are provided for illustrative purposes only. The investment theme illustrations presented herein do not represent transactions currently implemented in any fund or product managed by LFIS.

Read more: “How Alterntive Data Identifies Controversies Before Mainstream Sources With Examples”

Wirecard: ESG sentiment and volume as predictive indicators

The Wirecard scandal broke on June 21, 2020, when newswires carried the story that the major German payment processor had filed for bankruptcy after admitting that €1.9 billion ($2.3 billion) of purported escrow deposits did not exist.

Could SESAMm’s TextReveal Streams platform have provided investors with an early warning that the scandal was about to break?

The following chart derived from the platform shows how key ESG metrics, including ESG scores (volumes) and ESG scores (sentiment), reacted to the news.

An analysis of the charts pinpoints a shallow rise in the ESG scores (volumes) time series in the early part of June before the eruption on June 21.

The ESG scores (sentiment) metric also shows a steady increase in negative sentiment for governance, the most relevant of the three ESG factors regarding the scandal.

Additionally, before the crash, governance was the most negative of the three ESG factors most of the time. This was especially the case from late March to early April, and then before the scandal in early June, negative governance sentiment diverged higher from the other two.

The rate-of-change of negative governance sentiment as it rose and peaked in early June before the scandal broke was also extremely high, perhaps providing the basis for an early warning signal.

Portfolio managers who had been keeping an eye on the reputational slide in Governance for Wirecard may have decided the company was at high risk of a negative controversy emerging, giving them cause to drop the stock before the event.

In this way, it can be seen how while not providing a hard and fast early warning signal, SESAMm’s ESG scores can, nevertheless, be used as the basis for developing a data-driven, rules-based portfolio management approach that can help investors avoid high-risk candidates like Wirecard.

SESAMm takes on ESG data challenges

SESAMm’s NLP and AI tools analyze over four million data sources daily to identify thousands of public and private companies and their related products, brands, identifiers, and nicknames, turning reams of unstructured text into structured and actionable data.

SESAMm’s TextReveal Streams platform can be used in many quantitative, quantamental, and ESG investment use cases. TextReveal is a solution that allows you to fully leverage NLP-driven insights and receive high-quality results through data streams, modular API and dashboard visualization, and signals and alerts.

Learn how SESAMm can support you in your investment decision-making and request a demo today.

To request a demo or for access to the full SESAMm Wirecard or LFIS reports, contact us here:

Environmental, social, and governance (ESG) data. It has become a standard input for institutional investors, asset managers, and private equity firms, shaping investment decisions from first screening to portfolio monitoring and regulatory reporting. Yet the kind of ESG data most investors rely on is often self-reported, slow to update, and hard to compare from one provider to the next.

If you're reading this, you likely use ESG data regularly. You may be comparing ESG data providers, or you may have noticed that a company's ESG rating and its headlines don't always tell the same story. In this guide, we cover where ESG data comes from, why traditional ESG metrics fall short, how AI and natural language processing fill the gaps, and how to choose an ESG data provider.

Who created ESG?

Kofi Annan, former United Nations Secretary-General, invited a group of financial institutions to develop guidance on how to better integrate environmental, social, and governance issues into asset management, securities brokerage, and research. In 2004, they published "Who Cares Wins: Connecting Financial Markets to a Changing World." The work fed into the UN Principles for Responsible Investment, launched in 2006, and helped make ESG criteria a standard part of how investors evaluate companies.

Stronger, more resilient, and sustainable

According to the "Who Cares Wins" report, the contributors were convinced that "in a more globalized, interconnected, and competitive world, the way that environmental, social, and corporate governance issues are managed is part of companies' overall management quality needed to compete successfully."

The endorsing institutions concluded that "a better consideration of environmental, social, and governance factors will ultimately contribute to stronger and more resilient investment markets, as well as contribute to the sustainable development of societies."

Two decades later, that idea supports a large market for ESG data and ESG ratings, and a growing body of regulation, from SFDR and CSRD in Europe to the EU ESG Rating Regulation.

Where does ESG data come from?

ESG data covers a wide range of issues: carbon emissions and other environmental impacts, climate change exposure, labor practices and human rights, product safety, corporate governance, and business ethics. It reaches investors through three main channels.

• Company disclosures and sustainability reporting. Most ESG data starts with what companies publish: sustainability reports, annual filings, and policy documents. ESG frameworks such as the Global Reporting Initiative (GRI) and the Task Force on Climate-related Financial Disclosures (TCFD) aim to make ESG reporting more standardized, and regulations like the EU's CSRD are making parts of it mandatory. Standardization helps with benchmarking, but it doesn't change a basic fact: ESG data mainly comes from the same companies it is meant to evaluate.

• ESG ratings and ESG rating agencies. ESG rating agencies, including some data vendors, such as MSCI, Sustainalytics, and Bloomberg, collect disclosures, add their own research, and turn them into ESG scores. These scores make it easier to compare a company's ESG performance against peers, and they feed into ESG funds, indices, and screening rules.

• Outside-in data. The third channel is what others report about a company: news coverage, NGO investigations, regulatory actions, court records, and trade press. This is where a company's actual conduct shows up, often well before it reaches a sustainability report or a rating. It is also the hardest source to use, because it is spread across millions of documents in dozens of languages.

Today's ESG data challenges

At their core, ESG metrics capture a company's performance on a given ESG issue. Measuring that consistently has proven difficult. In the Journal of Applied Corporate Finance, Sakis Kotsantonis and George Serafeim identified four recurring problems:

1. ESG measurement, data collection, and company reporting practices remain inconsistent.
2. A lack of transparency in benchmarking makes peer rankings hard to rely on.
3. ESG data providers fill data gaps in different ways, which can produce large discrepancies.
4. Differences in how providers interpret the data are considerable, and they grow as more data becomes available.

The problem shows up clearly in the ratings themselves. Researchers at MIT Sloan's Aggregate Confusion Project found that ESG ratings from six prominent ESG rating agencies correlate at just 0.54 on average. By comparison, credit ratings from Moody's and Standard & Poor's correlate at 0.92.

Beyond inconsistency, traditional ESG data has four structural blind spots:

• It's self-reported. Companies decide what to disclose, and few volunteer their own controversies, which is what makes greenwashing possible.
• It's slow. Sustainability reports are annual, and ratings often refresh on a similar cycle.
• It dilutes severe issues. Blending many ESG factors into one score lets strong governance metrics offset a serious environmental or human rights breach.
• It misses private companies. Coverage is thin in low-disclosure markets, where much of private equity invests.

Artificial intelligence (AI) to meet rising ESG data demands

Even as the debate around ESG investing has become more polarized, demand for ESG data keeps growing. In BNP Paribas' 2025 ESG survey of 420 institutional investors, 85% said they integrate sustainability criteria into their investment decisions.

"Nearly half of investors (48%) anticipate allocating more budget to their sustainable investment strategy on ESG data acquisition." (BNP Paribas ESG Survey 2025)

Spending more on the same disclosure-based data doesn't close the gaps above. However, advances in machine learning, natural language processing, and large language models make it possible to extract ESG information from unstructured sources such as news, NGO reports, regulatory filings, and court records. This outside-in data surfaces ESG controversies and events as they are reported, often long before they appear in a rating, and it covers companies that publish little or nothing about themselves.

How to get ESG data using natural language processing (NLP)

Natural language processing (NLP) algorithms can read billions of documents across languages, detect which ones describe an ESG issue, and separate a genuine allegation from a passing mention. Large language models add the ability to summarize what happened, map it to ESG frameworks such as SFDR, CSRD, or the UN Global Compact, and assess its severity.

Two steps make the difference between noise and reliable ESG data:

• Attributing each mention to the right company. Entity resolution and a regularly updated knowledge graph link every article to a single, verified entity, including private companies, subsidiaries, and brands. Without this step, a scandal at a namesake ends up in the wrong portfolio.
• Keeping every signal traceable. Each flag should link back to its source documents. As regulations like the EU ESG Rating Regulation raise the bar for transparency, investors need to show where an assessment came from, not just what it says.

Several ESG rating agencies now integrate NLP-derived datasets, and asset managers and private equity firms use them in their ESG risk management. The same approach powers adverse media screening and adverse media monitoring, which banks, insurers, and investors use to check clients and counterparties for reputational and KYC risk.

Why does ESG risk show up in controversies first

An ESG controversy is an adverse event or conduct attributed to a company in public sources: a chemical spill, a corruption probe, a data breach, a forced labor allegation, a misleading climate claim. Controversies are where stated policy meets actual practice. That is why they are often the earliest visible sign of ESG risk, and why investors increasingly track them alongside ESG ratings.

Controversies also capture risks that disclosures tend to understate. Social factors like labor conditions in a supply chain rarely appear in a sustainability report until something goes wrong. Neither does the climate risk behind a delayed decarbonization plan, or the environmental impact of a single project.

Not every controversy matters equally, so severity is the key question. Two factors drive it: whether the harm can be reversed, and how far it reaches. SESAMm's Controversy Exposure Score is built around that idea. It rates a company's exposure to reported ESG controversies on a 0 to 100 scale over a rolling 24-month window. Rather than averaging, it applies "worst-of" logic, so a company's most severe controversy drives the score, whichever pillar it falls under. It measures impact on people and the environment and reflects what has already been reported, which makes it a complement to ESG ratings rather than a prediction.

For ESG due diligence and portfolio monitoring, that shift from what a company says to what is reported about it changes what investors can see, especially for private companies with little or no disclosure.

How to choose an ESG data provider

ESG data providers fall into roughly three groups. Traditional ESG rating agencies score companies mainly from disclosures. ESG reporting and compliance platforms help companies collect and publish their own data. Risk intelligence providers monitor what is reported about companies from the outside in. Many investors combine more than one. Whichever you choose, five questions separate a strong ESG data provider from the rest:

1. Does it cover the companies you actually hold? If your portfolio is mostly private or in emerging markets, a provider built on listed-company disclosures will leave much of it uncovered.
2. Can you trace every data point to its source? You should be able to click from an ESG score or alert to the documents behind it.
3. Is the methodology public? You need to know how ESG metrics are weighted, how severity is judged, and what a score doesn't measure.
4. How fast does it update? Annual refreshes suit reporting. Due diligence and monitoring need data that updates as events happen.
5. Is it ready for regulation? Ask how the provider supports SFDR, CSRD, and the EU ESG Rating Regulation, and whether it is seeking authorization with ESMA.

For a deeper checklist, see our 7-question buyer's guide to choosing an ESG data provider.

‍

Swiss institutions are being asked to measure exposures that were never disclosed in the first place. FINMA Circular 2026/1, in force since January 2026, moves banks and insurers from climate disclosure to nature-related risk management, and the measurement problem lands well before the 2028 deadline.An asset owner and a private bank open the discussion: where they actually struggle to measure, what they built internally, and what they chose not to build. Inrate and SESAMm answer from the data side. One question runs through the session: does sustainable investing have a data problem, or a classification problem? Our panellists do not agree, which is the point.

‍

‍

Speakers

Nicolas Barben

Group Head of Sustainable Investment
UBP

Sasha Cisar

Specialist for Sustainable Investments
Nest

Saurabh Srivastava

Managing Director
Inrate

Sylvain Forté

CEO
SESAMm

Romain Leroy-Castillo (Moderator)

Director Artificial Intelligence and Education
Swiss Sustainable Finance

‍