The internal causal mechanisms of Large Language Models (LLMs) are in the spotlight.

In a brand-new paper, Jack Lindsey, Wes Gurnee, Nicholas Sofroniew, and colleagues from Anthropic Research present J-Lens, a novel interpretability technique.

J-Lens surfaces concepts from internal LLM activity during inference. Those concepts could be analogous to consciously accessible thoughts in humans.

This interpretability technique allows us to know, roughly speaking, what the model is thinking. The model could even, as the authors show, be thinking about one thing while answering something different.

I found two elements in this paper that might be of particular interest to the causal community.

How J-Lens Works

The first is how J-Lens works. As the authors explain, “the basic idea is to characterize an intermediate activation vector by its first-order causal effect on the model’s outputs, over a broad distribution of potential contexts.” (Emphasis added.).

This method asks how an output would be affected by the presence of a concept within a given intermediate vector.

Our readers might remember a review we wrote earlier this year on a Max Planck Institute paper about how to produce intervenable features inside LLMs using Sparse Autoencoders (SAEs). (By the way, that paper was accepted this month at COLM.)

Instead of deliberately producing those features, Anthropic researchers found this representational space to be an emergent property of LLMs that is discoverable with J-Lens. They call this representational space J-Space.

But interestingly, some of the examples they present follow the same logic as the paper from the Max Planck Institute.

In one example, the authors ask the LLM to tell “the number of legs on the animal that spins webs”. The model answers ‘8’, and it is possible to find through the J-Lens the word ‘spider’. Now, if we replace the vector representing ‘spider’ in the J-Space with the vector representing ‘ant’, then the model’s answer changes to ‘6’.

A New Training Technique

The second interesting element in the paper from a causal standpoint is a new training technique based on counterfactual reasoning. They call it Counterfactual Reflection Training.

The training implemented in the models consists of doing one thing: if interrupted mid-task and asked to reflect, articulate a set of ethical principles.

The hypothesis is that training the model to articulate principles in counterfactual reflective continuations of a context will populate the J-Space in the original context with concepts related to those principles and thereby shape the model’s behavior.

Code is available here.

Explore the paper

Written by Alberto D. Horner

The opportunity to leverage Causal Inference for LLM development

To say that Causal Inference is underutilized in the biggest area of the AI industry is not a statement that could be passed inadvertently. And that is exactly the case.

In a brand-new KDD paper, an interdisciplinary group (LMU Munich, Carnegie Mellon, Imperial College London, and University of Toronto) presents where, how and why the opportunity to leverage causal methods can be found in the Large Language Models (LLMs) development and evaluation lifecycle.

Most decisions within the LLM lifecycle can be better framed as treatments.

Where to apply it

Imagine that, before fine-tuning your model, you could effectively calculate how much advantage and improvement your team and end users would get. You could be confident whether that decision deserves your effort or not.

In addition to that, most logged data that comes from LLM apps is biased.

Think about a router that sends more complex queries to higher reasoning LLMs. Based on outcomes, it could seem that the strongest LLM is performing worse than a distilled or lighter one.

Causal methods could help you debias your data, and even more. You can answer a variety of questions such as whether the data you have is enough to ground up a confident decision, something that causal inference traditionally calls identifiability.

You will find a clarifying note at the beginning of the paper: the intersection of causal reasoning and LLMs has been explored regarding how LLMs could be equipped with causal reasoning abilities.

What we have here is a novel approach: how could causal inference add value to LLM development projects and guide them in building, evaluating, and optimizing LLM systems more reliably.

Enriching for a variety of roles

From that perspective, this paper could be enriching for a variety of role: business owners, project managers, engineers, researchers and sales professionals.

A stakeholder can know how much value is being created with a feature improvement. Developers will discover a whole new horizon to improve their models and align them with the expected task at hand.

The paper explicitly flags research opportunities that allow researchers to focus on one problem and contribute to this area; from a research standpoint, this is just the beginning.

Sales professionals, on the other hand, will find a powerful tool to frame value propositions and ROI measurements that enable them to build trust and long-term value for their clients.

The authors present clear examples regarding pre-training, evaluation, retrieval, routing, and offline policy learning.

Changing chunking strategy, lowering temperature, using prompt engineering, and increasing some domain weights during pretraining are all treatments linked to outcomes such as model performance, user preference or retrieval relevance.

Explore the paper

Written by Alberto D. Horner

What Matters to You, Individually

A growing field of study within causal inference is personalized decision-making.

While population-based decision-making concerns a subpopulation resembling a given individual, this area of study includes your individual beliefs and priorities as part of the analysis, that is, what matters particularly to you.

Personalized decision-making focuses on Individual Treatment Effects (ITEs), defined as the difference between an individual’s outcomes under two alternative interventions, such as treatment and control.

Two important caveats for ITEs are:

1. In general, they are not point-identifiable.

2. From the estimation standpoint, ITE will be the same for all individuals that share the same measured attributes.

Even so, once we consider causal effects at the individual level, we open the door to include distinct beliefs and utility functions for every individual.

For high-stakes scenarios like medical applications, this approach turns out to be considerably advantageous compared to subpopulation approaches such as Conditional Average Treatment Effect (CATE) optimization.

A crucial moment for ITEs

Personalized decision-making has been motivated by medical applications.

Healthcare is estimated to be globally a $14 trillion market, as Nate Gross, Head of Health at OpenAI, said in a recent podcast by The Economist.

In that podcast, he focuses on the potential of GenAI for Health on information availability, patient awareness, and process automation. As far as I see, there are no theoretical guarantees on individual alignment for those applications.

With personalized decision-making, we are talking about something entirely different.

It means suspending the assumption that the physician knows what is personally important to the patient and enables an approach that includes all individual circumstances.

The probability of harm

Two events that turn out to be of particular interest are harm and benefit.

For different individuals pertaining to the same subpopulation, the same drug might save one of them and doom the other.

The probability of harm is nothing but the probability for a given individual to get an undesired outcome under treatment that would not have been observed without the treatment.

Formalizing the probabilities of these events allows us to further investigate how to leverage the effectiveness of a drug and avoid its undesired effects.

In a recent paper, Scott Mueller and Judea Pearl present an example where the effectiveness of a decision strategy is raised from 28% to 62% due to this personalized approach.

State of the art

Conceptually, probabilities of harm and benefit, along with personalized decision making, are tightly related to the probabilities of necessity and sufficiency.

The following are three non-exhaustive important references in the field:

1. Judea Pearl’s book Causality (Chapter 9) presents theoretical bounds for individual-level probabilities.

2. In 2000, Jin Tian and Judea Pearl published a paper on how to define bounds for probabilities of necessity and sufficiency by combining experimental with non-experimental data.

3. In addition to many published papers (2022, 2023, 2025), Scott Mueller (UCLA) wrote a PhD dissertation on “Personalized and situation-specific Decision Making”, this is, as far as I know, the most comprehensive resource on the topic.

Written by Alberto D. Horner

Counterfactual Fairness

In fairness research we often talk about so-called sensitive attributes.

Two popular examples are gender and race.

We typically wish to prevent these attributes from playing a role in settings such as credit lending or recidivism prediction.

Would the bank have approved the credit had person X been male rather than female?

Counterfactual Fairness (CF) is achieved when the prediction for an individual is the same as that in a counterfactual world with a different value for a given sensitive attribute.

The challenge

But here’s the challenge:

How to prevent a given sensitive attribute from influencing our predictions?

If we remove it from the dataset, the model could be implicitly inferring it based on other attributes.

Another option could be to set that variable to a different value, but that wouldn’t be a valid counterfactual, since other attributes are also affected by the sensitive ones.

Furthermore, there is a trade-off: CF usually comes at the expense of accuracy.

For the CFO of a bank, lowering accuracy by 2% means losing 2% of revenue.

If a bank gives up too much accuracy, it could go bankrupt and end up incapable of lending credit to anyone.

A brand-new Counterfactual Fairness estimation model

In a brand-new paper, Yuchen Ma, Valentyn Melnychuk and colleagues (MCML at LMU), present A Consistent End-to-End Estimation for Counterfactual Fairness.

The authors train a Generative Adversarial Network (GAN) called the Generative Counterfactual Fairness Network (GCFN), which ensures counterfactual fairness with theoretical guarantees in a tailored fashion.

Their approach follows the Standard Fairness Model, which assumes a causal graph with the structure presented in Figure 1 above.

The focus lies on mediators because those variables capture the entire influence of the sensitive attribute and its descendants on the target. Thus, this method removes information from the sensitive attribute while keeping remaining useful information to maintain high performance.

The trade-off parameter

There is a lemma presented in the paper on how a Counterfactual Mediator Regularization effectively forces the predictor to be more counterfactually fair.

Moreover, the authors introduce a weight parameter λ that defines how much regularization affects the overall model.

This way, it is possible to balance the trade-off between counterfactual fairness and performance.

Enabling decisions

The following plot shows how accuracy and counterfactual fairness change for one of the analyzed datasets:

Let’s say that our CFO considers it unacceptable to give up more than 0.0075 of accuracy. Then, it is possible to choose a fairness weight (say, λ=0.6) that maximizes CF under the given constraints:

Alberto’s verdict

Definitely, a significant step towards Responsible AI.

Written by Alberto D. Horner

Explore the paper

Explore the code

A brand-new book on causality just landed!

What to expect?

A clear conceptual introduction to causal reasoning for decision-makers.

What’s in it?

While most books on causal inference target data scientists, statisticians, or computer scientists, this book contains neither a single math formula nor a line of code. The Causal Mindset by Quentin Gallea focuses on how a causality-aware leader would be deliberate, act, and evaluate. For that purpose, the author has designed The Causal Mindset Framework, which consists of five key steps:

1. Identify the comparison being made between your actual case and the counterfactual scenario and evaluate its quality.
2. Search for hidden factors. Is there something else?
3. Determine whether these hidden factors are amplifying or attenuating the results.
4. Iterate and refine the analysis.
5. Assess whether the findings can be reliably applied to a different context or to the future. Can we extrapolate?

Why it matters

These steps may sound familiar to the causal community. What makes the framework special is how the author proposes them as part of a decision-making process in a big company, and many other real-world situations.

One of the highlights of the book (spoiler alert) is when the author discusses Key Performance Indicators (KPIs). When people are expected to achieve a KPI, they might end up focusing on the appearance of success rather than the underlying real goal.

Doctors, for instance, might take mortality rates as a KPI, since a doctor is expected to save lives. However, that might nudge them to be unwilling to treat risky patients.

Leaders with a causal mindset can avoid these side effects and focus on producing the right ones.

Our verdict

Managers, businessmen and policymakers will love it!

Get a copy!

When it comes to knowing what is going on inside a Large Language Model (LLM), Sparse Autoencoders (SAEs) are popular mechanisms that can discover features in the model’s representation space. This makes models more interpretable.

In a brand-new paper, Moritz Miller, Florent Draye, and Bernhard Schölkopf from the Max Planck Institute for Intelligent Systems, take a step further. Inspired by the idea of Independent Causal Mechanisms (ICM), they fine-tuned a model to let the features be almost orthogonal.

Then, in addition to being human understandable, this enhances the model with additional properties without significantly compromising its accuracy:

1. For a given input, it is possible to identify how a feature is independently involved in producing the output.

2. It is possible to intervene in a feature and observe how the output changes in that respect while remaining invariant in every other aspect.

Additionally, the authors suggest that intervenable features might mitigate adversarial vulnerabilities, leading towards more robust and safer models, but this requires further investigation.

The figure above shows how intervention correctness increases as the SAE’s features are enforced to be more distinct, while the model’s accuracy remains essentially unchanged. The parameter λ is the penalty for non-orthogonality in the loss function.

In contrast to traditional causality, which studies interventions as distributional shifts in relation to an underlying data-generation-process, here the authors study intervenability on the modeling level.

The purpose is to uncover causal mechanisms inside the model’s representation space.

The results are limited to the MetaMathQA dataset (math problems), and one SAE variant (TopK). Therefore, the authors invite future research to focus on tasks different from mathematical reasoning, and other SAE variants like JumpReLU.

This is an important step towards a more human-aligned and safer AI, making LLM internals more transparent and controllable.

Code for the paper is available here.

Explore the Paper

In-context learning is an emergent property of foundation models that raised a lot of excitement when it first became to known.

In simple terms, it allows a model to learn from the data and context available at inference time.

As it turns out, it can also be immensely useful in causal inference.

I decided to test out three major foundation models for causal inference using my personal Windows machine.

The three models are:

• DoPFN (Prior Labs and Max Planck Institute)

• CausalFM (LMU Munich and MCML)

• CausalPFN (Toronto and Layer 6 AI)

All three algorithms are built upon TabPFN, a foundational model developed by Prior Labs and originally designed to address the abundance of small tabular datasets.

In this short article, I want to share with you my learnings, assuming that for some of our readers, using these algorithms in a local environment might be far easier and more appealing than managing to get access to an NVIDIA A100.

Here’s what I found.

From the accessibility standpoint, CausalPFN was the easiest one to get running.

A simple “uv add causalpfn” was sufficient to make the model ready for use.

On the other end of the spectrum was Do-PFN.

Despite many hours spent attempting to set it up, I was not able to make it run on my Windows machine.

After many failed attempts, I decided to replace Do-PFN with an S-Learner with TabPFN as the base model.

I focused on CATE estimation and used PEHE (Precision in Estimation of Heterogeneous Effect) to evaluate them.

I generated test datasets using the code available in the CausalFM repo (datasets 1-10) and on the Prior Labs website (dataset 11).

The results are plotted in the figure above (the lower the PEHE, the better the performance).

You can find the numerical values in the table below:

To play with the code yourself, check the notebook.

Over the past few years, large language models (LLMs) have been increasingly gaining attention in the causal community as potential helpers in causal discovery, even though they’re not perfect experts.

A recent paper presented at the ICLR by Aniket Vashishtha, Amit Sharma and a cross-disciplinary team (Microsoft Research, UIUC, CISPA, MIT, and IIT Hyderabad), argues that causal order is a more effective approach for leveraging expert knowledge than forcing it to convey a completely defined causal graph.

As they point out, an optimal (perfect) expert prompted with only two variables at a time could incorrectly predict the underlying causal graph but will always predict the causal order correctly.

The authors also propose a new strategy to prompt any expert (human or not, perfect or not).

Instead of asking about each pair of variables in isolation, include a third variable for context.

This triplet method improves performance across several metrics, including the number of cycles, Structural Matching Error (SME), and a novel measure based on topological divergence.

By balancing additional context with minimal added complexity, the triplet method reliably reduces error.

In fact, the probability of error when using triplets is always less than or equal to the probability of error with standard pairwise prompts.

The code is not currently available, but the authors promised to add it soon here.

Explore the paper

Written by Alberto D. Horner

Most of the widespread Machine Learning algorithms are based on purely statistical methods. They are based on the dependence between available data and a target variable (or set of variables). For instance, a drivers app algorithm collects data from other users’ trajectories and outputs the probably fastest path for you to drive through. In those cases, purely statistical algorithms are perhaps the best approach you can have.

Now, think about a second example: a company that wants to evaluate whether a marketing campaign was successful or not. They may find an association between the marketing campaign and an increase in sales; they may also have achieved the purported goals on their sales and revenue, which are the reason why they launched that marketing campaign. Therefore, they dare to conclude that the marketing campaign was successful. Should they run that campaign again next year? In these cases, Causal AI is surely a better approach.

The main difference between the former and the later example is that the former is looking for optimization, while the later is looking for a decision rule. Drivers do not have control over the streets. They’ll pick always the quickest path. On the other hand, business leaders do have control over marketing campaigns. Then, before deciding to relaunch the same campaign, it would be nice to know that the increase in sales they’ve observed was, in fact, due to the campaign. They can intervene in the situation in order to achieve their goals. And that’s the point with causal AI: it takes a step beyond association and deals with interventional and counterfactual data.

The main idea in causal reasoning is condensed in the ladder of causation, an analogy that tells that you can ascend into a more structural knowledge than the associative one. There are three kinds of causal knowledge, which can be mapped to three kinds of questions. The first kind is associative knowledge: was the increase in sales associated with our marketing campaign? The second kind is interventional knowledge: what if we relaunch the campaign? Will it yield the same results? The third kind is counterfactual: what would have happened if we hadn’t launched that campaign? Would sales have increased anyway? The second and third levels can only be reached using causal reasoning (or simulating it), and that is the kind of information that managers really want to have.

The Ladder of Causation. A drawing by Maayan Harel.

It’s a business thing

Causal AI has its own history as a scientific project. Judea Pearl must be noted as the scientist who laid out the sound foundations of causal inference; his book Causality: Models, Reasoning and Inference is now a canonical book. But what about causal AI as a business project?

Currently, the leading project for developing causal AI applications is pywhy, a python library that emerged from the joined efforts of Microsoft and Amazon research. We may focus on that project to determine the current state of causal AI for business. A good approach to take is that of Hurwitz and Thompson.

Judith S. Hurwitz and John K. Thompson recently published a book titled Causal Artificial Intelligence: The Next Step in Effective Business AI. They assess innovative software as a process that takes, at least, three iterations: “The first iteration proves that the idea will work. The second attempt refines the approach and implementation, and the third round of work standardizes, hardens, and builds software that is reliable, robust, and scalable.” They depict these three steps as the beginning of a continuous improvement process. Most important of all, they say that “the causal AI software market is in the process of moving past the three-cycle model”. In other words, we know that causal AI tools work well, but they are also at their very beginning inside the business world, in such a way that they may be a competitive advantage for innovators.

In their book, Hurwitz and Thompson also warn that Causal AI tools have a long way to walk. “The software that is the basis of causal AI market needs to evolve, it is too hard to use, it has functional gaps, and it is not robust enough for reliable enterprise-class use today.” If you are a business leader, these are good news. It means that you still have time to get prepared for the next industrial breakthrough of AI.

Take a look into the causal levels

Let’s dive deeper into the previous example. Ideally, a good marketing campaign produces an increase in sales. But how could I know if the increase in sales that I have observed was produced by the campaign? Was it perhaps just coincidence and other factors were responsible for that increase? A causal analysis can tell the answer and provide guidelines to keep those increasing results. It is essentially actionable.

You may wonder why a mere plot of sales and revenue before and after the marketing campaign won’t work. Usually, business decisions are not taken in isolation. Around the same lapse of time, the pricing team may have changed their strategy, and the engineers may have come up with a more ergonomic design. It is possible to discover that the last two caused the increase in sales, and not the first one. A causal diagram of the situation looks as follows:

That would mean that the marketing campaign needs a change and that, at least for this hypothetical company, it will be better to focus efforts on design and pricing. While performing the analysis, your team may discover insightful relations among variables, like the role of customer satisfaction as a mediator between the quality of the product’s design and the number of sales (look at the diagram). Furthermore, your causal model will enable you to answer questions like ‘what is the best way to allocate our budget to maximize sales?’ Obviously, assuming that budget allocation has a causal effect on design improvement and pricing optimization. Achieving those results is feasible with the current state of causal AI.

The meaning of this breakthrough

Four years ago, Regina Barzilay, MIT professor whose research has focused on early cancer diagnosis, was asked how far we are from understanding the human body’s biology or some of its diseases, from the perspective of computer science. “One of the reasons that we succeeded in the areas that we as a computer scientists succeeded,” she said, “is that we are not trying to understand in someways. If you think about e-commerce: Amazon doesn´t really understands you, and that’s why it recommends you certain book or certain products, correct? […] If you look on what they do with the recommendation system, they are not claiming that they´re understanding something. They’re just managing to, from patterns of human behavior, recommend you a product.”

Nowadays, then, it may sound surprising that Amazon is part of the pywhy project. The next step in AI is about achieving a better understanding. Think about a movie recommendation algorithm; two users may have watched the same set of movies. A traditional algorithm would recommend them the same set of movies. A causal algorithm, instead, might detect that the reason one user chose those movies was the soundtrack, photography, and all other aesthetic details, while the other user was captivated mainly by the plots, therefore the causal algorithm will recommend different and more adequate set of movies to each user.

Amazon and Microsoft know that causal AI is profitable. But there are more reasons to implement causal AI, like ethical compliance and side-effects prevention. For instance, there is interesting work on algorithmic fairness and accountability for the sake of preventing gender and race bias. Also AI will be much more useful to fight climate crisis if it is enhanced with causal reasoning. It is a field of research and development where we can strive to align technological growth with human goals.