The Representation Illusion: Why Your LLM's Biggest Layer Shifts Don't Drive Fine-Tunin...
A landmark mechanistic interpretability study from the University of Liverpool reveals that transformer layers undergoing the most dramatic representational shifts during fine-tuning have virtually zero correlation with the sparse causal circuits driving downstream accuracy. By demonstrating that just 400 attention and MLP edges deliver over 90% of task performance while shared causal circuits trigger up to 84% cross-task degradation, the findings overturn conventional heuristics for model adaptation.

By Ajinkya Pawar
Head of Search & AI Intelligence • The AI NEWS
Key Developments & Executive Briefing
Sub-Graph Sparsity in Fine-Tuning
Milestone≥90% PerformanceAttribution patching across GPT-2, Qwen2, and LLaMA models confirms that retaining as few as 400 critical computational edges reproduces at least 90% of full downstream task performance, proving task adaptation is governed by isolated micro-circuits.
The Causal-Representational Decoupling
Ecosystem0.40–0.74 EntropyWhile Centered Kernel Alignment (CKA) demonstrates that representational shifts concentrate heavily in the final 10% of layers (90th percentile to output), true causal importance localizes tightly within sparse intermediate circuits with low normalized entropy (0.40–0.74).
Mitigating Negative Transfer via Causal Freezing
Action2.05%–84.34% GainFreezing intersecting causal components prior to adapting models across conflicting task pairs prevented catastrophic gradient interference, recovering between 2.05% and 84.34% of secondary task capability across evaluated architectures.
Executive Summary: The Fallacy of Layer-Wise Representational Drift
In modern deep learning engineering, fine-tuning is universally treated as the gold standard for adapting pre-trained foundation models to specialized domain tasks. Whether adjusting an enterprise conversational agent, deploying a specialized question-answering system, or fine-tuning open-weights models for high-stakes clinical and legal workflows, the operating assumption has remained largely unchallenged: layers that undergo the greatest representational transformation during training must be the primary engines driving downstream performance.
A seminal mechanistic interpretability investigation released in September 2026 by researchers Lingfang Li, Procheta Sen, Shubham Das, and Danushka Bollegala from the University of Liverpool directly refutes this intuition. Titled "Decoupling Internal Representational Changes and Causal Importance in Fine-Tuned Large Language Models" (arXiv:2609.21113), the study provides rigorous empirical and causal proof that macroscopic representational shifts across transformer layers have almost zero correlation with the sparse computational circuits that actually execute the downstream task.
By employing Edge Attribution Patching (EAP) alongside activation patching across four distinct model families—GPT-2 Small, Qwen2-0.5B, LLaMA-3.2-1B, and LLaMA-2-7B—and six benchmark datasets across three canonical natural language processing paradigms, the researchers established two foundational discoveries. First, a mere 400 computational edges out of millions of network connections account for more than 90% of downstream task performance. Second, the layers where representations change the most—predominantly the terminal 10% of the network—are largely decoupled from the localized intermediate layers where causal relevance actually resides.
For machine learning architects, these findings mark a strategic inflection point: traditional parameter-efficient adaptation strategies that distribute adapter rank evenly or focus computational attention on late-stage layers are optimizing for superficial feature drift rather than functional task execution.
+-------------------------------------------------------------------------+
| THE CAUSAL-REPRESENTATIONAL DISCONNECT |
+-------------------------------------------------------------------------+
| Layer Depth: 0% (Inputs) ------------ 80% ------------ 100% (Logits) |
| |
| Observed Feature Shift (CKA / KL-Divergence): |
| [Low Drift: Layers 1-20] --------------------> [Severe Drift: 90%-100%]|
| |
| True Causal Importance (EAP Saliency Scores): |
| [Negligible] ----> [High Causal Density: Layers 10-24] ----> [Sparse] |
| |
| Engineers optimizing where the network shifts most are optimizing |
| for linguistic reformatting, not functional decision logic. |
+-------------------------------------------------------------------------+The Experimental Testbed: Profiling Four Architectures Across Three Task Families
To systematically isolate how fine-tuning alters internal model mechanics, the University of Liverpool team constructed a multi-architecture, multi-task experimental matrix:
- 1.Evaluated Architectures: The investigation covered parameter regimes from lightweight edge models to standard production sizes:
- GPT-2 Small (124M parameters)
- Qwen2-0.5B (490M parameters)
- LLaMA-3.2-1B (1.23B parameters)
- LLaMA-2-7B (6.74B parameters)
All models underwent full-parameter supervised fine-tuning on a standardized budget of 30,000 training instances per task family, executed across NVIDIA A100 80GB hardware within an efficient computational footprint of roughly 14 GPU hours.
- 1.Downstream Task Families & Datasets:
- Sentiment Classification (Discrete Label Prediction): Stanford Sentiment Treebank (SST-2) and Large Yelp Review Dataset.
- Question Answering (Context-Grounded Span Extraction): Stanford Question Answering Dataset (SQuAD v1.1) and Conversational Question Answering (CoQA).
- Machine Translation (Sequence-to-Sequence Generation): KDE4 English-to-French parallel corpus and the Tatoeba multilingual corpus.
- 1.Dual-Lens Analytical Framework:
- Model-Level Representational Metrics: Layer-wise row-wise Kullback-Leibler (KL) divergence over multi-head attention distributions, output-mediated intermediate Logit Lens decoding accuracy/BERTScore/F1, and unmediated Centered Kernel Alignment (CKA) between base and fine-tuned hidden states.
- Causal Importance Attribution: Edge Attribution Patching (EAP), which computes the saliency of every computational edge linking attention heads, MLP blocks, embeddings, and logit unembedding nodes using a first-order Taylor series approximation against systematically corrupted inputs.
To guarantee that EAP attribution scores reflected authentic computational necessity rather than first-order linearization artifacts, the researchers validated their circuits through direct activation patching. Replacing the clean activations of the top 400 EAP-identified components with corrupted counterparts triggered catastrophic task failure, far exceeding the degradation induced by patching identical numbers of randomly sampled components across 20 randomized seeds.
Core Finding 1: Macro Representational Drift vs. Micro Causal Localization
The primary revelation of the study centers on the structural divergence between where representations mutate and where task reasoning occurs.
The Dynamics of Feature Preservation Across Scale
Analyzing attention matrix KL divergence revealed a stark inverse scaling law between parameter count and representational volatility. Under full-parameter fine-tuning on identical data, the lightweight LLaMA-3.2-1B model exhibited severe average KL divergence values fluctuating between 7 and 10 across its attention heads. In contrast, the 7-billion-parameter LLaMA-2-7B maintained remarkable representational stability, exhibiting KL divergence values tightly constrained between 0 and 1.
This disparity corroborates foundational theoretical insights on representational collapse: larger foundation models enter downstream adaptation with rich, pre-crystallized linguistic and reasoning primitives. Consequently, they require only minor navigational adjustments to their existing manifold, whereas smaller architectures undergo violent internal restructuring to accommodate task objectives.
The Late-Layer Trap
When measuring unmediated representational similarity using Centered Kernel Alignment (CKA), the authors observed that representational change is heavily concentrated in the final decile of transformer depth. From approximately the 90th percentile layer through the final unembedding projection, CKA similarity plummeted to ranges between 0.7 and 0.9. The early and middle layers remained largely static in geometric feature space. Probing classifiers (Logit Lens) mirrored this trend: intermediate decodability improved almost exclusively within late layers, while initial layers preserved foundational linguistic syntax.
However, when plotting the distribution of EAP-identified causal edges across layers, an entirely different topology emerged:
- Hyper-Localized Functional Circuits: The layer-wise causal importance distribution exhibited extremely low normalized Shannon entropy (H_EAP = 0.40 to 0.74), whereas attention pattern change was broadly diffused across the network (H_attn approaching near-maximal entropy).
- Zero Correlation to Representational Peaks: Across all four model families and six datasets, the layers containing the highest concentration of causally critical attention heads and MLP blocks did not coincide with the late layers undergoing massive CKA drift. The functional workhorse circuits were concentrated in specific intermediate tiers of the transformer stack.
In short: the layers that shift the most during fine-tuning are not doing the heavy computational lifting. They are primarily adjusting output probability calibrations and vocabulary projection geometries, while the actual task reasoning is executed by compact, undisturbed intermediate circuits.
Core Finding 2: The 400-Edge Threshold and Component Topography
Perhaps the most striking technical data point is the extreme computational sparsity of task adaptation:
Across every evaluated model—from 124M-parameter GPT-2 to 7B-parameter LLaMA—retaining solely the top 400 computational edges identified by Edge Attribution Patching and zeroing out all remaining components was sufficient to preserve at least 90% of the model's peak fine-tuned task performance.
When examining the architectural composition of these critical 400 edges, the authors mapped the functional roles of specific transformer modules:
- Raw Component Distribution: Raw counts were heavily dominated by multi-layer perceptron (MLP) sub-layers and multi-head self-attention heads.
- Architecture-Normalized Over-Representation: Because MLPs and attention heads naturally dominate parameter counts, the researchers calculated an architecture-normalized scaling factor (r_c). Under normalized frequency, logit unembedding nodes and specific intermediate MLP blocks exhibited dramatic over-representation (r_c > 1.0), proving that specific MLP key-value memories and direct unembedding paths are disproportionately vital to task execution.
Task-Dependent Induction Head Engagement
The authors extended their circuit analysis to induction heads—specialized two-head attention circuits identified in mechanistic literature that perform in-context sequence copy and pattern completion. Using the synthetic repeated-sequence protocol via TransformerLens, they evaluated how frequently induction heads appeared within the top causally important attention heads:
- Extractive QA Relies on Induction Circuits: For LLaMA-3.2-1B and LLaMA-2-7B, extractive question answering on SQuAD v1.1 demonstrated high overlap with induction heads, aligning with the requirement to locate and copy specific answer spans from prompt contexts.
- Generative Translation Bypasses Induction: Machine translation exhibited significantly lower induction head overlap in LLaMA models, relying instead on distributed MLP semantic lookups.
- The Qwen2-0.5B Anomaly: In the compact Qwen2-0.5B model, induction head reuse spiked to 7.19% on the KDE4 translation benchmark. Due to extreme attention-head scarcity in sub-billion parameter configurations, compact models are forced to repurpose generic pattern-matching induction circuits for token translation mappings.
Core Finding 3: Destructive Interference in Multi-Task Adaptation
Machine learning teams frequently fine-tune models sequentially or merge task-specific checkpoints under the assumption that shared representations facilitate cross-task transfer. The Liverpool study's findings directly challenge this assumption.
Across 24 distinct task-pair evaluations, fine-tuning a model on Task 1 (e.g., Sentiment Analysis) consistently caused performance degradation on Task 2 (e.g., Machine Translation) relative to the original pre-trained baseline. Crucially, the researchers observed a strong positive correlation across Qwen2, LLaMA-3.2, and LLaMA-2:
The higher the overlap in EAP-identified causal components between Task 1 and Task 2, the more severe the performance collapse on Task 2.
+-------------------------------------------------------------------------+
| THE MECHANISM OF TASK INTERFERENCE |
+-------------------------------------------------------------------------+
| |
| Task A (Classification) Task B (Generative QA) |
| Causal Circuit Set: {C_A} Causal Circuit Set: {C_B} |
| |
| Intersecting Edges: {C_A} ∩ {C_B} |
| |
| Standard Fine-Tuning on Task A: |
| --> Destructive gradient updates overwrite {C_A} ∩ {C_B} |
| --> Task B performance experiences catastrophic degradation |
| |
| Causal Intervention (Freezing Intersecting Edges): |
| --> Freeze parameters in {C_A} ∩ {C_B} |
| --> Fine-tune only disjoint edges {C_A} \ {C_B} |
| --> Recovers 2.05% to 84.34% of Task B performance |
+-------------------------------------------------------------------------+When two distinct tasks rely on intersecting attention heads or MLP neurons, standard backpropagation on the first task forces those shared components to optimize for its specific loss manifold. Because the tasks differ in objective (e.g., categorical classification vs. autoregressive token generation), this adaptation corrupts the routing logic required by the secondary task.
The Causal Freezing Intervention
To prove that shared components were directly responsible for this interference rather than merely correlated, the researchers conducted a surgical intervention: they mapped the intersecting EAP components between task pairs, froze those specific parameters, and fine-tuned the model on Task 1 using only the remaining disjoint network parameters.
The results were definitive: across every evaluated model and task pair, protecting the shared subcircuits recovered between 2.05% and 84.34% of the performance otherwise lost on Task 2. This represents empirical proof that catastrophic forgetting in fine-tuned LLMs is a localized circuit collision rather than a diffuse network-wide failure.
Technical Implementation: Causal Saliency Estimation
To illustrate how Edge Attribution Patching can be integrated into production fine-tuning pipelines to isolate and protect causal subgraphs, the following Python reference demonstrates edge saliency estimation using first-order Taylor expansion:
import torch
from typing import Dict, List, Tuple
def compute_edge_attribution_saliency(
model: torch.nn.Module,
clean_batch: Dict[str, torch.Tensor],
corrupted_batch: Dict[str, torch.Tensor],
target_metric_fn: callable
) -> Dict[str, float]:
"""
Computes first-order Taylor approximation of edge attribution saliency (EAP).
Estimates causal importance: S_{u->v} = |< (a_v(clean) - a_v(corrupted)), grad_{a_v} L >|
"""
model.eval()
# Step 1: Forward pass on corrupted inputs to capture corrupted activations
corrupted_activations = {}
def hook_corrupted(name):
def fn(module, input, output):
corrupted_activations[name] = (output[0] if isinstance(output, tuple) else output).detach()
return fn
handles = []
for name, module in model.named_modules():
if "mlp" in name or "self_attn" in name:
handles.append(module.register_forward_hook(hook_corrupted(name)))
with torch.no_grad():
_ = model(**corrupted_batch)
for h in handles:
h.remove()
# Step 2: Forward and backward pass on clean inputs to capture sensitivity gradient
clean_activations = {}
handles = []
def hook_clean(name):
def fn(module, input, output):
act = output[0] if isinstance(output, tuple) else output
act.retain_grad()
clean_activations[name] = act
return fn
for name, module in model.named_modules():
if "mlp" in name or "self_attn" in name:
handles.append(module.register_forward_hook(hook_clean(name)))
clean_outputs = model(**clean_batch)
loss = target_metric_fn(clean_outputs)
loss.backward()
for h in handles:
h.remove()
# Step 3: Compute edge saliency via inner product
saliency_scores = {}
for name in clean_activations:
if name in corrupted_activations and clean_activations[name].grad is not None:
delta_act = clean_activations[name] - corrupted_activations[name]
grad = clean_activations[name].grad
# Saliency is the absolute dot-product over sequence and batch dimensions
score = torch.sum(delta_act * grad).abs().item()
saliency_scores[name] = score
return saliency_scores
def freeze_overlapping_circuits(
model: torch.nn.Module,
task_a_saliency: Dict[str, float],
task_b_saliency: Dict[str, float],
top_k: int = 400
) -> List[str]:
"""
Identifies top-K components for each task, finds their intersection,
and freezes those parameters to prevent destructive cross-task interference.
"""
top_a = set(sorted(task_a_saliency, key=task_a_saliency.get, reverse=True)[:top_k])
top_b = set(sorted(task_b_saliency, key=task_b_saliency.get, reverse=True)[:top_k])
shared_circuit = top_a.intersection(top_b)
frozen_modules = []
for name, param in model.named_parameters():
for module_name in shared_circuit:
if module_name in name:
param.requires_grad = False
frozen_modules.append(name)
break
return frozen_modulesArchitectural & Strategic Playbook for AI Teams
The insights from the University of Liverpool's research require immediate changes to how engineering organizations approach fine-tuning, model pruning, and continuous learning:
1. Shift Parameter-Efficient Adaptation to Intermediate Routing Layers
Most teams applying Low-Rank Adaptation (LoRA) apply uniform rank allocations across all attention heads or target the final projection layers where gradient activity appears highest. The EAP evidence shows this is counterproductive: terminal layers primarily undergo superficial lexical remapping. Teams should bias adapter ranks toward intermediate MLP blocks and attention heads that host task-specific causal circuits.
2. Implement Circuit Auditing in Multi-Tenant Checkpoints
When training multi-task foundation models or deploying sequential continual learning pipelines, engineering teams must abandon naive sequential fine-tuning. By integrating attribution patching into the pre-training evaluation harness, teams can catalog task-specific subgraphs and systematically freeze overlapping causal nodes, preventing up to 84% secondary-task degradation.
3. Sparse Pruning Based on Causal Attribution, Not Weight Magnitude
Model compression and pruning frameworks frequently rely on weight magnitude pruning (e.g., L1/L2 norm) or activation magnitude. Given that only 400 edges maintain over 90% of model performance on specific capabilities, pruning algorithms that leverage Taylor-approximated edge attribution can achieve extreme sparsification ratios without sacrificing downstream domain accuracy.
Sources & References
Related Coverage
Discussion (0)
Be the first to share insights on this story.