Kateřina Fajmanová (LinkedIn Profile) recently published a post arguing that model interpretability should be the topic of 2026 — not agents, not orchestration, not the next frontier model. Having spent months building agentic infrastructure in production, her argument landed differently than most thought leadership does. It landed because it’s right.
There is a growing gap between what we can build with agentic AI and what we can reliably control. Every practitioner building these systems knows the feeling: the proof of concept works brilliantly, the demo impresses stakeholders, and then you spend the next six months chasing edge cases that prompt engineering alone cannot solve. You optimize one metric and silently degrade another. You add a feedback loop that improves accuracy and doubles your token spend. You stack guardrails that work until they don’t.
Fajmanová frames this precisely. The bottleneck isn’t model capability — it’s our limited ability to control output reliability. And she identifies three specific failure modes that anyone building production agentic systems will recognize immediately: misalignment behavior, bias, and prompt-following reliability.
What makes her piece worth engaging with seriously is that she doesn’t stop at diagnosis. She points to concrete research demonstrating that internal model interventions — not more prompting, not more orchestration — can address each of these problems in targeted, measurable ways.
Three Problems, Three Research Directions
Fajmanová’s article maps three production reliability problems to three active research programs. Each represents a different mechanism for intervening at the model’s internal level rather than wrapping it in another layer of external logic.
Misalignment Behavior
Models inherit patterns from training data that produce outputs misaligned with intended behavior. This isn’t emergent superintelligence — it’s a direct consequence of how pattern learning works. And whoever trained your model may have introduced it, intentionally or not.
Internal Feature Steering
By identifying and steering specific internal features, particular misaligned behaviors can be suppressed in a targeted and measurable way — without retraining the entire model.
Wang et al. (2025) — Persona Features Control Emergent MisalignmentBias
Your system may speak differently to different users depending on language, promote specific products to certain demographics, or offer different advice — without you intending it. The magnitude of this impact is not under your control. And LLMs-as-judges are biased too.
Attention Head Pruning
Pruning attention heads that disproportionately contribute to biased behavior can reduce unfair outputs without retraining. A surgical intervention rather than a full rebuild.
Dasu et al. (2025) — Attention Pruning: Automated Fairness Repair via Surrogate Simulated AnnealingPrompt Reliability
Prompt tuning is a multidimensional optimization problem. Carefully optimizing one success metric can silently de-optimize others you haven’t measured yet. The result: fragile systems, endless iteration, and a reliability mechanism that is itself unreliable.
Attention Mechanism Intervention
By intervening at the level of attention mechanisms, instruction-following reliability can be improved in ways that prompt tuning alone fundamentally cannot achieve — inner-loop control, not outer-loop patching.
Venkateswaran & Contractor (2025) — Spotlight Your Instructions: Instruction-following with Dynamic Attention SteeringPost-Hoc vs. Intrinsic: The Deeper Question
Fajmanová’s referenced research represents one interpretability paradigm: post-hoc intervention. Take an existing model and modify specific internal mechanisms — steer features, prune attention heads, intervene at activation layers. This is practical, useful, and increasingly well-supported by research.
But there is a second paradigm emerging that goes further, and it challenges a foundational assumption: that interpretability has to be reverse-engineered after the fact.
Post-Hoc Intervention
Take a trained model. Identify problematic internal mechanisms. Steer, prune, or modify them to change behavior. Practical for teams using existing foundation models today.
Limitation: the model wasn’t built to be understood. You’re doing neuroscience on an opaque system — partial, and sometimes unreliable.
Repair After TrainingIntrinsic Interpretability
Rethink architecture, datasets, and training from the ground up so the model is interpretable by design. Every output is traceable. Every concept is controllable. No reverse-engineering required.
Julius Adebayo’s GuideLabs has demonstrated this is possible at scale — their Steerling-8B is an 8-billion parameter LLM where every token can be traced to training data origins.
Interpretable by DesignGuideLabs, founded by Julius Adebayo (LinkedIn Profile) — whose widely cited 2018 MIT research exposed the unreliability of existing model explanation methods — recently open-sourced Steerling-8B alongside several foundational components: Atlas, a system that annotates trillion-token datasets with human-interpretable concepts; a causal diffusion language model architecture; and PRISM, which traces outputs directly to training data patterns. Backed by Y Combinator and a $9.3M seed round led by Initialized Capital, Adebayo’s team has spent over 20 years collectively focused on interpretability and reliability research.
The kind of interpretability people do is neuroscience on a model, and we flip that. What we do is actually engineer the model from the ground up so that you don’t need to do neuroscience.
— Julius Adebayo, Founder & CEO of GuideLabs (TechCrunch, Feb 2026)Adebayo’s research team has published work demonstrating that post-hoc explanations — including chain-of-thought self-explanations — are often unreliable and “mostly unrelated to the model’s decision-making process.” This directly reinforces Fajmanová’s core argument that prompt-level and orchestration-level controls are necessary but insufficient.
What This Means for Production Agentic Systems
For those of us building agent infrastructure today, both paradigms matter — but for different reasons and on different timelines.
Near-Term: Post-Hoc Intervention
The research Fajmanová cites is actionable now. If you’re running production agents on existing foundation models, techniques like feature steering, attention pruning, and dynamic attention mechanisms represent concrete paths to improving reliability without waiting for next-generation architectures. These are inner-loop improvements that reduce your dependence on prompt engineering as the primary reliability mechanism.
Medium-Term: Intrinsic Interpretability
GuideLabs’ work signals where this is heading. As interpretable-by-design models mature and scale, the entire reliability equation changes. Bias becomes auditable at the concept level. Misalignment becomes traceable to training data. Prompt reliability becomes less of a bottleneck because the model’s internal logic is transparent. This is interpretability as infrastructure — not as a debugging afterthought.
The gap between what we can build with agentic AI and what we can reliably control is widening. Fajmanová is right — interpretability is the topic of the year. Not because it sounds important, but because it addresses the exact problems we’re hitting in production every day. The research is catching up. The infrastructure is emerging. Pay attention.
