Nine: Neel Nanda on the race to read AI minds

Nobody knows how AIs think, or why they do what they do.

Or at least, we don’t know much. Not the companies building them, the researchers studying them, or the governments beginning to rely on them.

This is only becoming more troubling as AIs grow more capable and appear on track to wield enormous cultural influence, directly advise on major government decisions, and even operate military equipment autonomously. We simply can’t tell what models, if any, should be trusted with such authority.

Neel Nanda of Google DeepMind is one of the founders of mechanistic interpretability — the field of trying to give us insight into what’s happening inside AI models.

The project has generated enormous hype, exploding from a handful of researchers five years ago to hundreds today — all working to make sense of the jumble of tens of thousands of numbers that frontier AIs use to process information and decide what to say or do.

But Neel now has a warning for us: the most ambitious vision of mechanistic interpretability is probably dead. He doesn’t see a path to deeply and reliably understanding what AIs are thinking. The technical and practical barriers are too great to get us there before competitive pressures push us to deploy human-level or superhuman AIs. 

Neel argues no single approach will guarantee safe alignment, and our only choice is the 'Swiss cheese' model of protection: layering multiple safeguards on top of one another.

That doesn’t mean mechanistic interpretability has failed. It won’t be a silver bullet for AI safety, but it will be one of the best tools in our arsenal.

For example, by inspecting the neural activations in the middle of an AI’s thoughts, we can see many of the concepts the model is thinking about — from refusing to answer a question, to the option of deceiving the user. 

We can’t track every thought a model is having at every moment, but catching 90% of the concepts it uses 90% of the time should help us muddle through — as long as mechanistic interpretability is paired with other techniques to fill the gaps.

In this episode, Neel takes us on a tour of the race to understand what AIs are really thinking. He and host Rob Wiblin cover:

• The best tools we’ve come up with so far, and where mechanistic interpretability has failed
• Why the best techniques have to be fast and cheap
• The fundamental reasons we can’t reliably know what AIs are thinking, despite having perfect internal access to them
• What we can and can’t learn by reading models’ ‘chains of thought’
• Whether models will be able to trick us when they realise they’re being tested
• The best protections to add on top of mechanistic interpretability
• Why he thinks the hottest technique in the field (SAEs) is overrated
• How to break into mechanistic interpretability and get a job 

Learn more and read the full transcript on the 80,000 Hours website.

This episode was originally released in September 2025.

Chapters:

• Cold open (00:00:00)
• Who’s Neel Nanda? (00:01:04)
• How would mechanistic interpretability help with AGI (00:02:01)
• What's mech interp? (00:05:12)
• How Neel changed his take on mech interp (00:09:50)
• Top successes in interpretability (00:16:00)
• Probes can cheaply detect harmful intentions in AIs (00:20:13)
• In some ways we understand AIs better than human minds (00:26:58)
• Mech interp won't solve all our AI alignment problems (00:29:30)
• Why mech interp is the 'biology' of neural networks (00:38:17)
• Interpretability can't reliably find deceptive AI – nothing can (00:40:38)
• 'Black box' interpretability — reading the chain of thought (00:49:51)
• 'Self-preservation' isn't always what it seems (00:53:17)
• For how long can we trust the chain of thought (01:02:25)
• We could accidentally destroy chain of thought's usefulness (01:11:58)
• Models can tell when they’re being tested and act differently (01:17:14)
• Top complaints about mech interp (01:24:11)
• Why everyone's excited about sparse autoencoders (SAEs) (01:38:24)
• Limitations of SAEs (01:47:55)
• SAEs performance on real-world tasks (01:55:38)
• Best arguments in favour of mech interp (02:09:15)
• Lessons from the hype around mech interp (02:13:11)
• Where mech interp will shine in coming years (02:18:58)
• Why focus on understanding over control (02:22:12)
• If AI models are conscious, will mech interp help us figure it out (02:25:19)
• Neel’s new research philosophy (02:27:29)
• Who should join the mech interp field (02:39:42)
• Advice for getting started in mech interp (02:48:10)
• Keeping up to date with mech interp results (02:56:02)
• Who’s hiring and where to work? (02:59:06)

Video editing: Simon Monsour, Luke Monsour, Dominic Armstrong, and Milo McGuire
Audio engineering: Ben Cordell, Milo McGuire, Simon Monsour, and Dominic Armstrong
Music: Ben Cordell
Camera operator: Jeremy Chevillotte
Coordination, transcriptions, and web: Katy Moore