The Future of Empirical Research in the Age of AI

In this episode, we sit down with Stanford political scientist Andy Hall and PhD candidate Graham Straus to unpack their new paper, “How Accurately Did Claude Code Replicate and Extend a Published Political Science Paper?” — an empirical audit of what happens when an AI agent is asked to replicate and extend a real research project. Last January, Andy asked Claude Code to generate an extension of an existing empirical political science paper in under an hour. The results were surprising: Claude correctly replicated the original estimates exactly and collected new data with very high accuracy. But did Claude make mistakes? Straus independently audited Claude’s work to see how accurate, reliable, and scientifically sound it really was.