Reasoning effort, not tool access, buys first-try reliability in agentic code generation: an observational study

This observational study challenges the assumption that giving coding AI agents more tools (like browser testing) automatically improves output quality. Across ninety independent runs building the same application, the dominant factor in performance was model capability tier and reasoning effort—not extra tools, which raised costs without improving reliability. Notably, increasing reasoning effort from "High" to "xHigh" boosted first-try perfect runs from 28% to 89%. Container deployment emerged as the most common failure point. This has practical implications for teams building AI coding agents: investing in reasoning depth is more cost-effective than adding tool complexity for reliability. Authors: Achint Mehta Paper: https://arxiv.org/abs/2607.02436v1