S6E10 - Why CRO “Best Practices” Keep Failing (And AI Might Make It Worse)

In this episode, we dive into CRO “best practices” that completely failed in real-world experimentation. From homepage carousels unexpectedly winning tests to why most experimenters still don’t trust AI-generated CRO workflows, we unpack the messy reality behind optimization advice.We also debate synthetic AI users for research, discuss the most useful survey questions in CRO, roast dark-mode websites, and explain why emotional branding can sometimes hurt usability.Join our slack: https://join.slack.com/t/experiment-nation/shared_invite/zt-3zowejvoj-ujS_tbqfXcPkS9O0rnWhTQTopics include:Why homepage carousels occasionally winAI-powered A/B testing platformsSynthetic users & fake research dataBest and worst CRO survey questionsDark mode website UXWhy qualitative context still mattersJoin the Experiment Nation Slack community to participate in future discussions and question threads.Chapters00:00 Intro & CRO community questions01:46 CRO best practices that failed in real life02:28 Do homepage carousels actually work?05:02 Hero videos, agencies & emotional design06:25 When carousels might make sense08:24 Should AI run the entire CRO workflow?11:48 Why experimenters still don’t trust AI14:49 Best & worst survey questions in CRO17:30 Emotional branding vs usable websites18:26 Dark mode websites & readability debate21:29 Synthetic AI users for testing & research23:56 Why fake users can’t mimic real humans26:32 Using AI as a research starting point28:41 Closing thoughts & Slack community invite

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit experimentnation.substack.com