How Randomness Rules our Lives

The deep-seated human discomfort with randomness often leads us to "legislate" against it, much like the NFL did after a 2022 playoff game between the Buffalo Bills and Kansas City Chiefs was decided by a coin toss.

This act of changing rules to reduce the power of chance reflects a fundamental need to believe that success is earned through skill rather than arbitrary forces.

However, research into human psychology reveals that we are "pattern matching machines" that often find order where none exists. In experiments where subjects predict biased light flashes, humans consistently underperform compared to rats because our complex brains insist on finding a hidden order in a random system, a liability that plagues us in arenas ranging from sports to high finance.

This "illusion of skill" is frequently reinforced by a phenomenon known as regression to the mean. In professional baseball, a "hot streak" or a "slump" is often just a temporary deviation from a player's long-term average, yet we mistake these random peaks for permanent plateaus of talent.

A similar illusion exists on Wall Street, where "star" fund managers are often the beneficiaries of survivorship bias—we see the winners but ignore the "massive graveyard" of failed managers.

As the SPIVA scorecards demonstrate, nearly 95% of active funds fail to beat the market over twenty years, suggesting that many "geniuses" are simply lucky coin-flippers in a massive tournament.

Ultimately, as psychologist Daniel Kahneman observed with flight instructors, we often attribute the natural ebbing and flowing of performance to our own rewards or punishments, failing to see the unseen random forces that pull every outlier back toward the center.