At a high level
Sparsity makes almost everything you bump into
rare
When the space between meaningful signals, rewards, or connections is mostly empty, the few hits that
do
Below are five complementary mechanisms—spanning information theory, brain/cognition, social networks, reinforcement-learning, and information-seeking systems—that show why this happens.
Self-information (surprisal) is I(x)=-log₂ p(x). A ten-times rarer event carries ten-times more bits of surprise—so in an extremely sparse distribution every encounter packs a high informational punch and is therefore more likely to trigger the “aha!” feeling we label serendipity.
Neurons (or model parameters) that fire only for a few stimuli minimize overlap between representations. Because concepts are stored far apart in representational space, any accidental co-activation bridges distant ideas, producing the unexpected juxtapositions that underlie insight and creativity. Sparse coding theories in neuroscience emphasize exactly this advantage for associative recall.
In a dense graph you mostly hear what your close contacts already know; but with few links, the bridges between clusters (weak ties) carry disproportionately fresh information. Mark Granovetter’s classic labor-market study showed that weak, infrequent contacts—i.e., structurally sparse links—were far more likely to deliver new job leads than strong ones, a canonical example of serendipitous benefit.
Reinforcement-learning agents (human or artificial) confronted with infrequent rewards must wander broadly or invent intrinsic-motivation “curiosity” signals. When a distant state finally yields a payoff, it is highly salient and often opens an unforeseen solution path—a serendipitous discovery produced because rewards were sparse, not despite it.
Librarians once suggested shelving books semi-randomly so browsers would stumble onto unrelated volumes; modern recommender systems inject low-probability items into feeds for exactly the same reason. Information-behaviour researchers frame serendipity as “unexpected yet valuable encounters,” and note that exploratory search contexts with little topical density are most likely to create them.