Game Theory · GT-38
A strategy that, once it dominates a population, can't be successfully invaded by any rare alternative — natural selection's version of equilibrium.
A strategy is evolutionarily stable if, when almost the entire population adopts it, no rare mutant strategy can invade and achieve a higher payoff (fitness) — the strategy is stable against invasion, without requiring conscious rationality from any individual, since the mechanism is differential reproductive success, not deliberate reasoning.
Introduced by biologist John Maynard Smith and George Price in a landmark 1973 Nature paper, applying Nash equilibrium concepts to biological evolution without assuming any player is consciously rational.
The Mechanism
Hawks and Doves — the population converges on a stable mix, not a single pure strategy
Neither pure extreme is stable — an all-Dove population is vulnerable to invasion by a rare aggressive Hawk mutant (who wins every contest against passive Doves), while an all-Hawk population is vulnerable to invasion by a rare Dove (who avoids the costly fights Hawks inflict on each other). The population settles at a specific stable mixture where neither strategy has an advantage over the other.
01 · IT REQUIRES NO CONSCIOUS RATIONALITY AT ALL
The key departure from classical game theory
Unlike Nash equilibrium, which typically assumes rational, forward-thinking players, an evolutionarily stable strategy can emerge purely through differential reproduction — individuals playing a higher-fitness strategy simply out-reproduce those playing a lower-fitness one over generations, with no individual needing to understand or reason about the game at all. This is what made Maynard Smith and Price's 1973 insight so powerful: game-theoretic equilibrium concepts apply even to non-rational biological populations.
02 · EVERY ESS IS A NASH EQUILIBRIUM, BUT NOT EVERY NASH EQUILIBRIUM IS AN ESS
A stronger stability requirement
An ESS must additionally be robust against invasion by rare alternative strategies, a stability condition stricter than the basic Nash equilibrium requirement (which only requires no player wants to unilaterally deviate) — this extra robustness requirement is what makes ESS particularly useful for predicting which strategies persist over long evolutionary or cultural timescales, not just which are momentarily stable.
03 · IT EXTENDS WELL BEYOND BIOLOGY INTO SOCIAL AND ECONOMIC EVOLUTION
Cultural and institutional 'evolution' follows similar logic
Beyond literal biological evolution, the ESS concept has been productively applied to the evolution of social norms, business strategies, and cultural conventions that spread and persist through differential success and imitation rather than conscious optimization — a business practice or social norm can become 'evolutionarily stable' in a market or culture through the same logic, even without any individual actor consciously solving a game.
Where It Fails / Inversion
Where it fails / inversion
ESS analysis assumes a large, well-mixed population where a rare mutant strategy interacts with the existing population at random — in smaller, more structured populations (where individuals interact predominantly with genetically or socially similar neighbors, as in kin-selection or spatially structured settings), different strategies can persist stably that pure large-population ESS analysis would predict should be invaded, since the interaction structure itself changes which strategies actually get tested against which.
How To Use It
Worked example · aggressive vs. cooperative business cultures in a market
In a competitive market, an entirely 'cooperative, never-undercut' industry norm (analogous to all-Dove) can be invaded and outcompeted by a single firm willing to aggressively price-cut and grab share (analogous to a Hawk mutant) — but an entirely cutthroat, always-undercut market (all-Hawk) leaves room for a firm that avoids costly price wars and instead competes on service or differentiation to profit more than firms locked in constant price battles (analogous to a Dove invading an all-Hawk population). Real markets often settle into a stable mix of aggressive and accommodating competitive postures for exactly this reason — neither pure extreme persists.
How to use it
When trying to understand why a market, culture, or social norm has settled into a particular stable mix of strategies rather than one single 'best' approach, don't assume the mix is an accident or a failure to converge — check whether either pure extreme would actually be vulnerable to invasion by the opposite strategy. If so, the observed mix may itself be the stable equilibrium, and pushing hard toward either pure extreme could make you vulnerable rather than dominant.
See Also