Speciation in Progress: Barriers, Hybrids, and the Fluid Boundaries Between Species
- Ash
- May 30
- 9 min read
Updated: Jul 24
Table of Contents
Introduction – What Counts as a Species?
Sidebar: Reproductive Isolation – Prezygotic vs Postzygotic
How Speciation Begins: Four Evolutionary Modes
Allopatric Speciation – When Geography Divides
Sympatric Speciation – New Species Side by Side
Parapatric Speciation – Divergence Across a Gradient
Sidebar: Ring Species – When the Modes Overlap
Peripatric Speciation – Small Populations, Big Changes
The Dynamics of Hybrid Zones
Where Species Meet and Mix
Narrow Tension Zones
Broad, Stable Hybrid Zones and Introgression
Sidebar: Moving Hybrid Zones in a Changing World
Hidden Genetic Conflicts: Dobzhansky–Muller Incompatibilities
The Genetics of Hybrid Dysfunction
D-M Incompatibilities in Action
Sidebar: Mitonuclear Incompatibilities
When Speciation Reverses
Barriers Broken: Hybrid Swarms and Genetic Extinction
Sticklebacks in British Columbia
Cichlids of Lake Victoria
Hybridisation and Conservation Threats
Sidebar: Eutrophication and Barrier Breakdown
Watching Speciation from the Middle
Field Signs and the Role of the Observer
Checklist: Field Signs of an Active Hybrid Zone
Conclusion – Species as Shifting Patterns
Glossary
References
Introduction – What Counts as a Species?
The natural world is not as sharply divided as the tidy boxes of a biology textbook might suggest. As any observant naturalist or hobbyist quickly learns, the boundaries between species are often blurred, dynamic, and surprisingly complex. What, then, truly defines a species? There is no single answer. Scientists have developed several definitions, each suited to different contexts. The biological species concept focuses on the ability to interbreed and produce fertile offspring; the morphological species concept relies on distinct physical traits; and the phylogenetic species concept considers evolutionary history and genetic divergence.
Yet, in nature, these lines are not always clear-cut. New species are constantly forming through a process called speciation, while others blend back together when circumstances change. Rather than seeing species as fixed points, modern biology views them as products of ongoing evolutionary processes, shaped by barriers, hybridisation, and genetic drift. This article explores how speciation occurs, how hybrid zones expose hidden boundaries, and why those boundaries sometimes dissolve.
Sidebar: Reproductive Isolation – Prezygotic vs Postzygotic
Speciation depends on the evolution of barriers to gene flow. These can be prezygotic (preventing mating or fertilisation, such as different mating seasons, habitats, or behaviours) or postzygotic (acting after fertilisation, such as hybrid sterility or inviability). Understanding these categories sets the stage for exploring the mechanisms and outcomes of speciation.
How Speciation Begins: Four Evolutionary Modes
The Diversity of Speciation Mechanisms
Biologists recognise four principal modes by which new species can arise, each involving a different arrangement of populations and barriers to gene flow. Real-world examples show that these modes are not theoretical curiosities, but living processes observable in many animal and plant lineages.
Allopatric Speciation – When Geography Divides
The classic and perhaps most intuitive form of speciation is allopatric speciation, which occurs when populations are physically separated by a barrier such as a mountain, river, or, in one dramatic case, the Grand Canyon. Consider the Kaibab and Abert’s squirrels of the American Southwest. About 10,000 years ago, a single squirrel population was split by the formation of the canyon. Over time, the two groups diverged: the Kaibab squirrel developed unique traits such as a black belly and white tail, while the Abert’s squirrel retained different features. With no gene flow between them, isolation allowed independent evolutionary trajectories, leading to distinct species.
Sympatric Speciation – New Species Side by Side
In sympatric speciation, new species emerge within the same geographic area, without any physical barrier. The key is usually a shift in ecological preferences or mating behaviours. One well-studied example is the apple maggot fly (Rhagoletis pomonella). Originally, these flies laid eggs only in hawthorn fruit. But in the 1800s, some shifted to cultivated apples, which ripen earlier than hawthorn. Flies began mating and emerging in synchrony with their chosen fruit, and genetic differences accumulated between the “hawthorn race” and “apple race,” despite their overlapping ranges. Today, the two groups are partially reproductively isolated, demonstrating incipient sympatric speciation.
Parapatric Speciation – Divergence Across a Gradient
Parapatric speciation arises when populations occupy neighbouring but distinct environments, with limited gene flow between them. In the Welsh countryside, populations of sweet vernal grass (Anthoxanthum odoratum) growing on metal-contaminated mine tailings have evolved to tolerate toxic soils. These mine-adapted grasses flower earlier than their counterparts in uncontaminated meadows just metres away, sharply reducing cross-pollination. Strong local selection maintains this divide, and the result is a narrow but stable genetic differentiation, a real-world example of parapatric speciation driven by environmental gradients.
Sidebar: Ring Species – When the Modes Overlap
Some of the most fascinating evolutionary puzzles are ring species: populations that form a geographic ring around a barrier, with neighbouring groups interbreeding, but the terminal ends unable to do so. The greenish warbler (Phylloscopus trochiloides) and the Ensatina salamanders of North America are classic cases, illustrating the continuum between allopatric and parapatric speciation.
Peripatric Speciation – Small Populations, Big Changes
In peripatric speciation, a small group becomes isolated at the edge of a species’ range, often in a novel habitat. The London Underground mosquito (Culex pipiens f. molestus) is a striking example. In the 19th century, a population of mosquitoes colonised the city’s underground tunnels, becoming separated from above-ground populations. Over roughly 150 years, these mosquitoes evolved to breed year-round and feed on mammals, developing reproductive isolation from their surface relatives. Such rapid speciation in a confined space highlights the power of founder effects and isolation.
The Dynamics of Hybrid Zones
Where Species Meet and Mix
Hybrid zones are regions where genetically distinct groups come into contact and interbreed, producing offspring of mixed ancestry. These zones are natural laboratories for studying how speciation is maintained or eroded. Two main types are recognised:
Narrow Tension Zones
When hybrids have lower fitness than either parent, selection maintains a narrow hybrid zone, a tension zone. This can persist for thousands of years, shifting little over time. The fire-bellied toads (Bombina bombina and B. variegata) of Central Europe meet along a strip just five to ten kilometres wide. Hybrids are less fit, and steep genetic clines are maintained by a balance of dispersal and selection against hybrids. The precise location of the zone coincides with an environmental transition from lowland to upland habitats, and its stability makes it a classic model for evolutionary study.
Broad, Stable Hybrid Zones and Introgression
In other cases, hybrids are not severely selected against, and gene flow between groups can be extensive. For example, the Alpine willows (Salix alpina and S. breviserrata) form a broad hybrid zone spanning more than 300 kilometres in the Alps. Here, hybrid swarms persist, and alleles flow freely between species, suggesting weak barriers to gene exchange. Such broad zones often correlate with environmental intermediacy or gradual transitions rather than sharp ecological boundaries.
Sidebar: Moving Hybrid Zones in a Changing World
Hybrid zones are not static. Many are now shifting due to climate change or human land use. Recent studies on birds and insects have shown hybrid zones moving northwards or to higher altitudes as temperature regimes change. Genomic tools, such as genome-wide cline analysis, now reveal that some genes introgress readily while others remain resistant, providing insights into the architecture of speciation.
Hidden Genetic Conflicts: Dobzhansky–Muller Incompatibilities
The Genetics of Hybrid Dysfunction
Even when populations come back into contact, genetic incompatibilities can impede gene flow. The Bateson–Dobzhansky–Muller (D-M) model describes how independent mutations in isolated populations may be benign on their own, but interact negatively when brought together in hybrids. This negative epistasis is a central mechanism of postzygotic isolation; hybrids may be sterile or inviable, not because of “bad genes”, but because novel combinations have never been tested by selection.
D-M Incompatibilities in Action
Invertebrates: In fruit flies (Drosophila melanogaster and D. simulans), hybrids are often inviable or sterile due to incompatible genes such as Hmr and Lhr. The parasitic wasp genus Nasonia also exhibits hybrid breakdown caused by multiple D-M incompatibilities. Similarly, the marine copepod Tigriopus californicus demonstrates reduced hybrid fitness due to mismatched mitochondrial and nuclear genes.
Vertebrates: House mice (Mus musculus musculus and M. m. domesticus) hybridise in nature, but male hybrids frequently have reduced fertility, often traced to incompatibilities between the Prdm9 gene and X-chromosomal loci. Recent research is mapping such DMIs in real time.
Plants: In Arabidopsis thaliana, hybrid necrosis results from incompatible immune gene alleles triggering autoimmune collapse in hybrids. In rice (Oryza), hybrid sterility is caused by incompatible gene pairs (for example, the S1 locus), which restricts gene flow between domesticated and wild species.
Sidebar: Mitonuclear Incompatibilities
Some of the most potent DMIs arise between mitochondrial and nuclear genomes (mitonuclear incompatibility), as seen in swordtail fish and copepods. These cross-genomic conflicts can evolve rapidly and reinforce reproductive isolation.
When Speciation Reverses
Barriers Broken: Hybrid Swarms and Genetic Extinction
Just as speciation can create new species, it can also reverse course. Environmental change, especially when driven by human activity, can erase the very barriers that once separated species.
Sticklebacks in British Columbia
In Enos Lake, Canada, threespine sticklebacks diverged into benthic and limnetic species, each adapted to different niches. But with the introduction of invasive crayfish and habitat disturbance, those ecological distinctions blurred. By 2000, the two forms had merged into a single hybrid swarm, demonstrating speciation in reverse. The outcome was the loss of two specialist species and their unique adaptations.
Cichlids of Lake Victoria
Lake Victoria’s diverse cichlid fishes evolved through rapid speciation, with mate recognition based on vivid colour patterns. However, increased eutrophication and resulting water turbidity have undermined visual cues. As a result, previously distinct species began hybridising, and some lineages have fused or disappeared. This phenomenon, linked to the loss of visual mating cues in the lake, is a textbook example of environmental change driving barrier collapse.
Hybridisation and Conservation Threats
Other instances abound. The golden-winged warbler in North America is disappearing as hybridisation with the blue-winged warbler spreads. In Scotland, the native wildcat is threatened by extensive hybridisation with domestic cats, creating a hybrid swarm and endangering the original wild genome. Such genetic extinction raises pressing conservation questions about how to manage hybridisation in a changing world.
Sidebar: Eutrophication and Barrier Breakdown
Freshwater habitats are especially prone to barrier collapse due to nutrient enrichment (eutrophication), which often precedes hybrid swarms in fishes such as cichlids and sticklebacks.
Watching Speciation from the Middle
Field Signs and the Role of the Observer
Speciation is not just the concern of scientists with lab coats and gene sequencers. Hobbyists and naturalists can witness these processes in their own back gardens, local reserves, or even terrariums. The key is knowing what to look for.
Checklist: Field Signs of an Active Hybrid Zone
Gradual changes in trait frequencies across a landscape (clines)
Presence of individuals with intermediate morphologies or behaviours
Variable fertility or viability among hybrids
Sharp habitat transitions with interbreeding at the boundary
Movement of hybrid zones over years or decades
Keeping notes on traits and behaviour in wild or captive populations can contribute valuable data to our understanding of evolutionary transitions. Even small-scale projects, such as observing variation in garden snails, local birds, or wildflowers, offer windows into speciation in action.
Conclusion – Species as Shifting Patterns
The story of speciation is not one of neat partitions, but of shifting patterns and ongoing change. Boundaries between species can be sharp, blurred, or moving, depending on evolutionary forces and environmental context. Hybrid zones, genetic incompatibilities, and the possibility of reversed speciation all remind us that nature resists simple categorisation.
For conservationists, the challenge is to decide when to preserve distinct lineages and when to accept introgression as part of the evolutionary process. For hobbyists and naturalists, it is an invitation to observe and appreciate the dynamic world of biodiversity. In the end, speciation is less about drawing lines and more about understanding the processes that shape life’s diversity, processes that are still unfolding around us.
Glossary
Introgression: The movement of genes from one species or population into another by repeated hybridisation and backcrossing.
Postzygotic isolation: Barriers to gene flow that act after fertilisation, often resulting in sterile or inviable offspring.
Tension zone: A narrow hybrid zone maintained by a balance between interbreeding and selection against hybrids.
Hybrid swarm: A population made up of many hybrids and backcrosses, blurring parental distinctions.
Reinforcement: Natural selection that strengthens reproductive isolation to prevent production of unfit hybrids.
Incomplete lineage sorting: The retention of ancestral genetic variation across multiple species, resulting in mismatched gene and species trees.
References
Kaibab Squirrel - Grand Canyon-Parashant National Monument (U.S. National Park Service)
Smith et al. (2019), Evolution Letters – Hybrid zone in Bombina toads
The London Underground Has Its Own Mosquito Subspecies, Smithsonian Magazine
Hybrid zones in the European Alps impact the phylogeography of alpine vicariant willow species (Salix L.), Frontiers in Plant Science
Bateson–Dobzhansky–Muller model, Wikipedia
Hybrid sterility genes in mice (Mus musculus): a peculiar case of PRDM9 incompatibility, PubMed
Autoimmune Response as a Mechanism for a Dobzhansky-Muller-Type Incompatibility Syndrome in Plants, PLOS Biology
Speciation in reverse: morphological and genetic evidence of the collapse of a three-spined stickleback (Gasterosteus aculeatus) species pair, PubMed
House Cats Are Overpowering Scottish Wildcats In The Battle For Survival, Forbes
Commentaires