Evolutionary Computation: Genetic Algorithms, CMA-ES & Neuroevolution
Evolutionary Computation harnesses the principles of Darwinian natural selection—reproduction, mutation, recombination, and survival of the fittest—to solve high-dimensional, non-convex, non-differentiable optimization problems where classical gradient descent fails completely.
1. The Canonical Genetic Algorithm (GA) Mechanics
Pioneered by John Holland (1975), a Genetic Algorithm encodes candidate solutions into "chromosomes" and iterates across generations to traverse complex fitness landscapes:
Population P(0)
Generates N random candidate chromosomes across the solution space.
Fitness f(x)
Computes the objective fitness score quantifying reproductive viability.
Survival Bias
Tournament or Roulette-wheel selection assigns higher mating probability to fit individuals.
Crossover & Mutation
Recombines parental schemas and injects random perturbations to escape local minima.
2. Evolutionary Algorithm Families
| Paradigm | Genotype Representation | Primary Operators | Ideal Application Domain |
|---|---|---|---|
| Genetic Algorithms (GA) | Binary bitstrings or real-valued vectors | Crossover (70-90%), Mutation (1-5%) | Discrete combinatorial optimization (Knapsack, Travelling Salesperson) |
| Evolution Strategies (ES / CMA-ES) | Real-valued parameter vectors μ & covariance Σ | Gaussian mutations, step-size adaptation (σ) | Continuous black-box optimization, deep reinforcement learning policy search |
| Genetic Programming (GP) | LISP-like syntax trees of code expressions | Subtree crossover, node mutation | Automated program synthesis, symbolic mathematical regression |
| Neuroevolution (NEAT) | Directed graphs of neural nodes and connections | Historical innovation tracking, structural speciation | Evolving neural topologies and weights without backpropagation gradients |
