Genetic algorithms: an

introduction
Artem Eremin,
j. researcher, IMMI KSU

Kuban State University

Krasnodar, Russia

Motivation

Genetic Algorithms: an introduction

Motivation

Genetic Algorithms: an introduction

Motivation
experimental data

Material properties (Cij) - ???

Doppler laservibrometry for measuring
out-of-plane velocities

Wavelet transform
TOF

Time-of-Flight (TOF) with wavelet

transform

d
c
TOF

Np

min F ?
Cij

F : (cg , j cg , j )2
j 1

Genetic Algorithms: an introduction

Optimization
Optimization is the process of making something better

Every day we subconsciously

solve some optimization problems!

Genetic Algorithms: an introduction

Optimization

Genetic Algorithms: an introduction

Minimum-seeking algorithms
1. Exhaustive Search = Brute Force

2. Analytical Optimization

2 f
f (x) 0,
?
x

3. Nelder-Mead downhill Simplex Method

4. Optimization based on Line Minimization (the

coordinate search method, the steepest
descent algorithm, Newtons method, DavidonFletcher-Powell (DFP) algorithm, etc )

Genetic Algorithms: an introduction

Minimum-seeking algorithms

1 4 can converge to a local minimum!

Natural optimization methods

Not the panacea, but
Simulated annealing
(Kirkpatrick et al., 1983)

Particle swarm optimization

(Parsopoulos and Vrahatis, 2002)

Genetic algorithms
(Holland, 1975)

Evolutionary algorithms
(Schwefel, 1995)

No derivatives, large search spaces, nature-based

Genetic Algorithms: an introduction

Biological background (Cell and Chromosomes)

Every animal cell is a complex of many small factories working
together; the center of this all is the cell nucleus; the nucleus contains
the genetic information in chromosomes - strings of DNA
Each chromosome contains a set of genes - blocks of DNA
Each gene determines some aspect of the organism (like eye colour)
A collection of genes is sometimes called a genotype
A collection of aspects (like eye colour) is sometimes called a
phenotype

Genetic Algorithms: an introduction

Biological background (Reproduction)

Organisms produce a number of offspring similar to themselves
but can have variations due to:
Mutations (random changes)

Sexual reproduction (offspring have combinations

of features inherited from each parent)

+
Genetic Algorithms: an introduction

Biological background (Natural Selection)

The Origin of Species: Preservation of favourable variations and

rejection of unfavourable variations.

There are more individuals born than can survive, so there is a

continuous struggle for life.

Individuals with an advantage have a greater chance for survive:

survival of the fittest.

Important aspects in natural selection are: adaptation to the

environment and isolation of populations in different groups which
cannot mutually mate

Genetic Algorithms: an introduction

Genetic algorithms (GA)

GA were initially developed by John Holland,
University of Michigan (1970s)

Popularized by his student David Goldberg

(solved some very complex engineering
problems, 1989)

Based on ideas from Darwinian Evolution

Provide efficient techniques for optimization and

machine learning applications; widely used in
business, science and engineering
Genetic Algorithms: an introduction

GA main features
Optimizes with continuous or discrete variables
Doesnt require derivative information
Simultaneously searches from a wide sampling of the cost surface

Deals with a large number of variables

Is well suited for parallel computers
Optimizes variables with extremely complex cost surfaces (they can
jump out of a local minimum)
Provides a list of optimum variables, not just a single solution
May encode the variables so that the optimization is done with the
encoded variables

Works with numerically generated data, experimental data, or analytical

functions.

Genetic Algorithms: an introduction

To start with
f ( x, y ) y sin 4 x 1.1x sin 2 y
0 x, y 10

Phenotype space

Genotype space =
{0,1}L

Encoding
(representation)

10

10010001
10010010

010001001
011101001

x 10

Decoding
(inverse representation)

Genetic Algorithms: an introduction

To start with
Gene a single encoding of part of the solution space, i.e. either
single bits or short blocks of adjacent bits that encode an element of
the candidate solution

1 0 0 1

4.1

x
Chromosome a string of genes that represents a solution

0 1 1 0

1 0 0 1

Population the number of chromosomes available to test

Genetic Algorithms: an introduction

Chromosomes
Chromosomes can be:
Bit strings (0110, 0011, 1101, )
Real numbers (33.2, -12.11, 5.32, )
Permutations of elements (1234, 3241, 4312, )
Lists of rules (R1, R2, R3, Rn)
Program elements(genetic programming)

Chromosome = array of Nvar variables (genes) pi

chromosome [ p1 , p2 , ... , pN var ]

cost f (chromosome)
N pop chromosomes
Genetic Algorithms: an introduction

How does it works?

So
produce an initial population of individuals
evaluate the fitness of all individuals
while termination condition not met do
select fitter individuals for reproduction
recombine between individuals
mutate individuals
evaluate the fitness of the modified individuals

generate a new population

End while

Genetic Algorithms: an introduction

How does it works?

The Evolutionary Cycle

selection

parents

modification
modified
offspring

initiate &
evaluate

population

evaluation
evaluated offspring
deleted
members

Or so

discard

Genetic Algorithms: an introduction

Generation of the initial population

pi [ai , bi ]

pi ai random[0,1] (bi ai )
i 1, Nvar
Coding: 4.25 01101101...

Ex.
Npop=6

s1 = 1111010101
s2 = 0111000101
s3 = 1110110101
s4 = 0100010011
s5 = 1110111101
s6 = 0100110000
Genetic Algorithms: an introduction

f (s1) = 7
f (s2) = 5
f (s3) = 7
f (s4) = 4
f (s5) = 8
f (s6) = 3

Selection

N keep X rate N pop

We are kind!
Lets save everybody!

X rate [0.4, 0.6]

or
i, fi ftr save!

Mating pool
Genetic Algorithms: an introduction

Selection
N keep , N pop N keep
1. Pairing from top to bottom
2. Random pairing

1 N keep
2 N keep 1
...

sc ,1 random[1, N keep ]
sc ,2 random[1, N keep ]

3. Weighted random pairing

roulette wheel weighting

Genetic Algorithms: an introduction

Selection
The roulette wheel method:
Individual i will have a probability

f (i )
f (i ) to be chosen
i

1
n

Area is
proportional to
fitness value

We repeat the extraction as many

times as it is necessary

3
4

Genetic Algorithms: an introduction

Selection
4. Tournament selection
1. Zenit
2. CSKA

16. Krylia Sovietov

50
48
14

a) randomly pick a small subset

b) perform a tournament
c) the winner takes it all
Tournament + Threshold = No SORTING!!!
Genetic Algorithms: an introduction

Mating (Crossover)
Simple 1-point crossover
Choose a random
point on
the two parents
Performance
with 1 Point
Crossover
depends
Split parents at this crossover point
on the order that variables occur in the
Create children by exchanging tails
representation
P typically in range (0.6, 0.9)
c

more likely to keep together genes that are near

each other
Can never keep together genes from opposite ends
of string
This is known as Positional Bias
Can be exploited if we know about the structure of
our problem, but this is not usually the case

Genetic Algorithms: an introduction

Mating (Crossover)
n-point crossover

Choose n random crossover points

Split along those points
Glue parts, alternating between parents
Generalisation of 1 point (still some positional bias)

Genetic Algorithms: an introduction

Mating (Crossover)
Uniform crossover
Uniform crossover looks at each bit in the parents and randomly
assigns the bit from one parent to one offspring and the bit from
the other parent to the other offspring

Genetic Algorithms: an introduction

Mutation
Alter each gene (or, bit) independently with a probability pm

pm is called the mutation rate

Typically between 1/Npop and 1/[s]

Genetic Algorithms: an introduction

Crossover or/and Mutation

A long debate: which one is better / necessary / mainbackground

Answer (at least, rather wide agreement):
it depends on the problem, but
in general, it is good to have both
both have another role
mutation-only-GA is possible, crossover-only-GA would not
work

Genetic Algorithms: an introduction

Crossover or/and Mutation

Exploration: Discovering promising areas in the search space,
i.e. gaining information on the problem
Exploitation: Optimising within a promising area, i.e. using
information
There is co-operation AND competition between them
Crossover is explorative, it makes a big jump to an area
somewhere in between two (parent) areas
Mutation is exploitative, it creates random small diversions,
thereby staying near (in the area of ) the parent
Only crossover can combine information from two parents

Only mutation can introduce new information

To hit the optimum you often need a lucky mutation

Genetic Algorithms: an introduction

Real valued problems

Mapping real values on bit strings
pi [ai, bi] R represented by {a1,,aL} {0,1}L
[ai, bi] {0,1}L must be invertible (one phenotype per
genotype)

: {0,1}L [ai, bi] defines the representation

bi ai L 1
(a1 ,..., aL ) ai L
( aL j 2 j ) [ ai , bi ]
2 1 j 0
Only 2L values out of infinite are represented
L determines possible maximum precision of solution
High precision long chromosomes (slow evolution)

Genetic Algorithms: an introduction

Floating point mutations

General scheme of floating point mutations

s ( p1, p2 ,..., pN pop ) s ' ( p '1, p '2 ,..., p 'N pop )

pi , p 'i ai , bi

Uniform mutation:

p 'i drawn randomly (uniform) from ai , bi

Analogous to bit-flipping (binary) or random resetting
(integers)

Genetic Algorithms: an introduction

Floating point mutations

Non-uniform mutations:
Many methods proposed,such as time-varying range of
change etc.
Most schemes are probabilistic but usually only make a
small change to value
Most common method is to add random deviate to each
variable separately, taken from N(0, ) Gaussian
distribution and then curtail to range
Standard deviation controls amount of change (2/3 of
deviations will lie in range (- to + )

Genetic Algorithms: an introduction

Crossover for real valued GAs

Discrete:
each gene value in offspring z comes from one of its
parents (x,y) with equal probability: zi = xi or yi
Could use n-point or uniform
Intermediate
exploits idea of creating children between parents
(hence a.k.a. arithmetic recombination)
zi = xi + (1 - ) yi where : 0 1.
The parameter can be:
constant: uniform arithmetical crossover
variable (e.g. depend on the age of the
population)
picked at random every time

Genetic Algorithms: an introduction

Single arithmetic crossover

Parents: x1,,xn and y1,,yn

Pick a single gene (k) at random,
child1 is:

reverse for other child. e.g. with = 0.5

x1 , ..., xk , yk (1 ) xk , ..., xn

Genetic Algorithms: an introduction

Simple arithmetic crossover

Parents: x1,,xn and y1,,yn

Pick random gene (k) after this point mix values
child1 is:

x , ..., x , y
(1 ) x
, ..., y (1 ) x
1
k
k 1
k 1
n
n

reverse for other child. e.g. with = 0.5

Genetic Algorithms: an introduction

Whole arithmetic crossover

Most commonly used

Parents: x1,,xn and y1,,yn
child1 is:

a x (1 a) y

reverse for other child. e.g. with = 0.5

Genetic Algorithms: an introduction

micro-GA
First generation (random values)

Tournament selection
SBX crossover
Select fittest
individual
Start new
generation

Good results?
Enough iterations?

No
Genetic Algorithms: an introduction

Yes

result

Benefits of GA
Concept is easy to understand
Modularseparate from application (representation);
building blocks can be used in hybrid applications
Supports multi-objective optimization
Good for noisyenvironment

Always results in an answer, which becomes better and

better with time
Can easily run in parallel
The fitness function can be changed from iteration to
iteration, which allows incorporating new data in the model
if it becomes available

Genetic Algorithms: an introduction

Issues with GA
Choosing parameters:
Population size
Crossover and mutation probabilities
Selection, deletion policies
Crossover, mutation operators, etc.
Termination criteria
Performance:
Can be too slow but covers a large search space
Is only as good as the fitness function

Genetic Algorithms: an introduction

Examples

Experimental specimens
4 CFRPplates
H 2, 25mm
[0o , 0o , 0o , 0o ]

H 2,35mm
[0o ,90o ,90o , 0o ]

V f 60%

V f 58%
Genetic Algorithms: an introduction

Material properties
Ex 107 2% GPa;

E y Ez 8.9 2% GPa

G yz 2.82 2% GPa; Gxz G yz 4.38 1% GPa

xz xy 0.25 0.32; yz 0.49 0.56

110.5

7.0
7.0
C
0
0

7.0

7.0

13.8

8.2

8.2
0

13.8
0

0
2.8

0
0

4.37

Genetic Algorithms: an introduction

0
0
GPa
0
0

4.37
0

Comparison of results

a0 mode
90o

Genetic Algorithms: an introduction

Comparison of results

s0 mode
90o

Genetic Algorithms: an introduction

Comparison of results

a0 mode

s0 mode

Genetic Algorithms: an introduction

GA for Permutations

Ordering/sequencing problems form a special type

Task is (or can be solved by) arranging some objects in a
certain order
Example: sort algorithm: important thing is which
elements occur before others (order)
Example: Travelling Salesman Problem (TSP) :
important thing is which elements occur next to each
other (adjacency)
These problems are generally expressed as a
permutation:
if there are n variables then the representation is as a
list of n integers, each of which occurs exactly once

Genetic Algorithms: an introduction

The Traveling Salesman Problem (TSP)

The traveling salesman must visit
every city in his territory exactly once
and then return to the starting point;
given the cost of travel between all
cities, how should he plan his itinerary
for minimum total cost of the entire
tour?
TSP NP-Complete

Search space is BIG:

for 30 cities there are
30! 1032 possible tours
Genetic Algorithms: an introduction

TSP (Representation, Initialization and Selection)

n cities
A vector v = (i1 i2 in) represents a tour (v is a
permutation of {1,2,,n})
Fitness f of a solution is the inverse cost of the
corresponding tour
Initialization: use either some heuristics, or a
random sample of permutations of {1,2,,n}
We shall use the fitness proportionate selection
Genetic Algorithms: an introduction

Mutation operations for permutations

Normal mutation operators lead to inadmissible
solutions
e.g. bit-wise mutation : let gene i have value j
changing to some other value k would mean
that k occurred twice and j no longer occurred
Therefore must change at least two values
Mutation parameter now reflects the probability
that some operator is applied once to the whole
string, rather than individually in each position

Genetic Algorithms: an introduction

Insert Mutation for permutations

Pick two allele values at random
Move the second to follow the first, shifting
the rest along to accommodate
Note that this preserves most of the order
and the adjacency information

Genetic Algorithms: an introduction

Swap mutation for permutations

Pick two alleles at random and swap their
positions
Preserves most of adjacency information
(4 links broken), disrupts order more

Genetic Algorithms: an introduction

Inversion mutation for permutations

Pick two alleles at random and then invert
the substring between them.
Preserves most adjacency information (only
breaks two links) but disruptive of order
information

Genetic Algorithms: an introduction

Scramble mutation for permutations

Pick a subset of genes at random
Randomly rearrange the genes in those
positions

(note subset does not have to be contiguous)

Genetic Algorithms: an introduction

Crossover for TSP (ex.)

Crossover builds offspring by choosing a subsequence of a tour from one parent and preserving
the relative order of cities from the other parent and
feasibility
Example:

p1 = (1 2 3 4 5 6 7 8 9) and
p2 = (4 5 2 1 8 7 6 9 3)
First, the segments between cut points are copied into offspring
o1 = (x x x 4 5 6 7 x x) and
o2 = (x x x 1 8 7 6 x x)
Genetic Algorithms: an introduction

Crossover for TSP (ex.)

Next, starting from the second cut point of one parent, the
cities from the other parent are copied in the same order

The sequence of the cities in the second parent is

934521876
After removal of cities from the first offspring we get
93218
This sequence is placed in the first offspring
o1 = (2 1 8 4 5 6 7 9 3), and similarly in the second

o2 = (3 4 5 1 8 7 6 9 2)

Genetic Algorithms: an introduction

Crossover for TSP (ex.)

Partially Mapped Crossover

Cycle crossover
Edge Recombination

Genetic Algorithms: an introduction

Thank you