Genetic Algorithms

An algorithm is a set of instructions that is

repeated to solve a problem.
A genetic algorithm conceptually follows steps
inspired by the biological processes of evolution.
Genetic Algorithms follow the idea of SURVIVAL
OF THE FITTEST- Better and better solutions
evolve from previous generations until a near
optimal solution is obtained.
Genetic Algorithms
Also known as evolutionary algorithms, genetic
algorithms demonstrate self organization and
adaptation similar to the way that the fittest
biological organism survive and reproduce.
A genetic algorithm is an iterative procedure that
represents its candidate solutions as strings of
genes called chromosomes.
Simulated Evolution
We need the following
Representation of an individual
Fitness Function
Reproduction Method
Selection Criteria
Representing an Individual
An individual is data structure representing the
“genetic structure” of a possible solution.
Genetic structure consists of an alphabet (usually 0,1)
Binary Encoding
Most Common – string of bits, 0 or 1.
Chrom: A = 1 0 11 0 0 1 0 1 1
Chrom: B = 1 1 1 1 1 1 0 0 0 0
Gives you many possibilities
Example Problem: Knapsack problem
The problem: there are things with given value and size.
The knapsack has given capacity. Select things to
maximize the values.
Encoding: Each bit says, if the corresponding thing is in
the knapsack
Permutation Encoding
Used in “ordering problems”
Every chromosome is a string of numbers, which
represents number is a sequence.
Chrom A: 1 5 3 2 6 4 7 9 8
Chrom B: 8 5 7 7 2 3 1 4 9
Example: Travelling salesman problem
The problem: cities that must be visited.
Encoding says order of cities in which salesman
willl visit.
Another Example
To find optimal quantity of three major ingredients
(sugar, wine, sesame oil) denoting ounces.
Use an alphabet of 1-9 denoting ounces..
Solutions might be 1-1-1, 2-1-4, 3-3-1.
Value Encoding
Used for complicated values (real numbers) and
when binary coding would be difficult
Each chromosome is a string of some values.
Chrom A: 1.2323 5.3243 0.4556
Chrom B: abcdjeifjdhdierjfd
Chrom C: (back), (back), (right), (forward), (left)
Example: Finding weights for neural nets.
The problem: find weights for network
Encoding: Real values that represent weights
Rule base system
Given a rule (if color=red and size=small and
shape=round then object=apple.
Assume that each feature has finite set of values
(e.g., size = small,large)
Represent the value as a substring of length equl
to the number of possible values. For example,
small = 10, large = 01.
The entire rule would be 100 10 01 0100 – set of
rules concatenating the values together.
How offspring are produced -
Reproduction
Reproduction- Through reproduction, genetic
algorithms produce new generations of improved
solutions by selecting parents with higher fitness
ratings
How offspring are produced
Crossover- Many genetic algorithms use strings of
binary symbols for chromosomes, as in our
Knapsack example, to represent solutions.
Crossover means choosing a random position in
the string (say, after 2 digits) and exchanging the
segments either to the right or to the left of this
point with another string partitioned similarly to
produce two new off spring.
Crossover Example
Parent A 011011
Parent B 101100
“by splitting each number as shown between the second
and third digits (position is randomly selected)
01*1011 10*1100
Crossover Example
Now combine the first digits of A with the last digits
of B, and the first digits of B with the last digits of A
This gives you two new offspring
011100
101011
If these new solutions, or offspring, are better
solutions than the parent solutions, the system will
keep these as more optimal solutions and they will
become parents. This is repeated until some condition
(for example number of populations or improvement
of the best solution) is satisfied.
How offspring are produced
Mutation- Mutation is an arbitrary change in a
situation. Sometimes it is used to prevent the
algorithm from getting stuck. The procedure changes
a 1 to a 0, or 0 to a 1. This change occurs with a very
low probability (say 1 in 1000)
Genetic Algorithm Operators
Mutation and Crossover

Parent 1 1 0 1 0 1 1 1

Parent 2 1 1 0 0 0 1 1

Child 1 1 0 1 0 0 1 1

Child 2 1 1 0 0 1 1 0 Mutation
Crossover Operators
 Single­point crossover:
 Parent A: 1 0 0 1 0| 1 1 1 0 1
 Parent B: 0 1 0 1 1 |1 0 1 1 0

 Child AB: 1 0 0 1 0 1 0 1 1 0
 Child BA: 0 1 0 1 1 1 1 1 0 1

 Two­point crossover:
 Parent A: 1 0 0 1 |0 1 1| 1 0 1
 Parent B: 0 1 0 1 |1 1 0 |1 1 0

 Child AB: 1 0 0 1 1 1 0 1 0 1
 Child BA: 0 1 0 1 0 1 1 1 1 0
Uniform Crossover and Mutation
Uniform crossover:
Parent A: 1 0 0 1 0 1 1 1 0 1
Parent B: 0 1 0 1 1 1 0 1 1 0

Child AB: 1 1 0 1 1 1 1 1 0 1
Child BA: 0 0 0 1 0 1 0 1 1 0

Mutation: randomly toggle one bit

Individual A: 1 0 0 1 0 1 1 1 0 1
Individual A': 1 0 0 0 0 1 1 1 0 1
Examples
Mutation:
The recipe example:
1-2-3 may be changed to 1-3-3 or 3-2-3, giving two new
offspring. How often? How many digits change? How
big? (parameters to adjust)
More examples:
Crossover
Recipe :
Parents 1-3-3 & 3-2-3. Crossover point after the first
digit. Generate two offspring: 3-3-3 and 1-2-3.
Can have one or two point crossover.
Crossover – Permutation Encoding
Single point crossover - one crossover point is selected, till this point
the permutation is copied from the first parent, then the second
parent is scanned and if the number is not yet in the offspring it is
added

(1 2 3 4 5 6 7 8 9) + (4 5 3 6 8 9 7 2 1) = (1 2 3 4 5 6 8 9 7)

Mutation
Order changing - two numbers are selected and exchanged
(1 2 3 4 5 6 8 9 7) => (1 8 3 4 5 6 2 9 7)
Crossover – Value Encoding
Crossover
All crossovers from binary encoding can be used

Mutation
Adding a small number (for real value encoding) - to
selected values is added (or subtracted) a small number
(1.29  5.68  2.86  4.11  5.55) => (1.29  5.68  2.73  4.22  5.55)
Selection Criteria
Fitness proportionate selection, rank selection
methods.
Fitness proportionate – each individual, I, has the
probability fitness(I)/sum_over_all_individual_j
Fitness(j), where Fitness(I) is the fitness function value
for individual I.
Rank selection – sorts individual by fitness and the
probability that an individual will be selected is
proportional to its rank in this sorted list.
Fitness Function
Represents a rank of the “representation”
It is usually a real number.
The function usually has a value between 0 and 1.
One can have a subjective judgment (e.g. 1-5 for
recipe 2-1-4.)
Similarly the length of the route in the traveling
salesperson problem is a good measure, because
the shorter the route, the better the solution.
Outline of the Basic Genetic Algorithm

1. [Start] Generate random population of n

chromosomes (suitable solutions for the
problem)
2. [Fitness] Evaluate the fitness f(x) of each
chromosome x in the population
3. [New population] Create a new population by
repeating following steps until the new
population is complete
Outline of the Basic Genetic Algorithm

4. [Selection] Select two parent chromosomes

from a population according to their fitness (the
better fitness, the bigger chance to be selected)
The idea is to choose the better parents.
5. [Crossover] With a crossover probability cross
over the parents to form a new offspring
(children). If no crossover was performed,
offspring is an exact copy of parents.
6. [Mutation] With a mutation probability mutate
new offspring at each locus (position in
chromosome).
Outline of the Basic Genetic Algorithm

7. [Accepting] Place new offspring in a new

population
8. [Replace] Use new generated population for a
further run of algorithm
9. [Test] If the end condition is satisfied, stop,
and return the best solution in current
population
10. [Loop] Go to step 2

 Flow Diagram of the Genetic
Algorithm Process
Describe
Problem

Generate
Initial
Solutions

Step 1 Test: is initial Yes Stop

solution good enough?

No

Step 2 Select parents

to reproduce

Step 3 Apply crossover process

and create a set of offspring
Step 4
Step 5 Apply random mutation
Example: The Knapsack Problem
You are going on an overnight hike and have a
number of items that you could take along. Each
item has a weight (in pounds) and a benefit or
value to you on the hike(for measurements sake
let’s say, in US dollars), and you can take one of
each item at most. There is a capacity limit on the
weight you can carry (constraint). This problem
only illustrates one constraint, but in reality there
could be many constraints including volume,
time, etc.
GA Example: The Knapsack Problem
Item: 1 2 3 4 5 6 7
Benefit: 5 8 3 2 7 9 4
Weight: 7 8 4 10 4 6 4
Knapsack holds a maximum of 22 pounds
Fill it to get the maximum benefit
Solutions take the form of a string of 1’s and 0’s
Solutions: Also known as strings of genes called
Chromosomes
1. 0101010
2. 1101100
3. 0100111
Example: The Knapsack Problem
We represent a solution as a string of seven 1s and
0s and the fitness function as the total benefit,
which is the sum of the gene values in a string
solution times their representative benefit
coefficient.
The method generates a set of random solutions
(initial parents), uses total benefit as the fitness
function and selects the parents randomly to
create generations of offspring by crossover and
mutation.
Knapsack Example
Typically, a string of 1s and 0s can represent a
solution.
Possible solutions generated by the system using
Reproduction, Crossover, or Mutations
1. 0101010
2. 1101100
3. 0100111
Knapsack Example
Solution 1
Item 1 2 3 4 5 6 7

Solution 0 1 0 1 0 1 0

Benefit 5 8 3 2 7 9 4

Weight 7 8 4 10 4 6 4

Benefit 8 + 2 + 9 = 19
Weight 8 + 10 + 6 = 24
Knapsack Example
Solution 2
Item 1 2 3 4 5 6 7
Solution 1 1 0 0 1 0 0

Benefit 5 8 3 2 7 9 4
Weight 7 8 4 10 4 6 4

Benefit 5 + 8 + 7 = 20
Weight 7 + 8 + 4 = 19
Knapsack Example
Solution 3
Item 1 2 3 4 5 6 7

Solution 0 1 0 0 1 1 1

Benefit 5 8 3 2 7 9 4

Weight 7 8 4 10 4 6 4

Benefit 8 + 7 + 9 + 4 = 28
Weight 8 + 4 + 6 + 4 = 22
Knapsack Example
Solution 3 is clearly the best solution and has met
our conditions, therefore, item number 2, 5, 6,
and 7 will be taken on the hiking trip. We will be
able to get the most benefit out of these items
while still having weight equal to 22 pounds.
This is a simple example illustrating a genetic
algorithm approach.
The Knapsack Problem
The knapsack
problem, though
simple, has many
important
applications
including
determining what
items to take on a
space ship
mission.
Genetic Algorithms
Genetic Algorithms are a type of machine
learning for representing and solving complex
problems.
They provide a set of efficient, domain-
independent search heuristics for a broad
spectrum of applications.
A genetic algorithm interprets information that
enables it to reject inferior solutions and
accumulate good ones, and thus it learns about its
universe.
Genetic Algorithm Application Areas
 Dynamic process control
 Induction of rule optimization
 Discovering new connectivity topologies
 Simulating biological models of behavior and evolution
 Complex design of engineering structures
 Pattern recognition
 Scheduling
 Transportation
 Layout and circuit design
 Telecommunication
 Graph-based problems
Business Applications
Schedule Assembly lines at Volvo Truck North America
Channel 4 Television (England) to schedule
commercials
Driver scheduling in a public transportation system
Jobshop scheduling
Assignment of destinations to sources
Trading stocks
Productivity in whisky-making is increased
Often genetic algorithm hybrids with other AI methods
A good site example
http://cs.felk.cvut.cz/~xobitko/ga/tspexample.html