Fundamental Region Elimination Rule

The fundamental rule for region-elimination methods is as follows: Let

us consider two points x1 and x2 which lie in the interval (a, b) and satisfy
x1 < x2 . For minimization of unimodal functions, we can conclude the
following:
I If g(x1 ) > g(x2 ) then the minimum does not lie in (a, x1 ).
I If g(x1 ) < g(x2 ) then the minimum does not lie in (x2 , b).
I If g(x1 ) = g(x2 ) then the minimum does not lie in (a, x1 ) and (x2 , b).
Exercise:
1. Consider two points x1 = 0.665 and x2 = 0.6523 which lies in the
interval [0.6392, 0.6736]. Given that f (x1 ) = −5.8342 and
f (x2 ) = −5.8309. Apply suitable region elimination method rule to
obtain the next interval in the process of finding the minimum of
unimodal function.
Region Elimination Methods:

I Interval halving method

I Fibonacci search method
I Golden section search method
Interval Halving Method
In this method, function values at three different points are considered.
Three points divide the search space into four regions. Fundamental region
elimination rule is used to eliminate a portion of the search space based on
function values at the three chosen points. Three points chosen in the interval
(a, b) are all equidistant from each other and equidistant from the boundaries
by the same amount. Figure 2.6 shows these three points in the interval. Two
of the function values are compared at a time and some region is eliminated.
There are three scenarios that may occur.
♦ If f (x1 ) ≤ f (xm ), then the minimum cannot lie beyond xm . Therefore,
we reduce the interval from (a, b) to (a, xm ). The point xm being the
middle of the search space, this elimination reduces the search space to
50 per cent of the original search space.
There are three scenarios that may occur.
♦ If f (x1 ) ≤ f (xm ), then the minimum cannot lie beyond xm . Therefore,
we reduce the interval from (a, b) to (a, xm ). The point xm being the
middle of the search space, this elimination reduces the search space to
50 per cent of the original search space.
♦ On the other hand, if f (x1 ) ≥ f (xm ), the minimum cannot lie in the
interval (a, x1 ).
There are three scenarios that may occur.
♦ If f (x1 ) ≤ f (xm ), then the minimum cannot lie beyond xm . Therefore,
we reduce the interval from (a, b) to (a, xm ). The point xm being the
middle of the search space, this elimination reduces the search space to
50 per cent of the original search space.
♦ On the other hand, if f (x1 ) ≥ f (xm ), the minimum cannot lie in the
interval (a, x1 ).
♦ The point x1 being at one-fourth point in the search space, this
reduction is only 25 per cent. Thereafter, we compare function values at
xm and x2 to eliminate further 25 per cent of the search space.
There are three scenarios that may occur.
♦ If f (x1 ) ≤ f (xm ), then the minimum cannot lie beyond xm . Therefore,
we reduce the interval from (a, b) to (a, xm ). The point xm being the
middle of the search space, this elimination reduces the search space to
50 per cent of the original search space.
♦ On the other hand, if f (x1 ) ≥ f (xm ), the minimum cannot lie in the
interval (a, x1 ).
♦ The point x1 being at one-fourth point in the search space, this
reduction is only 25 per cent. Thereafter, we compare function values at
xm and x2 to eliminate further 25 per cent of the search space.
♦ This process continues until a small enough interval is found.
There are three scenarios that may occur.
♦ If f (x1 ) ≤ f (xm ), then the minimum cannot lie beyond xm . Therefore,
we reduce the interval from (a, b) to (a, xm ). The point xm being the
middle of the search space, this elimination reduces the search space to
50 per cent of the original search space.
♦ On the other hand, if f (x1 ) ≥ f (xm ), the minimum cannot lie in the
interval (a, x1 ).
♦ The point x1 being at one-fourth point in the search space, this
reduction is only 25 per cent. Thereafter, we compare function values at
xm and x2 to eliminate further 25 per cent of the search space.
♦ This process continues until a small enough interval is found.
The complete algorithm is described in the next slide.
Algorithm for Interval Halving Method

Algorithm:
I Step 1: Choose a lower bound a and an upper bound b. Choose also a
a+b
small number . Let xm = , L0 = L = b − a. Compute f (xm ).
2
Algorithm for Interval Halving Method

Algorithm:
I Step 1: Choose a lower bound a and an upper bound b. Choose also a
a+b
small number . Let xm = , L0 = L = b − a. Compute f (xm ).
2
L L
I Step 2: Set x1 = a + , x2 = b − . Compute f (x1 ) and f (x2 ).
4 4
Algorithm for Interval Halving Method

Algorithm:
I Step 1: Choose a lower bound a and an upper bound b. Choose also a
a+b
small number . Let xm = , L0 = L = b − a. Compute f (xm ).
2
L L
I Step 2: Set x1 = a + , x2 = b − . Compute f (x1 ) and f (x2 ).
4 4
I Step 3:
 If f (x1 ) ≤ f (xm ) set b = xm ; xm = x1 ; go to Step 5;
Algorithm for Interval Halving Method

Algorithm:
I Step 1: Choose a lower bound a and an upper bound b. Choose also a
a+b
small number . Let xm = , L0 = L = b − a. Compute f (xm ).
2
L L
I Step 2: Set x1 = a + , x2 = b − . Compute f (x1 ) and f (x2 ).
4 4
I Step 3:
 If f (x1 ) ≤ f (xm ) set b = xm ; xm = x1 ; go to Step 5;
 Else go to Step 4.
Algorithm for Interval Halving Method

Algorithm:
I Step 1: Choose a lower bound a and an upper bound b. Choose also a
a+b
small number . Let xm = , L0 = L = b − a. Compute f (xm ).
2
L L
I Step 2: Set x1 = a + , x2 = b − . Compute f (x1 ) and f (x2 ).
4 4
I Step 3:
 If f (x1 ) ≤ f (xm ) set b = xm ; xm = x1 ; go to Step 5;
 Else go to Step 4.
I Step 4:
 If f (x2 ) ≤ f (xm ) set a = xm ; xm = x2 ; go to Step 5;
Algorithm for Interval Halving Method

Algorithm:
I Step 1: Choose a lower bound a and an upper bound b. Choose also a
a+b
small number . Let xm = , L0 = L = b − a. Compute f (xm ).
2
L L
I Step 2: Set x1 = a + , x2 = b − . Compute f (x1 ) and f (x2 ).
4 4
I Step 3:
 If f (x1 ) ≤ f (xm ) set b = xm ; xm = x1 ; go to Step 5;
 Else go to Step 4.
I Step 4:
 If f (x2 ) ≤ f (xm ) set a = xm ; xm = x2 ; go to Step 5;
 Else set a = x1 , b = x2 ; go to Step 5.
Algorithm for Interval Halving Method

Algorithm:
I Step 1: Choose a lower bound a and an upper bound b. Choose also a
a+b
small number . Let xm = , L0 = L = b − a. Compute f (xm ).
2
L L
I Step 2: Set x1 = a + , x2 = b − . Compute f (x1 ) and f (x2 ).
4 4
I Step 3:
 If f (x1 ) ≤ f (xm ) set b = xm ; xm = x1 ; go to Step 5;
 Else go to Step 4.
I Step 4:
 If f (x2 ) ≤ f (xm ) set a = xm ; xm = x2 ; go to Step 5;
 Else set a = x1 , b = x2 ; go to Step 5.
I Step 5:
 Calculate L = b − a. If |L| ≤ , Terminate;
Algorithm for Interval Halving Method

Algorithm:
I Step 1: Choose a lower bound a and an upper bound b. Choose also a
a+b
small number . Let xm = , L0 = L = b − a. Compute f (xm ).
2
L L
I Step 2: Set x1 = a + , x2 = b − . Compute f (x1 ) and f (x2 ).
4 4
I Step 3:
 If f (x1 ) ≤ f (xm ) set b = xm ; xm = x1 ; go to Step 5;
 Else go to Step 4.
I Step 4:
 If f (x2 ) ≤ f (xm ) set a = xm ; xm = x2 ; go to Step 5;
 Else set a = x1 , b = x2 ; go to Step 5.
I Step 5:
 Calculate L = b − a. If |L| ≤ , Terminate;
 Else go to Step 2.
Algorithm for Interval Halving Method

Algorithm:
I Step 1: Choose a lower bound a and an upper bound b. Choose also a
a+b
small number . Let xm = , L0 = L = b − a. Compute f (xm ).
2
L L
I Step 2: Set x1 = a + , x2 = b − . Compute f (x1 ) and f (x2 ).
4 4
I Step 3:
 If f (x1 ) ≤ f (xm ) set b = xm ; xm = x1 ; go to Step 5;
 Else go to Step 4.
I Step 4:
 If f (x2 ) ≤ f (xm ) set a = xm ; xm = x2 ; go to Step 5;
 Else set a = x1 , b = x2 ; go to Step 5.
I Step 5:
 Calculate L = b − a. If |L| ≤ , Terminate;
 Else go to Step 2.
Interval Halving Method

Remark
• At every iteration, two new function evaluations are performed and the
interval reduces to half of that at the previous iteration.
n
• Thus, the interval reduces to about 0.5 2 L0 after n function evaluations.
• Thus, the function evaluations required to achieve a desired accuracy 
can be computed by solving the following equation:
n
(0.5) 2 (b − a) = .
Example

Example
54
Find the minimum of the function f (x) = x2 + , x ∈ (0, 5) using the
x
Interval Halving Method with  = 10−3 .

Solution:
Iteration:1
Step 1. We choose a = 0, b = 5, and = 103 . The point xm is the midpoint of
(0 + 5)
the search interval. Thus, xm = = 2.5. The initial interval
2
length is L0 = L = 5 − 0 = 5. The function value at xm is
f (xm ) = 27.85.
Example

Example
54
Find the minimum of the function f (x) = x2 + , x ∈ (0, 5) using the
x
Interval Halving Method with  = 10−3 .

Solution:
Iteration:1
Step 1. We choose a = 0, b = 5, and = 103 . The point xm is the midpoint of
(0 + 5)
the search interval. Thus, xm = = 2.5. The initial interval
2
length is L0 = L = 5 − 0 = 5. The function value at xm is
f (xm ) = 27.85.
5 5
Step 2. We set x1 = 0 + = 1.25 and x2 = 5 − = 3.75. The corresponding
4 4
function values are f (x1 ) = 44.76 and f (x2 ) = 28.46.
Example

Example
54
Find the minimum of the function f (x) = x2 + , x ∈ (0, 5) using the
x
Interval Halving Method with  = 10−3 .

Solution:
Iteration:1
Step 1. We choose a = 0, b = 5, and = 103 . The point xm is the midpoint of
(0 + 5)
the search interval. Thus, xm = = 2.5. The initial interval
2
length is L0 = L = 5 − 0 = 5. The function value at xm is
f (xm ) = 27.85.
5 5
Step 2. We set x1 = 0 + = 1.25 and x2 = 5 − = 3.75. The corresponding
4 4
function values are f (x1 ) = 44.76 and f (x2 ) = 28.46.
Step 3. By comparing these function values, we observe that f (x1 ) ≥ f (xm ).
Thus we continue with Step 4.
Step 4. We again observe that f (x2 ) ≥ f (xm ). Thus, we drop the intervals
(0.00, 1.25) and (3.75, 5.00). In other words, we set a = 1.25 and
b = 3.75. The outcome of this iteration is pictorially shown in Figure
2.7.
Step 4. We again observe that f (x2 ) ≥ f (xm ). Thus, we drop the intervals
(0.00, 1.25) and (3.75, 5.00). In other words, we set a = 1.25 and
b = 3.75. The outcome of this iteration is pictorially shown in Figure
2.7.
Step 5. The new interval is L = 3.75 − 1.25 = 2.5, which is exactly half of that
in the original interval (L0 = 5). Since |L| is not small, we continue
with Step 2. This completes one iteration of the interval halving
method.
Interval Halving method

new |L|
a&b xi f (xi ) Condition interval <
a=0 x1 = 1.25 f (x1 ) = 44.76 f (x1 ) > f (xm )
x2 = 3.75 f (x2 ) = 28.46 (1.25, 3.75) No
b=5 xm = 2.5 f (xm ) = 27.85 f (x2 ) > f (xm )
a = 1.25 x1 = 1.875 f (x1 ) = 32.32 f (x1 ) > f (xm )
x2 = 3.125 f (x2 ) = 27.05 (2.5, 3.75) No
b = 3.75 xm = 2.5 f (xm ) = 27.85 f (x2 ) < f (xm )
a = 2.5 x1 = 2.812 f (x1 ) = 27.11 f (x1 ) > f (xm )
x2 = 3.437 f (x2 ) = 27.53 (2.812, 3.437) No
b = 3.75 xm = 3.125 f (xm ) = 27.05 f (x2 ) > f (xm )

At the end of third iteration, the minimum lies in (2.812, 3.437).