9/3/2023

Numerical and Engineering Optimization Method

(NEOM)

Reference Optimization Books

Kalyanmoy Deb, ‘Optimization for Engineering Design’ PHI

Learning Pvt Ltd (2nd Edn.)
G.S.S Bhishma Rao, ‘Optimization Techniques’ SCITECH Pub.

1
 9/3/2023

Reference MATLAB Books

• Brian D Hahn, ‘Essential Matlab For Scientists And Engineers’

(Pub : Butterworth-Heinemann)
• Stephen J. Chapman, ‘MATLAB Programming for Engineers’
• Shaillendra Jain, ‘Modeling and Simulation using MATLAB –
Simulink’
• Bansal, Goel & Sharma, ‘MATLAB and its Applications in
Engineering’
• Nagrath and Gopal, ‘Control System Engineering’
• S S Sastry, ‘Numerical Analysis’
• M H Rashid, ‘Ps-piece for Power Electronics’

What is Optimization ?

2
 9/3/2023

Basic components of optimization techniques

Objective function:
• It is the mathematical function which expresses the main aim
of the optimization model to be either minimized or
maximized subject to certain constraints.
For example, in a manufacturing process, the aim may be
to maximize the profit or minimize the cost.

Multiple objective functions:

• To optimize a number of different objectives concurrently.
For example, in the panel design problem, it would be good to
minimize weight and maximize strength simultaneously

Basic components of optimization Techniques-2

• Design Variables/ Vectors :
A set of unknowns or variables control the value of the objective
function. If there are no variables, defining the objective function
and the problem constraints is not possible.
The design variables are often bounded, that is, they have
maximum and minimum values.

• Constraints
A set of constraints are those which allow the unknowns to take on
certain values but exclude others.
For example, one cannot spend negative amount of time on
any activity in manufacturing process, so one constraint is
"time" variable that is to be non-negative.

3
 9/3/2023

Classification of optimization problems

 To achieve the best possible result under given circumstances. Engineers

have to take decisions in design, construction, maintenance, etc. The goal
of all decisions is either to minimize effort or to maximize benefit, and to
find the conditions that give the maximum or the minimum value of a
function.

 If a point x* corresponds to the minimum value of a function f(x), the same

point corresponds to the maximum value of the function −f(x).

 Mathematical perceptions:
• Existence of constraints: an optimization problem can be classified as a constrained or
an unconstrained one.

• Nature of the equations: Linear, Quadratic, Polynomial, Non-linear depending upon the
nature of the objective functions and the constraints. The nature of the involved functions
dictates the type of solution procedure.

• Admissible values of the design variables: Depending upon the values permitted for the
design variables, optimization problems can be classified as integer or real valued, and
deterministic or stochastic.

4
 9/3/2023

Concept

So, optimization is the process to find values of the

variables that minimize or maximize the objective
function while satisfying the constraints.

Optimality Criteria & Conditions for a point to be

an optimal point
Three Types of Optimal points:

Local optimal point (x*)- If there exist no point in the neighbourhood

of x* which is

better than x* ; if no point in the neighbourhood has a function value

smaller than f(x*).

Global optimal point (x**): If there exist no point in the entire search
space which is
better than the point x**; if no point in the entire search space has a
function value smaller than f(x**).
Inflection Point (x*): If the function value f(x*) increases locally as
x*increases, and decreases locally as x*reduces or, if the function
value decreases locally as x* increases and increases locally as x*
decreases., OR
It is a point where the function changes from being
concave (concave downward) to convex (concave upward), or vice
versa.

5
 9/3/2023

Stationary point :
A stationary point of a differentiable function of one variable is a point on the
graph of the function where the function's derivative is zero.

Informally, it is a point where the function "stops" increasing or decreasing (hence

the name).

6
 9/3/2023

Linear Opt. function

When the objective function is linear, the
maximum and minimum values will occur at a
constraint boundary.

Maximum at
Minimum at boundary
boundary

Non-linear Opt. function

If the objective function is nonlinear, the
optimum value can occur anywhere:

Maximum in Maximum at
interior boundary
Maximum at
boundary Minimum in
Minimum in Minimum at interior
interior boundary
This means that any procedure attempting to minimize with a nonlinear objective
function needs to have two components: one that considers the boundaries (including
the corners) and one that checks the interior

7
 9/3/2023

Formulation of optimization Problem

• In optimization mathematical problem of n variables:

subject to the constraints,

gi(X) <=0 , i = 1, 2, …., m
lj(X) =0, j = 1, 2, …., p
where, X is an n-dimensional vector called the design vector, f(X) is called
the objective function, and gi(X) and lj(X) are known as inequality and
equality constraints, respectively.

This type problem is called a constrained optimization problem. The set of values of
X that satisfy the equation gi(X) <=0 forms a boundary surface in the design space
called a constraint surface and the other in which gi(X) > 0 is called infeasible region.

0

Unconstrained Optimization Problem

8
9/3/2023

9
 9/3/2023

Types of
minima

Saddle Points
 A saddle point is a point on a function that is a stationary
point but not a local extremum. It is a point at which a
function has partial derivatives equal to zero, but at which the
function has neither a maximum nor a minimum value.

 First and second derivative tests can be used to distinguish

between saddle points and other types of stationary points,
such as local minima and maxima and a point of inflection..

10
 9/3/2023

Maxima, Minima (call them the relative maxima and minima or more generally the
relative extrema) and Inflection points :

11
 9/3/2023

Unconstrained Univariate Optimization

Univariate optimization means
optimization of a scalar function of a
single variable:
y = f(x) or P(x)
How to determine the minimum?
•Direct Search methods (Fibonacci, Golden-
Section)
i.e. bracketing, interval
elimination, approximation

When nth order of derivative (n >1) has non-

zero value,
• Start with the interval (“bracket”) such that  If n is odd, and derivative has non-zero
the minimum x* lies inside. value, x* is an inflection point;
• Evaluate f(x) at two point inside the bracket.  If n is even , and derivative has non-zero
• Reduce the bracket. positive value, x* is a local minimum;
 If n is even , and derivative has non-zero
• Repeat the process.
negative value, x* is a local maximum;
• Can be applied to any function and
differentiability is not essential

12
 9/3/2023

Root Finding Methods

• Roots of a non-linear equation f(x) = 0
where,
f(x) - real valued function and that x is a real variable.
f(x) - continuous on an interval [a, b],
f(a)*f(b) < 0 i.e. f(x) changes sign on [a, b], and f(x) = 0 has at
least one root on the interval.

BISECTION METHOD
Bisection Method
To find a root, repeatedly halve the interval [a, b], keeping the half for which
f(x) changes sign i.e. to check sign of f(a) and f(b) and decide a new
interval such that f(a)f(b) < 0 .
This procedure is guaranteed to converge to a root.

Let c1 = (a+b)/2 is the midpoint of the initial interval, and cn is the midpoint of
the interval in the nth step, then the difference between cn and a solution c is
bounded by

13
 9/3/2023

BISECTION METHOD ALGORITHM

(a0 ,b0 )>(a1,b1)>(a2 ,b2 )>(a3,b3)>

containing the root.

Examples of Bisection Problem

• Find the largest root of f(x) = x6 − x − 1 = 0
accurate to within ε= 0.001.
First find f(a) and f(b) where a < b and f(a)*f(b) < 0.

14
 9/3/2023

Coding
Prob: Find the largest root of f(x) = x3 − x − 1 = 0
accurate to within ε= 0.001.

After 9 iterations the value of f(c) is less than our defined tolerance (0.0072393 < 0.01). This means that the value that approximates
best the root of the function f is the last value of c = 3.1611328.

15
 9/3/2023

Method of false position or the regula-falsi

method.
The bisection method used the midpoint of the interval [a, b] as the next iterate
(bracketed value) . This method converges at a fairly slow speed.

A better approximation is obtained if we find the point (c, 0) where the secant line L
(secant line intersects the curve at a minimum of two distinct points) joining the
points (a, f (a)) and (b, f (b)) crosses the x-axis as shown.

Method of false position or the regula-falsi

method.
To find the value c, we write down two versions of the slope
m of the line L:
m = [f (b) − f (a)]/(b − a) ………………….(1)

where the points (a, f (a)) and (b, f (b)) are used,

and, m = [0 − f (b)] / (c − b) ………………….(2)

where the points (c, 0) and (b, f (b)) are used.

Equating the slopes in (1) and (2), we have,

m = [f (b) − f (a)]/(b − a) = [0 − f (b)] / (c − b) = − f (b) / (c − b) ….(3)

c= b − (f (b)(b − a)) / ( f (b) − f (a)) …………..….(4)

16
 9/3/2023

Method of false position or the regula-falsi

method.

Three possibilities, same as before:

1. If f (a) and f (c) have opposite signs, a zero lies in [a, c].
2. If f (c) and f (b) have opposite signs, a zero lies in [c, b].
3. If f (c) = 0, then the zero is c.
Convergence:
Thus the decision process is achieved by considering a sequence of intervals
[an, bn] and at each step the approximation of the zero is

…………….(5)

The regula-falsi
method…Algorithm

Algorithm for the Regula–Falsi Method: Given a continuous function f(x)

1.Determine the points a and b such that a < b and f(a) * f(b) < 0.

2.Determine the next value of x1.

3.If f(x1) = 0 then x1 is an exact root,

4. If f(x1) * f(b) < 0 then a = x1,

5. If f(a) * f(x1) < 0 then b = x1.

4. Continue steps 4 & 5 until f(xi) = 0 or |f(xi)| £ ε, degree of accuracy (DoA).

17
 9/3/2023

Example: Regula-falsi method.

Find the root of x sin(x) − 1 = 0 that is located in the interval

[0, 2] (the function sin(x) is to be evaluated in radians).

1. We have f (0) = −1.00000000 and f (2) = 0.81859485, so a

root lies in the

2. interval [0, 2].

3. Use formula (5), and obtain c0 and f(c0).

4. Work out the new bracketed value (a1,b1) and determine c1

and f(c1).

5. Continue 2 0r 3 more iteration and get the value of c or r

(root of the equation).

18
 9/3/2023

Coding
Prob: Find the largest root of f(x) = x3 − 2x − 5 = 0

end
answer=p
plot(error)
grid on;
title ('Plot of Error');
xlabel('iterations');
ylabel('error');
fprintf('\n Therefore,x2=%.4f \n Here,
f(x2)= %.4f', x2(i),f(x2(i)))

19
 9/3/2023

Newton –Raphson Method for solving nonlinear systems

• Finding those values of x (root) which satisfy an equation

of the form f(x) = 0 for a given function f(x).
• An initial estimate of the root is found by drawing a graph
of the function in the neighbourhood of the root. This
estimate is then improved.
• The method is based upon a knowledge of the tangent
(slope) to the curve near the root. It is an “iterative”
method to improve the accuracy of the root.

20
 9/3/2023

NR-Formula

N-R ……(from Taylor series)

• Newton-Raphson method, also known as Newton’s method, is an

iterative method for solving nonlinear systems: 𝑓(𝑥)=0, starting
from an initial point 𝑥0.

• The method generates a series of solutions (𝑥i) that are expected to

converge to a fixed point 𝑥∗ that represents a root of the equation.

• To develop the method, it is assumed that an estimate of the solution

is available as 𝑥i, and use first order Taylor series to approximate
𝑓(𝑥) around 𝑥i, i.e.,
𝑓(𝑥i+△𝑥)=𝑓(𝑥i)+𝑓′(𝑥i) △ 𝑥
[Note: Taylor Series: 𝑓(𝑥k+Δ𝑥)=𝑓(𝑥k)+𝑓′(𝑥k)Δ𝑥+1/2! 𝑓′′(𝑥k)Δ𝑥2+⋯]
• Then, by setting 𝑓(𝑥𝑘+ △ 𝑥)=0, we can solve for the offset 𝛿𝑥, and
use it to update our estimate 𝑥𝑘 as:
𝑥i+1=𝑥i−𝑓(𝑥i)/𝑓′(𝑥i)
 The N-R iteration can be terminated when the relative error
€a ( i.e. €a =(xn+1-Xn)/ (xn+1) x 100%) is less than say, 1%.

21
 9/3/2023

N-R ……(from Taylor series)

An infinite Taylor series expansion of 𝑓(𝑥) around 𝑥k is

given as (where Δ𝑥=𝑥−𝑥k):

𝑓(𝑥k+Δ𝑥)=𝑓(𝑥k)+𝑓′(𝑥k)Δ𝑥+1/2! 𝑓′′(𝑥k)Δ𝑥2+⋯

Take the partial derivative of each side wrt Δ𝑥,

𝑓’(𝑥k+Δ𝑥)=𝑓’(𝑥k)+𝑓′′(𝑥k)Δ𝑥

Our goal is 𝑓’(𝑥k+Δ𝑥) = 0, therefore kth iterate should be

𝑥k+Δ𝑥≈𝑥k - 𝑓’(𝑥k) / 𝑓’′(𝑥k)- New value

N-R : Quadratic Convergence

Convergence Rate.
The NR method requires a good initial guess for it to converge.

Further, if it converges, exhibits quadratic rate of convergence near the

solution point. The method can become unstable if 𝑓(𝑥∗)≈0.

Assuming 𝑓′(𝑥∗)≠0, and 𝑥𝑘 is sufficiently close to 𝑥∗, we can use

second order Taylor series to write:

𝑥𝑘+1−𝑥∗ ≈ (1/2) (𝑓′′(𝑥∗)/𝑓′(𝑥∗)) (𝑥𝑘−𝑥∗)2

It shows that Newton’s method has quadratic convergence with a rate

constant: 𝐶=(1/2) |𝑓′′(𝑥∗)/𝑓′(𝑥∗)|.
Note: The error in the current iteration is proportional to the square of the previous
error. The number of correct decimal places in the NR solution doubles after each
iteration.

22
 9/3/2023

N-R : Drawbacks

DRAWBACKS of NR:

• The guess value of the root should be near to the solution

• The soln may diverge near a point of inflectiona point (at which a
change in the direction of curvature occurs)

• Might oscillates new local minima or maxima near local minima or

maxima

• With near zero slope , the solution may diverge or reach a different
root

• Slow convergence

N-R : PROBLEMS
• f(x) = x−2+ln(x) has a root near x = 1.5. Use the Newton-Raphson formula
to obtain a better estimate

23
 9/3/2023

N-R
Newton’s method is an iterative method for
solving nonlinear systems
%The following script, to be saved under the name ‘newton.m’, to implements Newton’s
%method, to compute the root of the equation x3 − 3x +1 = 0.
%format
%short;
• clc;
• syms x;
• display (’Newton’’s method is an iterative method for solving
• nonlinear systems’);
• f = input(’Enter the function f(x)=’);
• x0 = input(’Enter the initial estimate of the solution: ’);
• n = input(’Enter the maximum number of iterations: ’);
• e = input(’Enter the margin of error: ’);
• i = 0;
• while (i<n)
• df = diff(f,x);
• dfx0 = subs(df,x,x0);
• fx0 = subs(f,x,x0);
• x1 = x0 - (fx0/dfx0);
• dx = x1 - x0;
• if abs(dx)/abs(x0)<=e
• fprintf(’The solution is = %i\n’,x1);
• fprintf(’The number of iterations required was = %i\n’,i);
• break
• end
• i = i+1;
• x0=x1;
• end
• if abs(dx)/abs(x0) >e
• fprintf(’Convergence was not achieved in n iterations ’);
• end

N-R-2
Input:
• Enter the function f(x)=x^3-3*x+1
• Enter the initial estimate of the solution: 0
• Enter the maximum number of iterations: 100
• Enter the margin of error: 0.001
Output:
• The solution is = 3.472964e-001
• The number of iterations required was = 2
• Therefore, the solution computed by this script is x = 0.3473.

24
 9/3/2023

N-Optimization

N-R Coding

25
 9/3/2023

N-R Coding
% N-R Solution for f(x)=x-cos(x)
clc
i=1;
p0=0.5*pi;
N=40;
error=0.0001;
syms x
f=x-2*cos(x)+1;
df=diff(f);
while i<=N
p=p0-(subs(f,p0)/subs (df,p0))
if abs ((p0-p)/p0)< error
fprintf('solution is %f \n',p)
disp(i)
return
end
i=i+1
p0=p
end
fprintf('Diverge after %d itrations at precision of %d \n',N,error)

Ans.0.6235 (4th iteration)

N-R Coding

26
 9/3/2023

Numerical Technique to solve ordinary differential

equations (First Order ODE)

• The diff equations to be solved in power system stability analysis are nonlinear
ordinary differential equations with known initial values:

• dy/dx = f(x,y) ………………………………………………………..(1)

Where, y - state vector of n dependent variables

x - independent variable.

 Objective is to solve y as a function of x, with initial values of y and x equal to y0

and x0 respectively.

Euler Method

27
 9/3/2023

Euler’s numerical method

Runge-Kutta second order and fourth order methods

(N.B. 2nd order RK method is also called midpoint method)

28
 9/3/2023

Runge-Kutta fourth order method

The value of y is defined as,

where,

Runge-Kutta fourth order method

where,
k1= (slope at the beginning of step size) x h
k2= (first approximation to slope at midstep) x h
k3= (second approximation to slope at midstep) x h
k4= (slop at the end of step) x h

The incremental value of y, △y = 1/6( k1 + 2 k2 +2 k3 + k4)

This is the weighted average of estimates based on slopes at the
beginning, midpoint and end of the time step.
In general terms,
yn+1 = yn + 1/6( k1 + 2 k2 +2 k3 + k4)
where, k1= f (xn, yn) * h and so on for k2,k3 &k4.

[If f is independent of t, the differential equation is equivalent to a simple integral, then

RK4 is Simpson's rule, and If f is independent of t (so called autonomous system, or time-
invariant system), and their increments are not computed at all and not passed to function f]

29
 9/3/2023

Example of Runge-Kutta4 Method

Solve
y’ = f (t, y) = y − t2 + 1
y(0) = 0.5

for 0 ≤ t ≤ 2 and h=0.5

Exact solution for the problem is,

y = t2 + 2t + 1 – ½(et )
Coding follows

%Rungekutta-4 soln for f=y'=y-t.^2+1;...........OK

% True solution x_true=t.^2+2*t+1.0-0.5*exp(t)%%y=(t+1).^+0.5et
% 0<=t<=2 and x(0)=0.5

%function f=f(t,y)
%f=y-t.^2+1;

%Main file
clc
h=0.25; % step size
t(1)=0;% zero time
x(1)=0.5;% initial condition
y_true=zeros(1,8)
for i=1:8
k1=h*f(t(i),x(i));
k2=h*f(t(i)+h/2,x(i)+k1/2);
k3=h*f(t(i)+h/2,x(i)+k2/2);
k4=h*f(t(i)+h,x(i)+k3);
x(i+1)=x(i)+(k1+2*k2+2*k3+k4)/6;
t(i+1)=t(i)+h;
y_true=t.^2+2*t+1.0-0.5*exp(t)
end
%t=[0:h:2]
%y_true=t.^2+2*t+1.0-0.5*exp(t)
plot(t,x,'k-',t,y_true,'m+')
xlabel('t')
ylabel('x')
legend('Runge Kutta 4th Order:True Soln vs. RK soln.')

30
 9/3/2023

RK4 method for Transient Studies of RLC series

circuit.

Parameters: R=300, L= 10mH,C=1uF and Vin=10V

dVc
i C
dt
di
L  iR  Vc  Vin
dt
d 2V dV
or , LC 2c  RC c  Vc  Vin
dt dt

1
Vc  2nVc  n 2Vc  Vin
LC
1
ω n= Where,
LC
R C 𝝎n=Natural frequency
ξ= ξ= Damping co-efficient
2 L

This is a second order differential equation.

For a change in the value of R, damping co-efficient
varies considerably. It can be observed that
(i) for R=300 Ω, ξ= ?? system is overdamped
(ii) For R=200 Ω, ξ= ?? system is critically damped
(iii) For R=100 Ω, ξ= ?? system is underdamped

31
 9/3/2023

Solutions: First order Equations for voltage and

current
dVc
i C
dt
dVc 1
 *i
dt C
1
x1 = * x 2 (1), where x 1  Vc
C
di d 2V dV
L  iR  Vc  Vin or , LC 2c  RC c  Vc  Vin
dt dt dt
di 1
 (Vin  Vc  iR )
dt L
1 1 R
x2  * Vin  * Vc  * i , where x 2  i
L L L
1 1 R
x 2 = * Vin - * x 1 - * x 2 (2)
L L L

CODING:
% Runge Kutta 4 Solution .
% Let’s get the transient analysis of circuit for an
overdamped system.
% R=300,200,100 L= 10mH,C=1uF and Vin=10V & W0
%=1000
% I(0)=0,Vc(0)=0%
clc
t=0; T=0.001; h=0.0001; x1=0; x2=0; i=1;
%c=0.000001;l= 0.01; r=300; Vin=10;
%c=0.000001;l= 0.01; r=200; Vin=10;
c=0.000001;l= 0.01; r=100; Vin=10;%underdamped
f=@(t,x1,x2) x2/c;
g= @(t,x1,x2)((Vin-x1-r*x2)/l);
fprintf('time\t\tvoltage\tcurrent\n');
for t=0:h:T
fprintf('%f\t%f\t%f\n',t,x1,x2);

32
 9/3/2023

X(i)=x1;% capacitor voltage Vc

Y(i)=x2;% Current i
f1=h*f(t,x1,x2);
g1=h*g(t,x1,x2);
f2=h*f((t+h/2),(x1+f1/2),(x2+g1/2));
g2=h*g((t+h/2),(x1+f1/2),(x2+g1/2));
f3=h*f((t+h/2),(x2+f2/2),(x2+g2/2));
g3=h*g((t+h/2),(x1+f2/2),(x2+g2/2));
f4=h*f((t+h),(x1+f3),(x2+g3));
g4=h*g((t+h),(x1+f3),(x2+g3));
x1=x1+((f1+f4)+2*(f2+f3))/6.0;
x2=x2+((g1+g4)+2*(g2+g3))/6.0;
i=i+1;
end
time=[0:h:T];
subplot(2,1,1)
plot(time,X,'-p');
xlabel('time(s)');
ylabel('voltage(v)');
grid on;
subplot(2,1,2) Underdamped
plot(time,Y,'-k');
xlabel('time(s)');
ylabel('current(A)');
grid on;

Critically damped system

33
 9/3/2023

Over damped

THANKS

34