Engineering Mathematics III

Lecture 8: Roots of Equation

------ Secant method
------Systems of Non-linear Equations
------ Interpolation

1
Bisection Method
Based on the fact that the function will change signs as it
passes thru the root.

f(a)*f(b) < 0
Once we have a root bracketed, we simply evaluate the mid-
point and halve the interval.

f(a)>0

c=(a+b)/2

f(c)>0

a c b

f(b)<0
2
Absolute Error
What does this mean in binary mode?

err0 ≤ |b-a|

erri+1 ≤ erri/2 = |b-a|/2i+1
We gain an extra bit each iteration!!!
To reach a desired absolute error tolerance:
erri+1 ≤ errtol ⇒ b−a


n
≤ errtol
2
 b−a 
n ≥ log 2  
err
 tol 
If we need an accuracy of 0.0001 and our initial
interval (b-a)=1, then:
2-n < 0.0001 ⁼⇒
⁼ 14 iterations
3
Newton’s Method
Using an approximate value x0 obtained from the graph of f,
we let x1 be the point of intersection of the x-axis and the
tangent to the curve of f at x0.

hence,

f ( xn )
⇒ xn +1 = xn −
f ' ( xn )

CIV 303 Eng. Math. 4

III
Secant Method

What if we do not know the derivative of f(x)?

Secant line

Tangent vector

xi xi-1

5
Secant Method

As we converge on the root, the secant line

approaches the tangent.
Hence, we can use the secant line as an estimate
and look at where it intersects the x-axis (its root).

6
Secant Method

This also works by looking at the definition of

the derivative: f ′( x) = lim f ( x + h) − f ( x)
h →0 h
f ( xk ) − f ( xk −1 )
f ′( xk ) ≈
xk − xk −1

Therefore, Newton’s method gives:

 xk − xk −1 
xk +1 = xk −   f ( xk )
 f ( xk ) − f ( xk −1 ) 
Which is the Secant Method.
The secant method requires two initial approximations x0
and x1, preferrably both reasonably close to the solution x∗.
7
Secant Method
Example
Numerical Example :Find the root of x4-x-10 =0
Let the initial guess be 1 and 2
f(x) = x4-x-10
No. x f(x)
0 1 -10
1 2 4
2 1.714286 -3.07788
3 1.838531 -0.41280
4 1.857776 0.05391
5 1.855553 -0.00078
6 1.855584 0.00000
7 1.855585 0.00000

8
Higher--dimensional Problems
Higher

Consider the class of functions

f(x1,x2,x3,…,xn)=0,
where we have a mapping from ℜn→ℜ.
We can apply Newton’s method separately for
each variable, xi, holding the other variables fixed
to the current guess.

9
Higher--dimensional Problems
Higher
This leads to the iteration:
f ( x1 , x2 ,… , xn )
xi → xi −
f xi ( x1 , x2 ,… , xn )

Two choices, either I keep of complete set of

old guesses and compute new ones, or I use
the new ones as soon as they are updated.
Might as well use the more accurate new
guesses.
Not a unique solution, but an infinite set of
solutions.

10
Higher--dimensional Problems
Higher

Example:

x+y+z=3

Solutions:

x=3, y=0, z=0

x=0, y=3, z=0

…

11
Systems of Non-
Non-linear Equations

Consider the set of equations:

f1 ( x1 , x2 ,… , xn ) = 0
f 2 ( x1 , x2 ,… , xn ) = 0
⋮
f n ( x1 , x2 ,… , xn ) = 0

12
Systems of Non-
Non-linear Equations

Example:
Plane intersected with a
x+ y+z =3 sphere, intersected with a
more complex function.
x2 + y2 + z 2 = 5
e x + xy − xz = 1

Conservation of mass coupled with conservation

of energy, coupled with solution to complex
problem.

13
Vector Notation

We can rewrite this using vector notation:

f ( x) = 0

f = ( f1 , f 2 ,… , f n )
x = ( x1 , x2 ,… , xn )

14
Newton’s Method for
Non--linear Systems
Non
Newton’s method for non-linear systems can be
written as:

− f ′ ( x ) ( )
−1
x ( k +1)
=x (k ) (k )
 f x (k )

where f ( x ) is the Jacobian matrix

′ (k )

15
The Jacobian Matrix
The Jacobian contains all the partial
derivatives of the set of functions.
 ∂f1 ∂f1 ∂f1 
 ∂x ⋯
∂x2 ∂xn 
 1 
 ∂f 2 ∂f 2 ∂f 2 
⋯
J =  ∂x1 ∂x2 ∂xn 

 ⋮ ⋮ ⋱ ⋮ 
 
 ∂f n ∂f n
⋯
∂f n 
 ∂x1 ∂x2 ∂xn 

Note, that these are all functions and need to

be evaluated at a point to be useful.

16
The Jacobian Matrix

Hence, we write
 ∂f1 (i ) ∂f1 (i ) ∂f1 ( i ) 
 ∂x ( x ) ∂x2
( x ) ⋯
∂xn
( x )
 1 
 ∂f 2 (i ) ∂f 2 (i ) ∂f 2 ( i ) 

J (x ) =  ∂x1
(i ) ( x )
∂x2
( x ) ⋯
∂xn
( x )

 ⋮ ⋮ ⋱ ⋮ 
 
 ∂f n ( x (i ) ) ∂f n ( i )
( x ) ⋯
∂f n ( i ) 
( x )
 ∂x1 ∂x2 ∂xn 

17
Matrix Inverse

We define the inverse of a matrix, the same as the

reciprocal.

1
a =1
a
1 0 ⋯ 0
0 1 ⋯ 0 
AA −1 = I = 
⋮ ⋮ ⋱ ⋮
 
0 0 ⋯ 1

18
Newton’s Method
If the Jacobian is non-singular, such that its
inverse exists, then we can apply this to Newton’s
method.
We rarely want to compute the inverse, so instead
we look at the problem.
= x − f ′ ( x ) ( )
−1
x ( i +1) (i ) (i )
 f x (i )

= x(i ) + h(i )
Now, we have a linear system and we solve for h.
 J ( x ( k ) )  h ( k ) = −f ( x ( k ) )
 
x (i +1) = x( i ) + h ( i )
Repeat until h goes to zero.
19
Initial Guess

How do we get an initial guess for the root vector

in higher-dimensions?
In 2D, I need to find a region that contains the
root.
Steepest Decent is a more advanced topic not
covered in this course. It is more stable and can be
used to determine an approximate root.

20
Interpolation
Often, we have data sets from
experimental/observational measurements

Typically, find that the data/dependent variable/output
varies…

As the control parameter/independent variable/input varies.
Examples:

Classic gravity drop: location changes with time

Pressure varies with depth

Wind speed varies with time

Temperature varies with location
Scientific method: Given data identify underlying
relationship
Process known as curve fitting:

21
Interpolation & Curve-
Curve-fitting

Given a data set of n+1 points (xi,yi) identify a function

f(x) (the curve), that is in some (well-defined) sense
the best fit to the data
Used for:

Identification of underlying relationship

(modelling/prediction)

Interpolation (filling in the gaps)

Extrapolation (predicting outside the range of the

data)

22
Interpolation Vs Regression
Distinctly different approaches depending on the
quality of the data
Consider the pictures below:

extrapolate

interpolate extrapolate

Pretty confident: Unsure what the relationship is

there is a polynomial relationship Clear scatter
Little/no scatter Want to find an expression
Want to find an expression that captures the trend:
that passes exactly through all the minimize some measure of the error
points Of all the points…
23
Interpolation
Concentrate first on the case where we believe there is
no error in the data (and round-off is assumed to be
negligible).
So we have yi=f(xi) at n+1 points x0,x1…xi,…xn: xj > xj-1
(Often but not always evenly spaced)
In general, we do not know the underlying function f(x)
Conceptually, interpolation consists of two stages:

Develop a simple function g(x) that

Approximates f(x)

Passes through all the points xi

Evaluate f(xt) where x0 < xt < xn

24
Interpolation
Clearly, the crucial question is the selection of
the simple functions g(x)
Types are:

Polynomials

Splines

Trigonometric functions

Spectral functions…Rational functions etc…
Three common used possible approximations:
Polynomial interpolation
Spline (polynomial) interpolation
Least-squares (polynomial) approximation
If you know your function is periodic, then trigonometric functions
may work better.
Fourier Transform and representations
25
Polynomial Interpolation

Consider our data set of n+1 points yi=f(xi) at n+1

points x0,x1…xi,…xn: xj > xj-1
In general, given n+1 points, there is a unique
polynomial gn(x) of order n:

g n ( x) = a0 + a1 x + a2 x + … + an x
2 n

That passes through all n+1 points

26
Polynomial Interpolation

There are a variety of ways of expressing the same

polynomial
Lagrange interpolating polynomials
Newton’s divided difference interpolating
polynomials
We will look at both forms

27
Polynomial Interpolation

Existence – does there exist a polynomial that

exactly passes through the n data points?
Uniqueness – Is there more than one such
polynomial?

We will assume uniqueness for now and
prove it latter.

28
Linear Interpolation
Simple linear interpolation results from having
only 2 data points.

f ( x1 ) − f ( x0 )
p1 ( x) = f ( x0 ) + ( x − x0 )
x1 − x0

slope

x0 x1
Quadratic Interpolation
Three data points:

f ( x1 ) − f ( x0 )
p2 ( x) = f ( x0 ) + ( x − x0 ) + f [ x0 , x1 , x2 ]( x − x0 )( x − x1 )
x1 − x0
 f ( x2 ) − f ( x1 )   f ( x1 ) − f ( x0 ) 
  − 
f ( x1 ) − f ( x0 )  x2 − x1   x1 − x0  ( x − x )( x − x )
= f ( x0 ) + ( x − x0 ) +
x1 − x0
0 1
x 2 − x0
f ( x1 ) − f ( x0 )
= f ( x0 ) + ( x − x0 )
x1 − x0
  f ( x2 ) − f ( x1 )     f ( x1 ) − f ( x0 )  
 ( x − x )
1  ( x − x ) −
0   ( x − x )
0  ( x − x1 
)
 x − x    x − x 
+ 
2 1 1 0

x2 − x0