SUPPLEMENTARY LECTURE NOTES

System of non-linear equations

Objectives
At the end of this section you will learn how to solve non-linear systems using
1. functional fixed-point iteration,
2. the Gauss-Seidel(Extra Reading) and
3. Newton’s methods.

Introduction
Solutions x  x0 to equations of the form f ( x)  0 are often required where it is
impossible or infeasible to find an analytical expression for the vector x. If the scalar
function f depends on n independent variables x1 , x2 , , xn , then the solution x0 will
describe a surface in n – 1 dimensional space. Alternatively we may consider the vector
function f(x) = 0, the solutions of which typically collapse to particular values of x. For
this course we restrict our attention to n independent variables x1 , x2 , , xn and seek
solutions to F (x)  0 where F is vector valued.

Fixed Point Method for Functions of Several Variables

The general of a system of nonlinear equations is
f1 ( x1 , x2 , , xn )  0
f 2 ( x1 , x2 , , xn )  0

f n ( x1 , x2 , , xn )  0
where each function f i maps n-dimensional space, R n , into the real line R. The above
system can be defined alternatively by defining the function F (x)  0 , where
F : Rn  Rn , x  ( x1 , x2 , , xn ) and
F ( x1 , x2 , , xn )   f1 ( x1 , x2 , , xn ), f 2 ( x1, x2 , , xn ), , f n ( x1, x2 , , xn )  .

1.1 Functional or Fixed Point Iteration

Suppose a nonlinear system of the form F (x)  0 has been transformed into an
equivalent fixed point problem G(x)  x . The functional or fixed point iteration process
applied to G is as follows:
1. Select x (0)   x1(0) , x2(0) , , xn(0)  .
2. Generate the sequence of vectors x ( k )   x1( k ) , x2( k ) , 
, xn( k ) by
 
x ( k )  G x ( k 1) for each i  1, 2,3, or, component-wise,

1
Numerical Analysis Non-Linear Iterative Methods


x1( k )  g1 x1( k 1) , x2( k 1) , , xn( k 1) 
x2( k )  g x 2
( k 1)
1 , x2( k 1) , , xn( k 1) 


xn( k )  g n x1( k 1) , x2( k 1) , , xn( k 1) 
The following theorem provides conditions for the iterative process to converge.

Theorem
Let D  ( x1 , x2 , , xn ) : ai  xi  bi , for each i  1, 2, , n , for some collection of
constants (a1 , a2 , , an ) and (b1 , b2 , , bn ) . Suppose G is a continuous function with
continuous first partial derivatives from D  Rn into R n with the property that
G(x)  D whenever x  D . If a constant K  1 exists with
gi (x) K
 whenever x  D
x j n

 

for each j  1, 2, , n and each component function g i , then the sequence x ( k )
k 0

defined by x (k )
G x  ( k 1)
 for each i  1, 2,3, converges to the unique fixed point
Kj
p  D , for any x (0) in D , and x ( j )  p  x (1)  x (0) .
 1 K 

Example 1
3x1  cos( x2 x3 )  12  0
x12  81( x2  0.1) 2  sin x3  1.06  0
e x1x2  20 x3  1033  0

f1 ( x1 , x2 , , xn )  3x1  cos( x2 x3 )  12
f 2 ( x1 , x2 , , xn )  x12  81( x2  0.1) 2  sin x3  1.06
f3 ( x1 , x2 , , xn )  e  x1x2  20 x3  1033

F ( x1 , x2 , , xn )   f1 ( x1 , x2 , , xn ), f 2 ( x1 , x2 , , xn ), , f n ( x1 , x2 , , xn ) 

 3x1  cos( x2 x3 )  12 , x12  81( x2  0.1) 2  sin x3  1.06, e x1x2  20 x3  1033 
Example 2
3x1  cos( x2 x3 )  12  0
x12  81( x2  0.1) 2  sin x3  1.06  0
e x1x2  20 x3  1033  0

2
Numerical Analysis MATH351/352

If the ith equation is solved for xi , the system can be changed into the fixed point
problem
x1  13 cos( x2 x3 )  16 ,
x2  19 x12  sin x3  1.06  0.1,
x3   201 e x1x2  1060
 3
.

Let G : R3  R3 be defined by G(x)   g1 (x), g2 (x), g3 (x)  where

g1 ( x1 , x2 , x3 )  13 cos( x2 x3 )  16 ,
g 2 ( x1 , x2 , x3 )  19 x12  sin x3  1.06  0.1,
g3 ( x1 , x2 , x3 )   201 e  x1x2  1060
 3
.

g1 ( x1 , x2 , x3 )  13 cos( x2 x3 )  16  12 ,

g 2 ( x1 , x2 , x3 )  1
9 x12  sin x3  1.06  0.1

 19 1  sin1  1.06  0.1  0.90,

g3 ( x1 , x2 , x3 )  1
20 e  x1x2  1060
 3

 1
20 e  1060
 3
 0.61
.
so 1  gi ( x1 , x2 , x3 )  1, for each i  1, 2,3. Thus, G(x)  D whenever x  D .
Finding bounds on the partial derivatives on D gives the following:

g1 g1 1 g1 1

 0,  x3 sin( x2 x3 )  13 sin1  0.281,  x2 sin( x2 x3 )  13 sin1  0.281,
x1 x2 3 x3 3

g 2 x1 1 g 2
   0.238,  0,
x1 9 x 2  sin x  1.06 9 0.218 x2
1 3

g 2 cos x3 1
   0.119,
x3 18 x 2  sin x  1.06 18 0.218
1 3

g3 x g3 x
 2 e x1x2  201 e  0.14,  1 e x1x2
x1 20 x2 20

3
Numerical Analysis Non-Linear Iterative Methods

Since the partial derivatives are bounded on D, the above Theorem implies that these
functions are continuous on D. Consequently, G is continuous on D. Moreover, for every
xD
gi (x)
 0.281 for each i  1, 2,3 and j  1, 2,3, and the condition in the second part of
x j
Theorem 9.7 holds for K  0.843 . It can be shown that
gi (x) x j for each i  1, 2,3 and j  1, 2,3 is continuous on D. Consequently, G has a
unique fixed point on D and the nonlinear system has a solution in D.

Example 3
3x1  cos( x2 x3 )  12  0
x  81( x2  0.1)  sin x3  1.06  0
2
1
2

e x1x2  20 x3  1033  0

Solution
If the ith equation is solved for xi , the system can be changed into the fixed point
problem as
x1  13 cos( x2 x3 )  16 ,
x2  19 x12  sin x3  1.06  0.1,
x3   201 e x1x2  1060  3
.
and write the iterative process as
x1( k )  13 cos  x2( k 1) x3( k 1)   16 ,

x 
2
( k 1)
x2( k )  19 1  sin x3( k 1)  1.06  0.1,
( k 1) ( k 1)
x3( k )   201 e x1 x2
 1060
 3
.

x ( k )  x ( k 1)  105


k x1( k ) x2( k ) x3( k ) x ( k )  x ( k 1)



0 0.10000000 0.10000000 0.10000000 --

1 0.49998333 0.00944115 0.52310127 0.423
2 0.49999593 0.00002557 0.52336331 9.4 103
3 0.50000000 0.00001234 0.52359814 2.3 104
4 0.50000000 0.00000003 0.52359847 1.2 105
5 0.50000000 0.00000002 0.52359877 3.1107

4
Numerical Analysis MATH351/352

(0.843)5
x (5)  p  0.423  1.15
 1  0.843


x (3)  0.50000000,1.234 105 , 0.52359814 
(0.843)5
x (5)  p  (1.20 105 )  5.5 105
 1  0.843

p   0.5,0,  6    0.5,0, 0.5235987757 

x (5)  p  2 108



x1( k )  13 cos x2( k 1) x3( k 1)  16 , 
x 
2
x2( k )  19 (k )
1  sin x3( k 1)  1.06  0.1,

x3( k )   201 e  x1
(k ) (k )
x2
 1060
 3
.

k x1( k ) x2( k ) x3( k ) x ( k )  x ( k 1)



0 0.10000000 0.10000000 0.10000000 --

1 0.49998333 0.02222979 0.52304613 0.423
2 0.49997747 0.00002815 0.52359807 2.2 102
3 0.50000000 0.00000004 0.52359877 2.8 105
4 0.50000000 0.00000000 0.52359877 3.8 108

Newton’s Method
In order to construct an algorithm that led to an appropriate fixed-point method
 a11 (x) a12 (x) a1n (x) 
 a21 (x) a22 (x) a2 n (x) 
A(x)   
 
 an1 (x) an 2 (x) ann (x) 
where each of the entries aij (x) is a function from Rn  R . The procedure requires
that A(x) be found so that G(x)  x  A(x)1 F (x) gives quadratic convergence to the
solution F (x)  0 , provided that A(x) is nonsingular at the fixed point.

5
Numerical Analysis Non-Linear Iterative Methods

Theorem
Suppose p is a solution of G(x)  x for some function G  ( g1 , g2 , , gn ) , mapping R n
into R n . If a number   0 exists with the property that
i) gi x j is continuous on N  x : x  p    for each
i  1, 2, , n and j  1, 2, ,n,
ii)  2 gi (x) x j xk is continuous, and  2 gi (x) x j xk  M for some constant M
whenever x  N for each i  1, 2, , n, j  1, 2, , n and k  1, 2, ,n,
iii) gi (p) x j  0 for each i  1, 2, , n and j  1, 2, , n ,
then the sequence generated by x ( k )  G  x ( k 1)  converges quadratically to p
n2 M ( k 1) 2
for any choice of x (0)  N and x ( k )  p  x p for each k  1 .
 2 

n
Since G(x)  x  A(x)1 F (x) , gi (x)  xi   bij (x) f j (x) ;
j 1

 n
 f j bij 
1    bij (x) (x)  (x) f j (x)  , if i  k ,
g (x)  j 1  xk xk 
so i  n
xk  f 
   b (x) j (x)  ij (x) f (x)  , if i  k.
b
   ij
xk xk
j 
 j 1  

n f j
0  1   bij (p) (p)
j 1 xk

n f j
so  b (p) x
j 1
ij (p)  1
k

n f j
0   bij (p) (p)
j 1 xk

n f j
So  b (p) x
j 1
ij (p)  0
k

6
Numerical Analysis MATH351/352

 f1 (x) f1 (x) f1 (x) 

 x x2 xn 
 1 
 f 2 (x) f 2 (x) f 2 (x) 
J (x)   x1 x2 xn 
 
 
 f n (x) f n (x) f n (x) 
 x x2 xn 
 1

A(p)1 J (p)  I

so J (p)  A(p)
G(x)  x  J (x)1 F (x)
x ( k )  G  x ( k -1)   x (k -1)  J  x (k -1)  F  x (k -1) 
1

Example 1
Solve the nonlinear system
3x1  cos( x2 x3 )  12  0
x  81( x2  0.1)  sin x3  1.06  0
2
1
2

e x1x2  20 x3  1033  0
Solution
The Jacobian matrix for the system is given by
 3 x3 sin x2 x3 x2 sin x2 x3 
J ( x1 , x2 , x3 )   2 x1 162( x2  0.1) cos x3 
  x1 x2  x1 x2

  x2e  x1e 20 
and
 x1( k )   x1( k 1)   h1( k 1) 
 x ( k )    x ( k 1)    h( k 1) 
 2( k )   2( k 1)   2( k 1) 
 x3   x3   h3 
     
where
 h1( k 1) 
 h( k 1)     J x ( k 1) , x ( k 1) , x ( k 1)  1 F x( k 1) , x( k 1) , x( k 1) .
 2( k 1)    1 2 3   1 2 3 
 h3 
 

Thus, at the kth step, the linear system

7
Numerical Analysis Non-Linear Iterative Methods

3 x3( k 1) sin x2( k 1) x3( k 1) x2 sin x2( k 1) x3( k 1)   h1( k 1) 
 ( k 1)   ( k 1) 
 2 x1 162( x2( k 1)  0.1) cos x3( k 1)   h2( k 1) 
 x e 1 2
( k 1)  x ( k 1) ( k 1)
x
 x1 e 1 2
( k 1)  x ( k 1) ( k 1)
x   h3 
 2 20  
 3x1( k 1)  cos( x2( k 1) x3( k 1) )  12 
 ( k 1) 2 
   x1   81( x2( k 1)  0.1) 2  sin x3( k 1)  1.06 
  x1( k 1) x2( k 1) 
e  20 x3( k 1)  1033 
must be solved. The results obtained using the above iterative procedure is as shown
below

k x1( k ) x2( k ) x3( k ) x ( k )  x ( k 1)



0 0.10000000 0.10000000 0.10000000 --

1 0.50003702 0.01946686 0.52152047 0.422
2 0.50004593 0.00158859 0.52355711 1.79 102
3 0.50000034 0.00001244 0.52359845 1.58 103
4 0.50000000 0.00000000 0.52359877 1.24 105
5 0.50000000 0.00000000 0.52359877 0

p   0.5,0,  6    0.5,0, 0.5235987757 

Examples
1. The nonlinear system
 x 2  10 x  y 2  8
F( x, y )   2 0
 xy  x  10 y  8
can be transformed into the fixed point problem
 x   10 
x 2  y 2 8
G( x, y)      xy2  x 8 
 y   10 

(a) Starting with the initial estimates x0  y0  0 , apply functional iteration to

G to approximate the solution to an accuracy of 105 .
(b) Does Gauss-Seidel Method accelerate convergence?

8
Numerical Analysis MATH351/352

Solution
(a)
i x y g1(x,y) g2(x,y) Tol
0 0 0 0.8 0.8
1 0.8 0.8 0.928 0.9312 0.8
2 0.928 0.9312 0.972832 0.97327 0.1312
3 0.972832 0.97327 0.989366 0.989435 0.044832
4 0.989366 0.989435 0.995783 0.995794 0.016534
5 0.995783 0.995794 0.998319 0.998321 0.006417
6 0.998319 0.998321 0.999328 0.999329 0.002536
7 0.999328 0.999329 0.999732 0.999732 0.00101
8 0.999732 0.999732 0.999893 0.999893 0.000403
9 0.999893 0.999893 0.999957 0.999957 0.000161
10 0.999957 0.999957 0.999983 0.999983 6.44E-05
11 0.999983 0.999983 0.999993 0.999993 2.58E-05
12 0.999993 0.999993 0.999997 0.999997 1.03E-05
13 0.999997 0.999997 0.999999 0.999999 4.12E-06
14 0.999999 0.999999 1 1 1.65E-06

(b)
i x y g1(x,y) g2(x,y) Tol
0 0 0 0.8 0.88
1 0.8 0.88 0.94144 0.967049 0.88
2 0.94144 0.967049 0.982149 0.990064 0.14144
3 0.982149 0.990064 0.994484 0.99693 0.040709
4 0.994484 0.99693 0.998287 0.999045 0.012335
5 0.998287 0.999045 0.999467 0.999703 0.003803
6 0.999467 0.999703 0.999834 0.999907 0.00118
7 0.999834 0.999907 0.999948 0.999971 0.000367
8 0.999948 0.999971 0.999984 0.999991 0.000114
9 0.999984 0.999991 0.999995 0.999997 3.56E-05
10 0.999995 0.999997 0.999998 0.999999 1.11E-05
11 0.999998 0.999999 1 1 3.46E-06

(c)
From (a) and (b) it is seen that Gauss-Seidel Method accelerate convergence.
2. Convert the nonlinear system
3x  cos( yz )  12  0
x 2  81 y  101   sin z  1.06  0
2

e xy  20 z  1033  0
to a fixed point problem and use both functional iteration and the Gauss-Seidel
variant of functional iteration to approximate the root to within 105 in the l
norm, starting the initial estimate x0  y0  0.1 and z0  0.1 . [Note the exact
 12 , 0,  6 
T
root is ]

9
Numerical Analysis Non-Linear Iterative Methods

Solution
cos( yz )  12 x 2  sin z  1.06 1 e xy  1033
x , y  , z
3 81 10 20
For k  1, 2, , the functional iteration and the Gauss-Seidel variant of functional
iteration are given as
cos( y ( k 1) z ( k 1) )  12
x(k ) 
3
( x )  sin z ( k 1)  1.06 1
( k 1) 2
y(k )   and
81 10
( k 1)
e  x y ( k 1)  103 3
z (k )

20
( k 1) ( k 1)
cos( y z )  12
x(k ) 
3
( x )  sin z ( k 1)  1.06 1
(k ) 2
y(k )   respectively for the above fixed point
81 10
e x y  103 3
(k ) (k )

z (k )

20
problem.
Using the functional iteration we have the following table:

i x y z g1(x,y,z) g2(x,y,z) g3(x,y,z) Tol

0 0.1 0.1 -0.1 0.499983 0.009441 -0.5231013
1 0.499983 0.009441 -0.523101 0.499996 2.56E-05 -0.5233633 0.399983
2 0.499996 2.56E-05 -0.523363 0.5 1.23E-05 -0.5235981 0.009154
3 0.5 1.23E-05 -0.523598 0.5 3.42E-08 -0.5235985 4.07E-06
4 0.5 3.42E-08 -0.523598 0.5 1.65E-08 -0.5235988 1.2E-05
5 0.5 1.65E-08 -0.523599 0.5 4.57E-11 -0.5235988 6.95E-12
6 0.5 4.57E-11 -0.523599 0.5 2.2E-11 -0.5235988 1.6E-08
7 0.5 2.2E-11 -0.523599 0.5 6.1E-14 -0.5235988 0
8 0.5 6.1E-14 -0.523599 0.5 2.94E-14 -0.5235988 2.14E-11
9 0.5 2.94E-14 -0.523599 0.5 0 -0.5235988 0
10 0.5 0 -0.523599 0.5 0 -0.5235988 2.87E-14

Using the Gauss-Seidel variant of functional iteration we have the following

table:

i x y z g1(x,y,z) g2(x,y,z) g3(x,y,z) Tol

0 0.1 0.1 -0.1 0.499983 0.02223 -0.5230461
1 0.499983 0.02223 -0.523046 0.499977 2.82E-05 -0.5235981 0.399983
2 0.499977 2.82E-05 -0.523598 0.5 3.76E-08 -0.5235988 0.02165
3 0.5 3.76E-08 -0.523599 0.5 5.03E-11 -0.5235988 2.74E-05
4 0.5 5.03E-11 -0.523599 0.5 6.72E-14 -0.5235988 3.66E-08
5 0.5 6.72E-14 -0.523599 0.5 0 -0.5235988 4.9E-11
6 0.5 0 -0.523599 0.5 0 -0.5235988 6.55E-14

10
Numerical Analysis MATH351/352

3. Starting with the initial guess x0  y0  1.0 , use fixed point (functional iteration
to approximate the solution to the system
2 x2  y 2  4.32, x2  y 2  0
by performing 5 iterations.
4. Consider the nonlinear system
2 x  xy  1  0
2 y  xy  1  0
which has a unique root x  (1, 1)T . Starting with the initial estimate x0  y0  0
, compare the methods of functional iteration, Gauss-Seidel Newton when
approximating the root of this system (perform 5 iterations in each case).
5. Use Newton’s Method to approximate the solution of the nonlinear system
x 2  2 x  y  12  0
x2  4 y 2  4  0
starting with the initial estimate  x0 , y0    2, 14  and computing 3 iterations.
Solution
Using the Newton’s iterative method
x( k )  x( k 1)  J 1 ( x( k 1) ) F ( x( k 1) ), for k  1, with x(0)   x0 , y0    2, 14  , we
have
1
 2   2 1  0.25 
x (1)  x (0)  J 1 ( x (0) ) F ( x(0) )      
 0.25   4 2   0.25 
 2  1  2 1  0.25   2  1  0.75  1.90625 
        
 0.25  8  4 2  0.25   0.25  8  0.50   0.3125 
1
1 1.90625   1.8125 1   0.00879 
x  x  J ( x )F (x )  
(2) (1) (1) (1)
   
 0.3125   3.8125 2.5   1.70312 
1.90625  1  2.5 1  0.00879   2.10773 
    3.8125 1.8125  1.70312    0.68232 
 0.3125  8.34375     
6. Use Newton’s Method to approximate the two solutions of the nonlinear system
ye x  2, x2  y 2  4
by computing 2 iterations for each of the given initial estimates
(a)  x0 , y0    0.6, 3.7 
(b)  x0 , y0    1.9, 0.4
7. Use Newton’s Method to approximate the solution of the nonlinear system
x 2  y 2  0.6 y  0.16  0
x 2  y 2  x  1.6 y  0
by computing 3 iterations with the initial estimate of  x0 , y0    0.6,0.25 .
Using the more accurate initial estimate of  x0 , y0    0.3,0.1 , repeat the
process using the modified Newton’s method whereby the Jacobian is evaluated
and held constant for subsequent iterations. Compare the two results.

11