Introduction to Numerical Analysis

- Sources of Errors.

48.9
Prof. Samir Abdelmalek
(NSNN)

October 07, 2024

NSNN
NSNN 2024 111
Outline
− Definitions
− Sources of Error in Applying NA
− Errors in Implementation of NA
− Loss of Significance

NSNN 2024 222

Definitions
− There are two kinds of numbers exact and approximate numbers.
− Exact numbers are 1, 2, …, ½, 2 ,𝜋 ,𝑒 ,etc

− Approximate numbers are those that represent the numbers to a certain

degree of accuracy.

− Thus, an approximate value of 𝜋 is 3.1415, or if we desire a better

approxiamtion, it is 3,141659265. But we cant write the exact value of 𝜋.

NSNN 2024 333

Definitions
− The digits are used to express a number are called significant digits or
significant figures.

− The number 3.1416, 0.66667 and 4.0687 contain 5 significant digits each.
− The number 0.00023 has only 2 significant digits (2 and 3), since the zeros
serve only to fixe the position of the decimal point.

− Similarly, the numbers 0.00145, 0.000145 and 0.0000145 (all have 3

significant digits).

NSNN 2024 444

Definitions
− To round-off a number to 𝑛 significant digits each, discard all digits to the
right of the 𝑛th digit, and if this discarded number is :

− less than half a unit in the 𝑛th place, leave the 𝑛th digit unaltered.
− greater than half a unit in the 𝑛th place, increase the 𝑛th digit by unity.
− Exactly half a unit in the 𝑛th place, increase the 𝑛th digit by unity if it is odd;
otherwise, leave it unchanged.

− The number thus rounded-off is said to be correct to 𝑛 significant figures.

NSNN 2024 555

Definitions
Exemple. The numbers given below are rounded-off to 4 significant
figures

1.6583 to 1.658
30.0567 to 30.06
0.859378 to 0.8594
3.14159 to 3.142

NSNN 2024 666

Sources of Error in Applying Numerical Methods
Real World Problem

Revision of model Modeling

if necessary
Error
Mathematical
model
Error in Numerical
Computation
Exact Solution Numerical Solution

Interpretation
of Solution

Implementation of
Model

NSNN 2024 777/40

Sources of Error in Applying Numerical Methods
● In fact, the error is a multifaceted problem, but it mainly arises during
two stages:

1. Error occurred during the mathematical modeling of the real world

problem (arises because a mathematicl model is not perfect
reflection of reality),

2. and error when we solve the mathematical model numerically (arises

because our numericl methods are, by nature imperfect
approximations).

NSNN 2024 888

Sources of Error in Applying Numerical Methods
● Actually, we were fortunate in that case because the availability of
an analytical solution allowed us to compute the error exactly.

≫≫ For many applied engineering problems, we cannot obtain

analytical solutions.
≫≫ We cannot compute exactly the errors associated with our
numerical methods.

⇒ We must settle for approximations or estimates of the errors.

NSNN 2024 999

Sources of Error in Applying Numerical Methods

In professional practice, errors can be costly and

sometimes catastrophic. If a structure or device fails,
lives can be lost.

NSNN 2024 110

10 0
Sources of Error in Applying Numerical Methods
1. Absolute, Relative and Percentage Errors :

− The difference between the exact value and an approximate value of a

quantity is called error in the measurement. Its absolute value is called
absolute error.

− Let 𝒙 be the exact value and ෥

𝒙 be an approximate value of a given
quantity; then the absolute error (𝑬𝒂 ) is given by :

෥
𝑬𝒂 = 𝒙 − 𝒙

NSNN 2024 111

11 1
Sources of Error in Applying Numerical Methods
1. Absolute, Relative and Percentage Errors :

− The relative (𝑬𝒓 ) and percentage (𝑬𝒑 ) errors are defined as follows :

𝒙−෥𝒙 𝒙−෥𝒙
𝑬𝒓 = & 𝑬𝒑 = 𝟏𝟎𝟎𝑬𝒓 = 𝟏𝟎𝟎
𝒙 𝒙

− Let there exist a number 𝜀 > 0, such that 𝒙 − ෥

𝒙 ≤ 𝜺. Then 𝜺 is an
upper limit of the absolute error and measures the absolute accuracy.

NSNN 2024 112

12 2
Sources of Error in Applying Numerical Methods
1. Absolute, Relative and Percentage Errors :

− when 𝒙 ≈ 𝟏 ,there is not much difference 𝒙 ෥

𝒙 Absolute Relative
between absolute and relative error measures. error error
− when 𝒙 ≥ 𝟏 ,the relative error is more
meaningful.

− When the approximated value is small in

magnitude, things are a little more delicate,
and here is where relative errors may not be so
meaningful.

NSNN 2024 113

13 3
Sources of Error in Applying Numerical Methods
Exemple. An approximation to the value of 𝜋 is given by 22/7, while its
true value in 8 decimal digits is 3.1415926. Calculate the absolute, relative
and percentage errors in the approximation.
22
Answer. Exact value = 𝑥 = 3.1415926. Appro value = 𝑥෤ = 7 = 3.1428571

𝑬𝒂 = 𝑥 − 𝑥෤ = −0.0012645 ⟹ 𝑬𝒂 = 𝟎. 𝟎𝟎𝟏𝟐𝟔𝟒𝟓
𝑥 − 𝑥෤ 0.0012645
𝑬𝒓 = = ⟹ 𝑬𝒓 = 𝟎. 𝟎𝟎𝟎𝟒𝟎𝟐𝟓𝟎𝟐
𝑥 3.1415926
𝑥 − 𝑥෤
𝑬𝒑 = 100𝐸𝑟 = 100 ⟹ 𝑬𝒑 = 𝟎. 𝟎𝟒𝟎𝟐𝟓𝟎𝟐%
𝑥

NSNN 2024 114

14 4
Sources of Error in Applying Numerical Methods
Exemple. The derivative, 𝑓 ′ of a function 𝑓(𝑥) can be approximated by the
equation:
𝑓 𝑥+ℎ −𝑓 𝑥
𝑓ሚ ′ 𝑥 ≈ ℎ
.
If 𝑓 𝑥 = 7 𝑒 0.5 𝑥 and ℎ = 0.3:

a) Find the approximate value of 𝑓ሚ ′ 2

b) True value of 𝑓 ′ 2
c) Find the absolute error of part (a)

NSNN 2024 /40

115
15 5
Sources of Error in Applying Numerical Methods
Answer.
𝑓 2+0.3 −𝑓 2
a) For 𝑥 = 2 and ℎ = 0.3 : 𝑓ሚ ′ 2 ≈ 0.3
= 10.263

b) The exact value of 𝑓 ′ 2 can be found by using our knowledge of

differential calculus:
𝑓 𝑥 = 7 𝑒 0.5 𝑥 ⟹ 𝑓 ′ 𝑥 = 3.5 𝑒 0.5 𝑥 , so 𝑓 ′ 2 = 9.5140,

c) The absolute error : 𝑬𝒂 = 𝑓 ′ 2 − 𝑓ሚ ′ 2 = 0.722

NSNN 2024 /40

116
16 6
Sources of Error in Applying Numerical Methods
2. Errors in Modeling of Real World Problems:

− The errors in modeling are not directly connected with numerical

techniques, but they have a profound impact on the success of a
model. Thus, before implementation of a numerical method to the
mathematical model, we must have knowledge of the following errors

1. Modeling Error
2. Error in Original Data
3. Blunder

NSNN 2024 117

17 7
Sources of Error in Applying Numerical Methods
2. Errors in Modeling of Real World Problems:

− Modeling Error: is the discrepancy between the real-world system

and its mathematical representation or model. It arises from the
simplifications and assumptions made during the model formulation.
− Simplifications: Assumptions like linear behavior when the real system is nonlinear.
− Idealizations : Neglecting minor factors or interactions for simplicity.
− Inaccurate Representations: Using models that do not fully capture the complexity
of the real system.

NSNN 2024 118

18 8
Sources of Error in Applying Numerical Methods
2. Errors in Modeling of Real World Problems:

− Error in Original Data (Inherent Error): refers to inaccuracies in

the data used to build or validate the model. This includes errors in
measurements or estimates of input parameters.
− Measurement Errors: Inaccuracies in data collection due to instrument limitations
or human error.
− Estimation Errors: Approximation or estimation errors when actual measurements
are not available.
− Data Quality: Variability or uncertainty in the data used for model calibration and
validation.

NSNN 2024 119

19 9
Sources of Error in Applying Numerical Methods
2. Errors in Modeling of Real World Problems:

− Blunder Error (human errors), are mistakes made during the

modeling process, such as data entry errors or incorrect application of
methods.

− Data Entry Mistakes: Errors in inputting data into the model or software.
− Misapplication of Techniques: Incorrect use of modeling techniques or methods due
to misunderstanding or oversight.
− Calculation Errors: Errors made during manual or computational calculations.

NSNN 2024 220

20 0
Errors in Implementation of Numerical Methods
● In any numerical computation, we come across following types of
errors, which are due to the way that computers store numbers and do
arithmetic.
1. Round-off Error
2. Truncation Error
3. Loss of Significance: Condition and Stability.

≫≫ But, round-off and truncation errors can occur in any numerical

computation.

NSNN 2024 221

21 1
Errors in Implementation of Numerical Methods
● Round-off Errors : Arises from inexact representations of numbers in
computers.
● Example. Use three-digit rounding arithmetic to perform the following
2 9
calculations : 9 7 . Compute the absolute error and relative error with the
exact value determined to at least five digits.
● Ans.
Exact 3 Digit Rounding (Approx) Absolute Error Relative Error
2 0.222 × 1.29 = 0. 𝟐𝟖𝟔38 = 0.286 × 10−4 0.001 = 10−3
= 0. 𝟐𝟐𝟐22 …
9 0.286
9
= 𝟏. 𝟐𝟖57 …
7
2 9
× = 0.285714 …
9 7

NSNN 2024 222

22 2
Errors in Implementation of Numerical Methods
● Truncation errors (TE) results from using an approximation in place of
an exact mathematical procedure.

𝒅𝒗 ∆𝒗 𝒗 𝒕𝒊+𝟏 − 𝒗 𝒕𝒊
≅ =
𝒅𝒕 ∆𝒕 𝒕𝒊+𝟏 − 𝒕𝒊
derivative Approximated the derivative
of velocity of velocity finite-divided-difference
equation

● A TE error was introduced into the numerical solution because the

difference equation only approximates the true value of the derivative.

NSNN 2024 223

23 3
Errors in Implementation of Numerical Methods
● Truncated error committed in series approximation cab be evaluated by using
Taylor’s series.
● If 𝑥𝑖 and 𝑥𝑖+1 are two successive values of 𝑥, the we have :

ℎ 2 ℎ 𝑛
𝑓 𝑥𝑖+1 = 𝑓 𝑥𝑖 + ℎ 𝑓 ′ 𝑥𝑖 + 𝑓 ′′ 𝑥𝑖 + ⋯ + 𝑓 𝑛 𝑥𝑖 + 𝑅𝑛+1 𝑥𝑖+1
2! 𝑛!
where ℎ = 𝑥𝑖+1 − 𝑥𝑖

ℎ𝑛+1
𝑅𝑛+1 𝑥𝑖+1 = 𝑓 (𝑛+1) 𝜉 , 𝑥𝑖 < 𝜉 < 𝑥𝑖+1
𝑛+1 !
● 𝑅𝑛+1 is the remainder term.

NSNN 2024 224

24 4
Errors in Implementation of Numerical Methods
Exemple. Evaluate 𝑓 1 using taylor’s series of 𝑓 𝑥 , where
𝑓 𝑥 = 𝑥 3 − 3𝑥 2 + 5𝑥 − 1.
− Its is easly seen that 𝑓 1 = −7 but it will be instructive to see how the
Taylor series approximations of order 0 to 3 improve the accuracy of
𝑓 1 graduallly.

− Let ℎ = 1, 𝑥𝑖 = 0 and 𝑥𝑖+1 = 1. We then require 𝑓 𝑥𝑖+1 . The derivatives of 𝑓 𝑥 are

given by :
𝑓 ′ 𝑥 = 3𝑥 2 − 6𝑥 + 5;𝑓 ′′ 𝑥 = 6𝑥 − 6 and 𝑓 ′′′ 𝑥 = 6.

− 𝑓 ′𝑣 𝑥 and higher derivatives being all zero. Hence :

𝑓 ′ 𝑥𝑖 = 𝑓 ′ 0 = 5; 𝑓 ′′ 𝑥𝑖 = 𝑓 ′′ 0 = −6 and 𝑓 ′′′ 0 = 6.

NSNN 2024 /40

225
25 5
Errors in Implementation of Numerical Methods
Exemple. Also 𝑓 ′ 𝑥𝑖 = 𝑓 0 = −10

− Taylor’s series gives :

′ ℎ2 ′′ ℎ3 ′′′
𝑓 𝑥𝑖+1 = 𝑓 𝑥𝑖 + ℎ 𝑓 𝑥𝑖 + 𝑓
𝑥𝑖 +
2! 3!
𝑓 𝑥𝑖 .
− The zero-order approximation is given by :
𝑓 𝑥𝑖+1 = 𝑓 𝑥𝑖 + 𝑂(ℎ)
− And, therefore
𝑓 1 = 𝑓 0 + 𝑂 ℎ ≈ −10

− The error in which is -7+10, i.e., 3 units

NSNN 2024 /40

226
26 6
Errors in Implementation of Numerical Methods
− For the 1st approximation, we have :
𝑓 𝑥𝑖+1 = 𝑓 𝑥𝑖 + ℎ 𝑓 ′ 𝑥𝑖 + 𝑂(ℎ2 )

− And, therefore,

𝑓 1 = −10 + 5 + 𝑂(ℎ2 ) ≈ −10

− The error in which is -7+5, i.e., -2 units.

NSNN 2024 /40

227
27 7
Errors in Implementation of Numerical Methods
− Again, the 2nd order Taylor approximation is given :
′ ℎ2 ′′
𝑓 𝑥𝑖+1 = 𝑓 𝑥𝑖 + ℎ 𝑓 𝑥𝑖 + 2!
𝑓 𝑥𝑖 + 𝑂(ℎ3 ).

− And, therefore,

1
𝑓 1 = −10 + 5 + −6 + 𝑂(ℎ3 ) ≈ −8
2

− In which the error is -7+8, i.e., 1 unit.

NSNN 2024 /40

228
28 8
Errors in Implementation of Numerical Methods
− The 3rd Taylor series approximation is :
′
ℎ2 ′′ ℎ3 ′′′
𝑓 𝑥𝑖+1 = 𝑓 𝑥𝑖 + ℎ 𝑓 𝑥𝑖 + 𝑓 𝑥𝑖 + 𝑓 𝑥𝑖 .
2! 3!

− And, therefore,
′
ℎ2 ′′ ℎ3 ′′′
𝑓 1 = 𝑓 0 + ℎ 𝑓 𝑥0 + 𝑓 𝑥0 + 𝑓 𝑥0 ≈ −7
2! 3!

− Which is the exact value of 𝑓 1 .

NSNN 2024 /40

229
29 9
Loss of Significance: Condition and Stability
● The condition is used to describe the sensitivity of the function and stability is used to
describe the sensitivity of the process.
● Condition: The sensitivity of the function 𝒇 𝒙 with the change in the argument 𝒙 is
described by the condition number (CN). It is a relative change in the function 𝒇 𝒙 for per
unit relative change in 𝒙. CN of the function 𝒇 𝒙 at any point 𝒙 is given by :
𝒇 𝒙 −𝒇 ෥𝒙
𝒇(𝒙) 𝒇 𝒙 −𝒇 ෥
𝒙 𝒙
𝑪𝑵 = =
𝒙−෥𝒙 𝒙−෥
𝒙 𝒇 𝒙
𝒙
For small change in 𝒙, Lagrange mean value theorem gives :

෥
𝒇 𝒙 −𝒇 𝒙 𝒙𝒇′ (𝒙)
≈ 𝒇′ (𝒙) ⟹ 𝑪𝑵 =
𝒙−෥
𝒙 𝒇(𝒙)

NSNN 2024 330

30 0
Loss of Significance: Condition and Stability
● If CN ≤ 𝟏, then the function 𝒇 𝒙 is said to be well-conditioned. Otherwise,
it is said to be ill-conditioned. The function with large CN is more ill-
conditioned as compared to the function with small CN.

● Note: Let us consider a mathematical model of any system, in which

variable 𝒙 gives input, and output is the function 𝒇 𝒙 . If a small relative
change in 𝒙 (input) produces a large relative change in output 𝒇 𝒙 , then
the system is said to be a sensitive system as fluctuation in input may break
the system. Mathematically, if CN is large, then the function is more
sensitive to changes and function is ill-conditioned.

NSNN 2024 331

31 1
Loss of Significance: Condition and Stability
● If CN ≤ 𝟏, then the function 𝒇 𝒙 is said to be well-conditioned. Otherwise,
it is said to be ill-conditioned. The function with large CN is more ill-
conditioned as compared to the function with small CN.

● Example. Find the CNs of the functions 𝑓 𝑥 = 𝑥 and 𝑥 3

−1
1
𝑥𝑓′ (𝑥) 𝑥 2𝑥 2 1
● Ans. CN of the function 𝑓 𝑥 = 𝑥= = =2
𝑓(𝑥) 𝑥
𝑥𝑓′ (𝑥) 𝑥 3𝑥 2
CN of the function 𝑓 𝑥 = 𝑥 3 = = =3
𝑓(𝑥) 𝑥3

≫≫ CN of the function 𝑥 is less than 1, so the function 𝑥 is well conditioned.

The function 𝑥 3 is an ill-conditioned function as CN > 1.

NSNN 2024 332

32 2
Loss of Significance: Condition and Stability
● Stability of the Process:
1
● Example. Compute the function 𝑓 𝑥 = 𝑥 + 1 − 𝑥 = 𝑥+1+ 𝑥 by using
both the formulae at point 𝑥 = 11111. Use six significant digits floating
point rounding arithmetic.
1
● Ans. We have two formulas 𝑓 𝑥 = 𝑥 + 1 − 𝑥 and 𝑓 𝑥 = to
𝑥+1+ 𝑥
compute the function 𝑓 𝑥 at point 𝑥 = 11111.
○ Process-I: 𝑓 𝑥 = 𝑥 + 1 − 𝑥 = 105.413 − 105.409 = 0.004
1 1
○ Process-II: 𝑓 𝑥 = = = 0.00474334
𝑥+1+ 𝑥 105.413+105.409

≫≫ Note that, the exact result up to 6 significant digits is 0.00474330

NSNN 2024 333

33 3
NSNN 2024 334
34 4