FALL 2020

CE 305
NUMERICAL METHODS FOR ENGINEERS

MIDDLE EAST TECHNICAL UNIVERSITY

CIVIL ENGINEERING DEPARTMENT
These lecture notes have been prepared by multiple lecturers between 2007 and 2020 for the undergraduate
course CE 305 “Numerical Methods for Engineers” in METU Department of Civil Engineering. We acknowledge all
of these efforts of contributors Yalın Arıcı, Aysegul Askan, Alp Caner, Serdar Göktepe, Engin Karaesmen,
Shaghayegh Karimzadeh, Elçin Kentel, Mete Koken, Onur Pekcan, Kartal Toker and Ahmet Cevdet Yalçıner. The
initial typing of the original handwritings was performed by Eren Yağmur. The final typesetting was prepared by
Shaghayegh Karimzadeh. All rights are reserved by METU Department of Civil Engineering. Copying or distribution
without permission is forbidden.
 METU - Department of Civil Engineering

Course Code and Title: CE 305 Numerical Methods for Engineers (3-1) 3

CHAPTER 7: NUMERICAL DIFFERENTIATION

7.1. FINITE DIFFERENCE APPROXIMATION FOR 1ST DERIVATIVES

7.1.1. Forward, backward and central difference formulae

Taylor Series is typically simplified by defining a step size h

Taylor Series: Step size, ℎ = 𝑥𝑖+1 − 𝑥𝑖

Simplified Taylor series can be written in the form given below.

𝑓 ′′ (𝑥𝑖 ) 𝑓 (3) (𝑥𝑖 ) 𝑓 (𝑛) (𝑥𝑖 )

𝑓(𝑥𝑖+1 ) = 𝑓(𝑥𝑖 ) + 𝑓 ′ (𝑥𝑖 )ℎ + ℎ2 + ℎ3 + ⋯ + ℎ𝑛 + 𝑅𝑛
2! 3! 𝑛!

The last term is an nth order approximation can be written as

𝑓 (𝑛+1) (𝜉)
𝑅𝑛 = ℎ(𝑛+1) 𝑥𝑖 < 𝜉 < 𝑥𝑖+1
(𝑛+1)!

Suppose that we truncated the Taylor series after the zeroth-order term to yield:

𝑓(𝑥𝑖+1 ) ≅ 𝑓(𝑥𝑖 )

f(x)

𝑅0

x
𝑥𝑖 𝑥𝑖+1

𝑓 ′′ (𝑥𝑖 )
𝑅0 = 𝑓 ′ (𝑥𝑖 )ℎ + ℎ2 + ⋯
2!

When h is very small, each consecutive term gets smaller. 𝑅0 ≅ 𝑓 ′ (𝑥𝑖 )ℎ

7-1
 METU - Department of Civil Engineering

Course Code and Title: CE 305 Numerical Methods for Engineers (3-1) 3

The DERIVATIVE MEAN VALUE THEOREM states that if a function f(x) and its first derivative are
continuous over an interval form 𝑥𝑖 to 𝑥𝑖+1 , then there exists at least one point (𝜉) on the function
that has a slope, designated by 𝑓 ′ (𝜉) which is parallel to the line joining 𝑓(𝑥𝑖 ) and 𝑓(𝑥𝑖+1 ).

f(x)

𝑓 ′ (𝜉)

𝑅0
Slope=𝑅0 /ℎ

𝑓(𝑥𝑖 )

x
𝑥𝑖 𝜉 𝑥𝑖+1

𝑓 ′ (𝜉 ) = 𝑅0 /ℎ

𝑅0 = 𝑓 ′ (𝜉 )ℎ

𝑓 ′′ (𝜉)
Similarly, first order version can be derived as 𝑅1 = ℎ2 .
2!

**Using the Taylor Series to estimate the truncation error:

Taylor Series expansion:

𝑣 ′′ (𝑡𝑖 ) 𝑣 (𝑛) (𝑡𝑖 )

𝑣(𝑡𝑖+1 ) = 𝑣(𝑡𝑖 ) + 𝑣 ′ (𝑡𝑖 )(𝑡𝑖+1 − 𝑡𝑖 ) + (𝑡𝑖+1 − 𝑡𝑖 )2 + ⋯ + (𝑡𝑖+1 − 𝑡𝑖 )𝑛 + 𝑅𝑛
2! 𝑛!

First-Order approximation:

𝑣(𝑡𝑖+1 ) = 𝑣(𝑡𝑖 ) + 𝑣 ′ (𝑡𝑖 )(𝑡𝑖+1 − 𝑡𝑖 ) + 𝑅1 Here please note that 𝑅1 ≅ 𝑂(Δ𝑡 2 )

𝑣(𝑡𝑖+1 )−𝑣(𝑡𝑖 ) 𝑅
𝑣 ′ (𝑡𝑖 ) = (𝑡𝑖+1 −𝑡𝑖 )
− (𝑡 1 )
𝑖+1 −𝑡𝑖

𝑂(Δ𝑡 2 )
𝐸≅( ) = 𝑂(Δ𝑡)
Δ𝑡

7-2
 METU - Department of Civil Engineering

Course Code and Title: CE 305 Numerical Methods for Engineers (3-1) 3

Numerical Differentiation:

Let f be a function of a single variable. Taylor Series expansion around 𝑥0 is

𝑓 ′′ (𝑥0 ) 𝑓 ′′′ (𝑥0 )

𝑓(𝑥0 + Δ𝑥 ) = 𝑓 (𝑥0 ) + 𝑓 ′ (𝑥0 )Δ𝑥 + (Δ𝑥)2 + (Δ𝑥)3 + ⋯ (1)
2! 3!

𝑓 ′′ (𝑥0 ) 𝑓 ′′′ (𝑥0 )

𝑓(𝑥0 − Δ𝑥 ) = 𝑓 (𝑥0 ) − 𝑓 ′ (𝑥0 )Δ𝑥 + (Δ𝑥 )2 − (Δ𝑥)3 + ⋯ (2)
2! 3!

From equation (1) 𝑓(𝑥0 + Δ𝑥 ) ≅ 𝑓(𝑥0 ) + 𝑓 ′ (𝑥0 )Δ𝑥

𝑓(𝑥0 +Δ𝑥)−𝑓(𝑥0 )
𝑓 ′ (𝑥0 ) = (Forward Difference formula)
Δ𝑥

𝐸 ≅ 𝑂(Δ𝑥)

From equation (2) 𝑓(𝑥0 − Δ𝑥 ) ≅ 𝑓(𝑥0 ) − 𝑓 ′ (𝑥0 )Δ𝑥

𝑓(𝑥0 )−𝑓(𝑥0 −Δ𝑥)

𝑓 ′ (𝑥0 ) = (Backward Difference formula)
Δ𝑥

𝐸 ≅ 𝑂(Δ𝑥)

If the 2nd equation is subtracted from the 1st equation:

𝑓 (𝑥0 + Δ𝑥 ) − 𝑓(𝑥0 − Δ𝑥 ) ≅ 2Δ𝑥 𝑓 ′ (𝑥0 ) then

𝑓(𝑥0 +Δ𝑥)−𝑓(𝑥0 −Δ𝑥)

𝑓 ′ (𝑥0 ) ≅ (Central/Centered Difference formula)
2Δ𝑥

𝑂(Δ𝑥 3 )
𝐸≅ = 𝑂(Δ𝑥 2 ) More accurate
𝑂(Δ𝑥)

Ex.

t v(t)
0 1000 only forward
0.1 980 Backward, forward, central
0.2 950 Backward, forward, central
0.3 900 only Backward

7-3
 METU - Department of Civil Engineering

Course Code and Title: CE 305 Numerical Methods for Engineers (3-1) 3

𝜋 𝜋 𝜋
Ex: 𝑓(𝑥 ) = 𝑠𝑖𝑛𝑥 𝑥0 = of course 𝑓 ′ ( ) = 0.707106 𝑓 ′ ( ) =? h = 0.01
4 4 4

Using forward difference formula with error of 𝑂(ℎ) : h=0.01 we get

𝜋 𝜋 𝜋 𝜋
𝜋 𝑓( +0.01)−𝑓( ) 𝑠𝑖𝑛( +0.01)−𝑠𝑖𝑛( )
𝑓′ ( ) = 4 4
= 4 4
= 0.7035594
4 0.01 0.01

Using backward difference formula with error of 𝑂(ℎ) :

𝜋 𝜋 𝜋 𝜋
𝜋 𝑓( )−𝑓( −0.01) 𝑠𝑖𝑛( )−𝑠𝑖𝑛( −0.01)
𝑓′ ( ) = 4 4
= 4 4
= 0.7106305
4 0.01 0.01

Using central difference formula:

𝜋 𝜋
𝜋 𝑓( +0.01)−𝑓( −0.01)
𝑓′ ( ) = 4 4
= 0.7070949 (better than forward and backward)
4 2(0.01)

Ex.: Use forward and backward approximations of O(h) and a centered difference approximation of
O(h2) to estimate the first derivative of 𝑓(𝑥 ) = −0.1𝑥 4 − 0.15𝑥 3 − 0.5𝑥 2 − 0.25𝑥 + 1.2 at x=0.5. The
true derivative is 𝑓 ′ (0.5) = −0.9125 calculated based on the derivative function of 𝑓 ′ (𝑥 ) =
−0.4𝑥 3 − 0.45𝑥 2 − 𝑥 − 0.25

In your calculations use a step size of:

a) h=0.5

𝑓(0.5+0.5)−𝑓(0.5) 0.2−0.925
Forward diff. 𝑓 ′ (0.5) = = = −1.45
0.5 0.5

−1.45+0.9125
𝜀𝑡 % = | | × 100 = 58.9%
0.9125

𝑓(0.5)−𝑓(0.5−0.5) 0.925−1.2
Backward diff. 𝑓 ′ (0.5) = = = −0.55
0.5 0.5

𝜀𝑡 % = 39.7%

𝑓(0.5+0.5)−𝑓(0.5−0.5) 0.2−1.2
Centered diff. 𝑓 ′ (0.5) = = = −1.0
2(0.5) 1.0

𝜀𝑡 % = 9.6% (more accurate)

7-4
 METU - Department of Civil Engineering

Course Code and Title: CE 305 Numerical Methods for Engineers (3-1) 3

b) h=0.25

𝑓(0.5+0.25)−𝑓(0.5)
Forward diff. 𝑓 ′ (0.5) = = −1.155
0.25

𝜀𝑡 % = 26.5%

𝑓(0.5)−𝑓(0.5−0.25)
Backward diff. 𝑓 ′ (0.5) = = −0.714
0.25

𝜀𝑡 % = 21.7%

𝑓(0.5+0.25)−𝑓(0.5−0.25)
Centered diff. 𝑓 ′ (0.5) = = −0.934
2(0.25)

𝜀𝑡 % = 2.4%

7.2. FINITE DIFFERENCE APPROXIMATION FOR 2ND DERIVATIVES

Finite Difference Approximation of Higher Derivatives

𝑓 ′′ (𝑥)
1) 𝑓(𝑥 + 2ℎ) = 𝑓(𝑥 ) + 𝑓 ′ (𝑥 )2ℎ + (2h)2 + ⋯
2!
𝑓 ′′ (𝑥)
2) 𝑓(𝑥 + ℎ) = 𝑓(𝑥 ) + 𝑓 ′ (𝑥 )ℎ + (h)2 + ⋯
2!

From Eq. 1 and 2:

𝑓 ′′ (𝑥) 𝑓 ′′ (𝑥)
𝑓(𝑥 + 2ℎ) − 2𝑓(𝑥 + ℎ) = 𝑓(𝑥 ) − 2𝑓(𝑥 ) + (2h)2 − 2 (h)2 + ⋯
2! 2!

𝑓(𝑥+2ℎ)−2𝑓(𝑥+ℎ)+𝑓(𝑥)
Forward 𝑓 ′′ (𝑥 ) = + 𝑂(ℎ)
ℎ2

Similarly,

𝑓(𝑥)−2𝑓(𝑥−ℎ)+𝑓(𝑥−2ℎ)
Backward 𝑓 ′′ (𝑥 ) = + 𝑂(ℎ)
ℎ2

𝑓(𝑥+ℎ)−2𝑓(𝑥)+𝑓(𝑥−ℎ)
Centered 𝑓 ′′ (𝑥 ) = + 𝑂(ℎ2 )
ℎ2

• Please try to derive these backward and centered difference formulae.

• Important Home Exercise: Please derive the numerical difference formulae on Handout 13
by yourself.

7-5
 METU - Department of Civil Engineering

Course Code and Title: CE 305 Numerical Methods for Engineers (3-1) 3

Error Analysis in Numerical Differentiation

1. Truncation error to min. decrease step size or use a higher order approximation.

2. Round-off error to min. increase number of significant figures in the computations.

+

∑ = 𝑡𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑒𝑟𝑖𝑐𝑎𝑙 𝑒𝑟𝑟𝑜𝑟 𝑜𝑓 𝑎𝑝𝑝𝑟𝑜𝑥𝑖𝑚𝑎𝑡𝑖𝑜𝑛

But if we ↓ h too much, truncation error ↓, round-off error ↑ due to higher number of computations
(operations). So there is an optimum value of h which minimizes the total numerical error.

logԐ

Total numerical error

Truncation error

Round-off error

logh
h*: optimum h

(Errors are preferred to be plotted in logarithmic scale, since the changes can be shown slowly.)

7-6