Advanced Numerical Analysis: Numerical

Differentiation and Integration

Bishnu P. Lamichhane ∗
August 18, 2020

∗ University of Newcastle, School of Mathematical & Physical Sciences, Math-

ematics Building - V235, University Drive, Callaghan, NSW 2308, Australia,
[email protected]

1
1 Numerical Differentiation
Recall that the derivative of the function f at x0 is

f (x0 + h) − f (x0 )
f 0 (x0 ) = lim .
h→0 h

This formula gives a way of approximating the derivative of the function f (x) at
x = x0 . We just compute
f (x0 + h) − f (x0 )
h
for a small value of h. However, we do not have any information about the quality of
the approximation. In other words, we want to compute the truncation error. To this
end, let f ∈ C2 [a, b], x0 , x1 ∈ [a, b], and h = x1 − x0 . To get an idea of the truncation
error in approximating f 0 (x0 ) by

f (x0 + h) − f (x0 )
,
h
we consider a Lagrange polynomial of degree 1 for the function f determined by two
points x0 and x1 with its error term

f (x0 )(x − x0 − h) f (x0 + h)(x − x0 ) (x − x0 )(x − x0 − h) 00

f (x) = + + f (ξ(x)),
−h h 2
for some ξ(x) ∈ [a, b]. Differentiation gives
 
0 f (x0 + h) − f (x0 ) (x − x0 )(x − x0 − h) 00
f (x) = + Dx f (ξ(x))
h 2
f (x0 + h) − f (x0 ) 2(x − x0 ) − h 00 (x − x0 )(x − x0 − h)
= + f (ξ(x)) + Dx f 00 (ξ(x)).
h 2 2
If we set x = x0 , we have

f (x0 + h) − f (x0 ) h 00
f 0 (x0 ) = − f (ξ(x0 )).
h 2
Hence  
 0
 f (x0 ) − f (x0 + h) − f (x0 )  ≤ M h,

 h  2
where M is a bound on | f 00 (x)| for x ∈ [a, b]. The formula

f (x0 + h) − f (x0 )
f 0 (x0 ) ≈
h

2
is known as the forward difference formula if h > 0 and backward difference formula
if h < 0.
Let F( f , h)(x) be a finite difference approximation of f (p) (x) using step-size h
at x, where f (p) (x) is the pth derivative of a real-valued function f . We assume that
f is sufficiently smooth function. We call the approximation F( f , h) to be of oder n
accurate if
|F( f , h)(x) − f (p) (x)| ≤ O(hn ).

1.1 Central Difference Formula for the First and Second Deriva-
tives
The central difference formula to approximate f 0 (x0 ) can also be derived as follows.
Let us expand f in terms of a second order Taylor polynomial and evaluate at x0 + h
and x0 − h:
h2 00 h3
f (x0 + h) = f (x0 ) + h f 0 (x0 ) + f (x0 ) + f (3) (ξ),
2! 6
and
h2 00 h3
f (x0 − h) = f (x0 ) − h f 0 (x0 ) +
f (x0 ) − f (3) (η),
2! 6
where x0 − h < η < x0 < ξ < x0 + h. Subtracting the second equation from the first
one, we obtain
h3 (3) h3
f (x0 + h) − f (x0 − h) = 2h f 0 (x0 ) +f (ξ) + f (3) (η).
6 6
h i
Assume that f (3) is continuous on [x0 − h, x0 + h]. Since 12 f (3) (ξ) + f (3) (η) is
between f (3) (ξ) and f (3) (η), the Intermediate Value Theorem implies that a number
κ exists between η and ξ and hence in (x0 − h, x0 + h) with
1 h (3) i
f (3) (κ) = f (ξ) + f (3) (η) .
2
Hence we have the central difference formula:
1 h2
f 0 (x0 ) = [ f (x0 + h) − f (x0 − h)] − f (3) (κ),
2h 6
where κ lies between x0 − h and x0 + h.
We present a method to approximate the second derivative of a function by using
its tabulated values. For this purpose, we expand the function in a third order Taylor
polynomial about a point x0 and evaluate at x0 + h and x0 − h. Then
h2 00 h3 h4
f (x0 + h) = f (x0 ) + h f 0 (x0 ) + f (x0 ) + f 000 (x0 ) + f (4) (ξ),
2! 3! 24

3
and
h2 00 h3 h4
f (x0 ) − f 000 (x0 ) + f (4) (η),
f (x0 − h) = f (x0 ) − h f 0 (x0 ) +
2! 3! 24
where x0 − h < η < x0 < ξ < x0 + h. We now cancel terms involving f 0 (x0 ) and
f 000 (x0 ) by adding these equations:

h4 h (4) i
f (x0 + h) + f (x0 − h) = 2 f (x0 ) + f 00 (x0 )h2 + f (ξ) + f (4) (η) .
24
Solving this equation for f 00 (x0 ) yields

00 1 h2 h (4) (4)
i
f (x0 ) = 2 [ f (x0 + h) − 2 f (x0 ) + f (x0 − h)] − f (ξ) + f (η) .
h 24
h i
Assume that f (4) is continuous on [x0 − h, x0 + h]. Since 12 f (4) (ξ) + f (4) (η) is
between f (4) (ξ) and f (4) (η), the Intermediate Value Theorem implies that a number
κ exists between η and ξ and hence in (x0 − h, x0 + h) with
1 h (4) i
f (4) (κ) = f (ξ) + f (4) (η) .
2
Hence the formula for f 00 (x0 ) is

1 h2 (4)
f 00 (x0 ) = [ f (x0 + h) − 2 f (x0 ) + f (x0 − h)] − f (κ)
h2 12
for some κ ∈ (x0 − h, x0 + h). We have to assume the continuity of f (4) on (x0 −
h, x0 + h) to get its boundedness there.

1.2 Two-Dimensional Case

The central difference formula is extended to the two-dimensional case by using the
tensor product structure. Let Ω ⊂ R2 and u ∈ C2 (Ω). Then
∂u u(x0 + h, y0 ) − u(x0 − h, y0 ) ∂u u(x0 , y0 + h) − u(x0 , y0 − h)
(x0 , y0 ) ≈ and (x0 , y0 ) ≈ .
∂x 2h ∂y 2h
Similarly, for the second derivatives we have

∂2 u u(x0 + h, y0 ) − 2u(x0 , y0 ) + u(x0 − h, y0 )

2
(x0 , y0 ) ≈
∂x h2
and
∂2 u u(x0 , y0 + h) − 2u(x0 , y0 ) + u(x0 , y0 − h)
2
(x0 , y0 ) ≈ .
∂y h2

4
2 Numerical Integration
Sometimes we have to compute the definite integral of a function which does not have
an explicit anti-derivative or whose anti-derivative is not
Rb
easy to obtain. The method
of approximating the definite integral of a function a f (x) dx is called numerical
quadrature or simply quadrature. A numerical quadrature uses a sum ∑ni=0 wi f (xi )
to approximate the definite integral ab f (x).
R

2.1 Newton-Cotes Rule

The most commonly used numerical integration methods are called Newton-Cotes
formulas or Newton-Cotes rules. They are a family of formulas for numerical inte-
gration based on interpolating polynomials at equally-spaced points and named after
Isaac Newton and Roger Cotes. These formulas can be applied if the value of the
integrand at equally-spaced points is given. When the value of the integrand at some
equally-spaced points are given, we compute an interpolating polynomial of the func-
tion based on these points and integrate the interpolating polynomial. The accuracy
of the approximated integral depends on the accuracy of the interpolating polynomial
in approximating the given integrand. The basic idea is to construct an interpolation
polynomial pn (x) of the function f (x) at {x0 , · · · , xn } of distinct nodes in [a, b] and
then integrate this interpolant
n
pn (x) = ∑ li (x) f (xi ).
i=0

Thus Z b n Z b
 n
f (x) dx ≈ ∑ li (x) dx f (xi ) = ∑ wi f (xi ),
a i=0 a i=0
where Z b
wi = li (x) dx, i = 0, · · · , n.
a

Let f ∈ Cn+1 [a, b]. Then

f (n+1) (ξ(x))
f (x) = pn (x) + πn+1 (x),
(n + 1)!

where ξ(x) is a function of x and ξ(x) ∈ I, and πn+1 (x) = (x − x0 )(x − x1 ) · · · (x − xn ).

Thus we have
Z b n Z b
1
f (x) dx = ∑ wi f (xi ) + πn+1 (x) f (n+1) (ξ(x)) dx.
a i=0 (n + 1)! a

5
The error term En ( f ) in the quadrature formula
Z b n
f (x) dx ≈ ∑ wi f (xi )
a i=0

is given by
Z b
1
En ( f ) = πn+1 (x) f (n+1) (ξ(x)) dx.
(n + 1)! a

Definition 1. The degree of accuracy or precision of a quadrature formula is the

largest positive integer n such that the formula is exact for xk , for each k = 0, 1, 2, · · · , n.

There are two types of Newton-Cotes formulas. The closed ones and the open
ones. The closed formulas use the function value at all points, and the open formulas
do not use the function values at the endpoints.

2.2 Newton-Cotes formulas

Assume that the value of a function f defined on [a, b] is known at equally spaced
points {a = x0 , x1 , · · · , xn = b}, where xi = ih + x0 , with h = (xn −x
n
0)
= (b−a)
n . The
closed Newton-Cotes formula of degree n is stated as
Z b n
f (x) dx ≈ ∑ wi f (xi ),
a i=0

The wi ’s are called weights or coefficients and are derived from Lagrange basis func-
tions. The weights wi ’s are then given by
Z b
wi = li (x) dx, i = 0, 1, 2, · · · , n,
a

where li (x) is the ith Lagrange basis function and is defined by

n x−xj
li (x) = ∏ .
j=0 xi − x j
j6=i

The interpolating polynomial pn (x) for the set of points ∆ = {x0 , · · · , xn } can be
written as
n
pn (x) = ∑ li (x) f (xi ).
i=0
The integral is thus approximated by integrating pn (x) instead of the function f .
We now consider two examples.

6
Trapezoidal Rule
Z b
(b − a)
f (x) dx ≈ ( f (a) + f (b)).
a 2

Simpson’s Rule
Let c = (a + b)/2.
Z b
(b − a)
f (x) dx ≈ ( f (a) + 4 f (c) + f (b)).
a 6

2.3 Gaussian Integration

Gaussian integration rules are defined by using the zeros of Legendre polynomials.
Assume that we want to maximise the degree of accuracy of the following integration
formula Z 1 n
f (x) dx ≈ ∑ wi f (xi ),
−1 i=1
where {wi }ni=1are the quadrature weights and {xi }ni=1 are quadrature nodes. Newton-
Cotes rules are obtained by using the equidistant quadrature nodes {xi }ni=1 . The
degree of accuracy of these formulas are either n or n − 1, depending on whether
n is odd or even. The degree of accuracy of the formula is maximised if we use
the quadrature nodes {xi }ni=1 as the zeros of Legendre polynomials. An important
property of the Legendre polynomials is that they are orthogonal with respect to the
inner product h, i defined as:
Z 1
h f , gi = f (x) g(x) dx.
−1

Thus Z 1
2
δi j .
Pi (x)Pj (x) dx =
−1 2i + 1
These polynomials can be computed by using the Gram-Schmidt orthognalisation
process. For example, we start from P0 (x) = 1, and let P1 (x) = x + a. Then a is
determined by the equation
Z 1 Z 1
P0 (x)P1 (x) dx = 0, or (x + a) dx = 0, which gives a = 0.
−1 −1

Hence P1 (x) = x. Similarly, using P2 (x) = x2 + cx + d, we have two equations

Z 1 Z 1
P0 (x)P2 (x) dx = 0, and P1 (x)P2 (x) dx = 0,
−1 −1

7
which give
1
c = 0, d=− .
and
3
We can prceed to compute P3 (x), P4 (x), · · · , in a similar way. The first few monic
Legendre polynomials are

1 3 6 3
P0 (x) = 1, P1 (x) = x, P2 (x) = x2 − , P3 (x) = x3 − x, P4 (x) = x4 − x2 + .
3 5 7 35
Theorem 1. Suppose that {x1 , · · · , xn } are the roots of the nth Legendre polynomial
Pn (x) and that for each i = 1, 2, · · · , n, the numbers ci are defined by
Z 1 n
x−xj
ci = ∏ dx.
−1 j=1 xi − x j
j6=i

If P(x) is any polynomial of degree less than 2n, then

Z 1
P(x) dx = ∑ ci P(xi ).
−1 i=1

Proof. We first consider a situation for a polynomial P(x) of degree less than n. We
rewrite P(x) as the Lagrange interpolating polynomial of degree n − 1 with nodes at
the roots of the nth Legendre polynomial Pn (x). Thus
n nx−xj
P(x) = ∑ ∏ P(xi ),
i=1 j=1 xi − x j
j6=i

and thus Z 1 n Z 1 n
x−xj
P(x) dx = ∑ ci P(xi ) with ci = ∏ dx.
−1 i=1 −1 j=1 xi − x j
j6=i

If the polynomial P(x) is of degree at least n but less than 2n, we can divide this by
the nth Legendre polynomial Pn (x) to obtain

P(x) = Q(x)Pn (x) + R(x),

where R(x) is the remainder polynomial and is of degree less than n, and Q(x) is the
quotient polynomial and is also of degree less than n. Since xi is a root of Pn (x) for
each i = 1, 2, · · · , n, we have

P(xi ) = Q(xi )Pn (xi ) + R(xi ) = R(xi ).

8
Using the property of nth Legendre polynomial Pn (x), we have
Z 1 Z 1 Z 1 Z 1
P(x) dx = Q(x)Pn (x) dx + R(x) dx = R(x) dx.
−1 −1 −1 −1

Since R(x) is a polynomial of degree less than n, we have

Z 1 n n
R(x) dx = ∑ ci R(xi ) = ∑ ci P(xi ).
−1 i=1 i=1

Thus Z 1 n
P(x) dx = ∑ ci P(xi ).
−1 i=1

The Gaussian quadrature with a single point is the mid-point rule:

Z 1
1
f (x) dx ≈ f (0).
−1 2

The Gaussian quadrature with two points is given by

Z 1    
−1 1
f (x) dx ≈ f √ + f √ .
−1 3 3

The Gaussian quadrature with three points is given by

Z 1 r ! r !
5 3 8 5 3
f (x) dx ≈ f − + f (0) + f .
−1 9 5 9 9 5

Using the linear mapping y = 12 (b − a)x + 12 (a + b) from [−1, 1] to [a, b], we have
y = a when x = −1 and y = b when x = 1, and dy = 12 (b − a) dx. This transforms the
numerical integration over the interval [−1, 1] to [a, b]. Thus the Gaussian quadrature
with two points over [a, b] becomes
Z b
b−a
f (x) dx ≈ [ f (x1 ) + f (x2 )] ,
a 2

where
1 1 1 1 1 1
x1 = − (b − a) √ + (a + b) and x2 = (b − a) √ + (a + b).
2 3 2 2 3 2

9
2.4 Two-Dimensional Integration
The idea of two-dimensional integration is the same as one-dimensional case. We
approximate the integral as
Z n
f (x, y) dxdy ≈ ∑ wi f (xi , yi ).
Ω i=1

Numerical integration over quadrilaterals

The element integrals for quadrilaterals are approximated by using the tensor product
of a one-dimensional integral. For example, we consider approximating the follow-
ing two-dimensional integral using a quadrature method
Z bZ b
g(x, y) dxdy
a a

using the tensor product of the following one-dimensional quadrature

Z b n
f (x) dx ≈ ∑ wi f (xi ).
a i=1

We can apply the idea of iterated integral to compute the approximation. Thus
Z bZ b Z b n n n n n
g(x, y) dx dy ≈ ∑ wig(xi, y) dy ≈ ∑ w j ∑ wig(xi, y j ) = ∑ ∑ w j wig(xi, y j ).
a a a i=1 j=1 i=1 j=1 i=1

This is the tensor product of the above quadrature. The two-dimensional integra-
tion on a square is obtained by using the tensor product structure from the one-
dimensional case. For example, the Trapezoidal rule over the rectangle [a, b] × [c, d]
in the two-dimensional case becomes
Z dZ b
(b − a)(c − d)
f (x, y) dxdy ≈ [ f (a, c) + f (a, d) + f (b, c) + f (b, d)] .
c a 4
The Simpson’s rule over the rectangle [a, b] × [c, d] in the two-dimensional case be-
comes
Z dZ b
(b − a)(c − d)
f (x, y) dxdy ≈ [ f (a, c) + f (a, d) + f (b, c) + f (b, d) +
c a 36
4[ f (x1 , c) + f (x1 , d) + f (a, y1 ) + f (b, y1 )] + 16 f (x1 , y1 )].

where x1 = (b + a)/2 and y1 = (d + c)/2. Similarly, we can extend other quadrature

rules in higher dimensions.

10
Remark 1. The tensor product integration formulas are not optimal in the sense of
using a fewer number quadrature points. For example, a quintic polynomial has 21
monomials in two dimensions and using a non-tensor product formula of the form
Z 1Z 1 n
f (x, y)dx dy ≈ ∑ wi f (xi , yi )
−1 −1 i=1

will be exact using only n = 7 points. We need to use at least 9 points in tensor prod-
uct Gaussian quadrature rule. Non-tensor-product formulas are not easy to derive.
Quadrature rules are generally obtained by the method of undetermined coefficients
as illustrated below for triangles.

Numerical integration over triangles

Consider the one-point quadrature rule over the triangle K̂
Z
2
K̂ = {(ξ, η) ∈ R |ξ > 0, η > 0, ξ + η < 1}, with f (ξ, η) dξdη ≈ w1 f (ξ1 , η1 ).
K̂

We now use f (ξ, η) = 1, f (ξ, η) = ξ, and f (ξ, η) = η to obtain

1. f (ξ, η) = 1:
Z 1 Z 1−ξ
1
dηdξ = = w1 .
0 0 2
2. f (ξ, η) = ξ:
Z 1 Z 1−ξ
1
ξdηdξ = = w1 ξ1 .
0 0 6
3. f (ξ, η) = η:
Z 1 Z 1−ξ
1
ηdηdξ = = w1 η1 .
0 0 6
Thus the one-point quadrature rule over the triangle K̂ is
1
Z
f (ξ, η)dηdξ ≈ f (1/3, 1/3).
K̂ 2
This quadrature rule is exact for all linear polynomials on K̂. In this way, any quadra-
ture rule can be constructed. For example, we can have three points
p1 = (1/6, 1/6), p2 = (1/6, 4/6), and p3 = (4/6, 1/6),
with three weights w1 = w2 = w3 = 1/6 such that
Z 3
f (ξ, η) dξdη ≈ ∑ wi f (pi ),
K̂ i=1

11
which can integrate a quadratic function exactly.
The two important rules for numerical integration over a triangle K are
1. The three point rule with vertices
|K|
Z
φ dx ≈ (φ(p1 ) + φ(p2 ) + φ(p3 )) .
K 3
This rule gives the exact result for a polynomial of degree one.

2. The midpoint rule

|K|
Z
φ dx ≈ (φ(m1 ) + φ(m2 ) + φ(m3 )) ,
K 3
where m1 , m2 and m3 are midpoints of three edges of K. This rule gives the
exact result for a polynomial of degree two.
We give some quadrature rules over the unit triangle K̂, where K̂ is given by

K̂ = {(ξ, η) ∈ R2 |ξ > 0, η > 0, ξ + η < 1}.

Here n is the number of points, p is the degree of accuracy, (xi , yi ) the nodes and wi
the weights of the quadrature rule

Table 1: Symmetric quadrature for the unit triangle

n p i xi yi wi
1 1 1 1/3 1/3 1/2
3 2 1 1/6 1/6 1/6
2 2/3 1/6 1/6
3 1/6 2/3 1/6
4 3 1 1/3 1/3 -9/32
2 3/5 1/5 25/96
3 1/5 3/5 25/96
4 1/5 1/5 25/96
7 4 1 0 0 1/40
2 1/2 0 1/15
3 1 0 1/40
4 1/2 1/2 1/15
5 0 1 1/40
6 0 1/2 1/15
7 1/3 1/3 9/40

12