2015/06/16 v0.5.

1 IPOL article class

Published in Image Processing On Line on 2011–09–27.

Submitted on 2011–00–00, accepted on 2011–00–00.
ISSN 2105–1232 c 2011 IPOL & the authors CC–BY–NC–SA
This article is available online with supplementary materials,
software, datasets and online demo at
https://doi.org/10.5201/ipol.2011.g_lmii

Linear Methods for Image Interpolation

Pascal Getreuer
CMLA, ENS Cachan, France ([email protected])

Communicated by Gabriele Facciolo Demo edited by Pascal Getreuer

Abstract

We discuss linear methods for interpolation, including nearest neighbor, bilinear, bicubic, splines,
and sinc interpolation. We focus on separable interpolation, so most of what is said applies to
one-dimensional interpolation as well as N-dimensional separable interpolation.

Source Code

The source code (ANSI C), its documentation, and the online demo are accessible at the IPOL
web page of this article1 .

Keywords: interpolation; linear methods; separable interpolation

1 Introduction
Given input image v with uniformly-sampled pixels vm,n , the goal of interpolation is to find a function
u(x, y) satisfying
vm,n = u(m, n) for all m, n ∈ Z,

such that u approximates the underlying function from which v was sampled. Another way to
interpret this is consider that v was created by subsampling, and interpolation attempts to invert
this process.
We discuss linear methods for interpolation, including nearest neighbor, bilinear, bicubic, splines,
and sinc interpolation. We focus on separable interpolation, so most of what is said applies to
one-dimensional interpolation as well as N -dimensional separable interpolation.
1
https://doi.org/10.5201/ipol.2011.g_lmii

Pascal Getreuer, Linear Methods for Image Interpolation, Image Processing On Line, 1 (2011), pp. 238–259.
https://doi.org/10.5201/ipol.2011.g lmii
 Linear Methods for Image Interpolation

2 Notations
We use the following normalization of the Fourier transform,
Z ∞
ˆ
f (ξ) = f (x)e−2πixξ dx,
−∞

the transform of a function f is denoted by hat. We will also use the bilateral Z-transform. The
Z-transform of a sequence cn is
X∞
c(z) := cn z −n .
n=−∞

We denote the sequence with a subscript cn and its Z-transform as c(z).

3 Interpolation Kernels
Linear interpolation can be abstractly described as a linear operator Z mapping samples v to a
function u := Z(v). We define two properties on Z.

1. Let Sk,l denote shifting, Sk,l (v)(m, n) := vm−k,n−l . Z is said to be shift-invariant if for every v
and (k, l)
Z(Sk,l (v))(x, y) = Z(v)(x − k, y − l).

2. Let v N denote the restriction of v to the set {−N, . . . , +N }2 , then Z is said to be local if for
every v and (x, y)
Z(v)(x, y) = lim Z(v N )(x, y).
N →∞

If Z is a linear, shift-invariant, local operator from `∞ (Z2 ) to L∞ (R2 ), then there exists a unique
function K ∈ L1 called the interpolation kernel such that Z(v) can be written as a convolution [10]
X
Z(v)(x, y) = vm,n K(x − m, y − n).
m,n∈Z

The conditions for this result are mild and are satisfied by most linear interpolation methods of
practical interest.
The properties of the kernel K, like symmetry, regularity, and moments, imply properties of
the interpolation. For four-fold rotational symmetry, K is usually designed as a tensor product
K(x, y) = K1 (x)K1 (y), where K1 is a symmetric function. Such a kernel also has the benefit that the
interpolation is separable, the interpolation can be decomposed into one-dimensional interpolations
along each dimension.
Similarly in three (or higher) dimensions, it is convenient to use a separable kernel K(x, y, z) =
K1 (x)K1 (y)K1 (z) so that the interpolation decomposes into one-dimensional interpolations. More
generally, one can use different one-dimensional kernels along different dimensions, K(x, y, z) =
K1 (x)K2 (y)K3 (z). For instance, it may be more appropriate to use a different kernel for the temporal
dimension than for the spatial dimensions in video interpolation.
In order to have the interpolant agree with the known samples Z(v)(m, n) = vm,n it is easy
to see that the requirements are K(m, n) = 0 for all integer m and n, except at the origin where
K(0, 0) = 1. K is said to be an interpolating function if it satisfies these requirements.

239
 Pascal Getreuer

4 Two-Step Interpolation
That K must be interpolating can be an inconvenient restriction. It complicates the problem of
finding a kernel K that also satisfies other design objectives. It can also be an obstacle computa-
tionally. In some cases such as spline interpolation, K has infinite support, yet the interpolant can
be expressed as a linear combination of compact support functions (the B-splines).
Suppose that one dimensional data fm is to be interpolated. An alternative approach is to begin
with a basis function ϕ(t) and then express the interpolant as a linear combination of {ϕ(t − n)}
X
u(t) = cn ϕ(t − n),
n∈Z

where the coefficients cn are selected such that

X
fm = cn ϕ(m − n).
n∈Z

That is, interpolation is a two-step

X procedure: first, we solve for the coefficients cn and second,
the interpolant is constructed as cn ϕ(t − n).
P
To solve for the coefficients, notice that fm = cn ϕ(m−n) is equivalent to a discrete convolution
fn = (c ∗ p)m where pm = ϕm . This suggests solving for cn under the Z-transform as
1
f (z) = c(z)p(z) ⇒ c(z) = f (z).
p(z)
Let (p)−1 denote the convolution inverse of p, which is the sequence such that p ∗ (p)−1 = δ where
δ is the unit impulse (δ0 := 1 and δm := 0 otherwise). Using (p)−1 we obtain cn by “prefiltering” as
c = (p)−1 ∗ f.
A unique convolution inverse exists in many practical cases of interest [7]. If it exists and ϕ is real
and symmetric, then (p)−1 can be factored into pairs of recursive (infinite impulse response)
X filters,
allowing for an efficient in-place calculation. Finally, the interpolant is obtained as cn ϕ(t − n).
It is possible to express this two-step interpolation procedure in terms of an interpolation kernel,
X
u(t) = ((p)−1 ∗ f )n ϕ(t − n)
n∈Z

X X
u(t) = fm ((p)−1 )n−m ϕ(t − n)
m∈Z n∈Z

X X
u(t) = fm K1 (t − m), K1 (t) = ((p)−1 )n ϕ(t − n)
m∈Z n∈Z

For example, the figures 1 and 2 illustrate this relationship for cubic and septic (7th) B-spline
interpolation where ϕ are the cubic and septic B-splines. In both cases, ϕ is not an interpolating
function, but prefiltering with (p)−1 yields a kernel K1 that is interpolating.

5 Interpolation Properties
An interpolation method Z has approximation order J if it reproduces polynomials up to degree
(J − 1).
Let Z and K1 be defined from a function ϕ as in the previous section. The Strang–Fix condi-
tions [2] provide the following equivalent conditions to determine the approximation order.

240
 Linear Methods for Image Interpolation

Figure 1: Two-step interpolation where ϕ is the cubic B-spline.

Figure 2: Two-step interpolation where ϕ is the septic B-spline.

1. Z has approximation order J,

2. ϕ̂(0) = 1, ϕ̂j (2πk) = 0 ∀k ∈ Z∗ , j = 0, . . . , J − 1,
X
3. (x − k)j ϕ(x − k) = µj ∀x ∈ R, j = 0, . . . , J − 1.
k∈Z

where ϕ̂j denotes the jth derivative of ϕ̂ and Z∗ := Z/{0}. Aside from approximation X order, another
2 2
property of interest is the L interpolation error. Let f ∈ L (R) and define fδ (t) = f (δn)K1 (t−n)
for δ > 0 The interpolation error is approximately
Z
kf − fδ kL2 ≈ |f (ξ)|2 Eint (ξδ)dξ,
2

2
ϕ̂(ξ + k) + k∈Z∗ |ϕ̂(ξ + k)|2
P P
k∈Z∗
Eint (ξ) := P 2 .
k∈Z ϕ̂(ξ + k)
This approximation is exact for bandlimited functions. Otherwise, it is equal to the average L2
error over all translations of f . The function E is called the interpolation error kernel, and it can be
interpreted as predicting the error made in interpolating sin(2πζt). Asymptotically as δ → 0,
kf − fδ kL2 ∼ Cint δ J kf (J) kL2 .
The decay of the error is dominated by the approximation order J. When comparing methods of
the same approximation order, the constant Cint is also of interest [7].

6 Kernel Normalization
A basic and desirable quality of an interpolation method is that it reproduces constants, which is
equivalent to X
K1 (t − n) = 1 ∀t ∈ R.
n∈Z

241
 Pascal Getreuer

If the kernel does not reproduce constants, it can be the normalized to fix this as
K1 (t)
K̃1 (t) = X ,
K1 (t − m)
m∈Z

provided that the denominator does not vanish. Normalization ensures that the interpolation weights
sum to 1. Interpolation with the normalized kernel reproduces constants: suppose fn = c, then
P
X cK1 (t − n)
u(t) = cK̃1 (t − n) = Pn∈Z = c.
m∈Z K 1 (t − m)
n∈Z

7 Nearest Neighbor
The most widely used methods for image interpolation are nearest neighbor, bilinear, and bicubic
interpolation (see Figure 3).

Nearest Neighbor Bilinear Bicubic

Figure 3: Nearest neighbor, bilinear, and bicubic applied to the same uniformly-spaced input data.

The nearest neighbor interpolation of v is the piecewise constant function

u(x, y) = v[x],[y] ,
where [.] denotes rounding to the nearest integer. That is, u(x, y) is defined as the value of the input
sample that is closest to (x, y). For this reason, nearest neighbor interpolation is sometimes called
“pixel duplication”. The interpolation kernel for nearest neighbor is (Figure 4)
1 if − 12 ≤ t < 12 ,

K(x, y) = K1 (x)K1 (y), K1 (t) =
0 otherwise.

Figure 4: Nearest neighbor interpolation kernel.

The nearest neighbor idea can be applied to very general kinds of data, to any set of samples that
has a notion of distance between sample locations. The interpolant is constant within the Voronoi
cell around each sample location (Figure 5).

242
 Linear Methods for Image Interpolation

Figure 5: Nearest neighbor scattered data interpolation.

8 Bilinear
The bilinear interpolation of v is the continuous function

u(x, y) = (1 − hxi)(1 − hyi)vbxc,byc + hxi(1 − hyi)vbxc+1,byc

,
+ (1 − hxi)hyivbxc,byc+1 + hxihyivbxc+1,byc+1

where b.c denotes the floor function and hxi := x − bxc is the fractional part. The interpolation
kernel is (Figure 6)
K(x, y) = K1 (x)K1 (y), K1 (t) = (1 − |t|)+ ,

where (.)+ denotes positive part.

Figure 6: Bilinear interpolation kernel.

Within each cell [m, m + 1] × [n, n + 1], the interpolation is a convex combination of the samples
located at the cell corners vm,n , vm+1,n , vm,n+1 , vm+1,n+1 . Because this combination is convex, the
interpolation is bounded between min v and max v and does not produce overshoot artifacts. Bilinear
interpolation reproduces affine functions: if vm,n = am + bn + c then u(x, y) = ax + by + c.
Bilinear interpolation is arguably the simplest possible separable method that produces a con-
tinuous function. It is extremely efficient and on many platforms available in hardware, making it
practical for realtime applications.
A closely related method to bilinear interpolation is linear interpolation. Given a triangulation
of the sample locations, each triangle is interpolated by the affine function that satisfies the data at
the triangle vertices.

243
 Pascal Getreuer

9 Bicubic
Bicubic interpolation [3] uses the interpolation kernel (Figures 7 and 8)


 (α + 2)|t|3 − (α + 3)|t|2 + 1 if |t| ≤ 1



K1 (t) = α|t|3 − 5α|t|2 + 8α|t| − 4α if 1 < |t| < 2,




0 otherwise.


where α is a free parameter. This function is derived by finding a piecewise cubic polynomial with
knots at the integers that is required to be symmetric, C 1 continuous, and have support in −2 < t < 2.
These conditions leave one remaining degree of freedom represented by α. For any value of α, K1
has extremal points at t = 0 and ±4/3.

Figure 7: Cubic interpolation kernel.

The values −1, −0.75, and −0.5 have been proposed for α, motivated by various notions of
optimality [9]. The choice α = −0.5 is particularly compelling. Bicubic interpolation is third-order
accurate with α = −0.5 and only first-order accurate for any other α. Furthermore, α = −0.5 is
optimal in a sense of matching the sinc kernel and is also optimal in terms of Eint . All examples and
comparisons with bicubic here use α = −0.5.

Figure 8: Surface plot of K1(x)K1(y). Yellow circles indicate local maxima.

10 Sinc
The Whittaker–Shannon interpolation [1] of vm,n is
X
u(x, y) = vm,n sinc(x − m) sinc(y − n),
m,n∈Z

where sinc(t) := sin(πt)/(πt) for t 6= 0 and sinc(0) := 1 is the interpolation kernel (Figures 10
and 11). The interpolation is such that u is bandlimited, that is, its Fourier transform is zero for

244
 Linear Methods for Image Interpolation

Figure 9: Comparison of the nearest neighbor, bilinear, and bicubic kernels and the sinc.

frequencies outside of [− 12 , 12 ]. This interpolation is also often called “sinc interpolation” or “Fourier
zero-padding.” The extremal points of the sinc kernel are solutions of the equation πt = tan(πt),
which include t ≈ 0, ±1.4303, ±2.4590, ±3.4709, ±4.4774.

Figure 10: The sinc function.

Figure 11: Surface plot of sinc(x)sinc(y). Yellow circles indicate local maxima.

The powerful property of sinc interpolation is that it is exact for bandlimited functions, that is,
if vm,n = f (m, n) with
ZZ
1
f (x, y) exp−2πi(xξ+yη) dxdy = 0 for all |ξ| or |η| ≥ ,
2
then the interpolation reproduces f .

Proof. We can verify this result for f ∈ L1 from two Fourier properties: first, the Fourier transform
of sinc is the rectangular pulse

eπit − e−πit
Z
sin πt
1{|ξ|<1/2} e2πitξ dt = = ,
2πit πt

245
 Pascal Getreuer

and second, the Poisson formula

X X
f (m, n)e−2πi(mξ+nη) = fˆ(ξ − m, η − n).
m,n∈Z m,n∈Z

Applying these two properties, the Fourier transform of the interpolant is

ZZ X
û(ξ, η) = f (m, n) sinc(x − m) sinc(y − n)e−2πi(xξ+yη) dxdy
m,n∈Z ZZ
X
= f (m, n) sinc(x − m) sinc(y − n)e−2πi(xξ+yη) dxdy
m,n∈Z
= 1{|ξ|<1/2} 1{|η|<1/2}
X
f (m, n)e−2πi(mξ+nη)
m,n∈Z
= 1{|ξ|<1/2} 1{|η|<1/2}
X
fˆ(ξ − n, η − m).
m,n∈Z

In the second equality, the sum-integral interchange is justified by dominated convergence. The final
quantity is equal to fˆ(ξ, η) if fˆ is zero outside of the region [− 12 12 ] × [− 12 12 ]. 

In some ways, sinc interpolation is the ultimate interpolation. It is exact for bandlimited functions,
so the method is very accurate on smooth data. Additionally, Fourier zero-padding avoids staircase
artifacts, it is effective in reconstructing features at different orientations. Under an aliasing condition
[10], Fourier interpolation reproduces cylindrical functions f (x, y) = h(αx + βy).
The disadvantage of sinc interpolation is that in aliased images it produces significant ripple
artifacts (Gibbs phenomenon) in the vicinity of image edges (Figure 12). This is because sinc(x)
decays slowly, at a rate of 1/x, so the damage from meeting an edge is spread throughout the image.
Moreover, bandlimitedness can be a distorted view of reality. Thévenaz et al. [7] give the following
amusing example: consider the air/matter interface of a patient in a CT scan, then according to
classical physics, this is an abrupt discontinuity and cannot be expressed as a bandlimited function.
Antialias filtering is not possible on physical matter, and any attempt to do so would probably be
harmful to the patient.

Nearest Neighbor Bilinear Bicubic Lanczos-3 Cubic B-Spline Sinc

Figure 12: Linear interpolation of a step edge: a balance between staircase artifacts and ripples.

11 Windowed Sinc Approximations

A solution to limiting the ripple artifacts of the sinc kernel is to approximate it with a compactly-
supported function
K1 (t) = w(t)sinc(t),

246
 Linear Methods for Image Interpolation

where w is a window function. A popular choice in image processing is the Lanczos window,

sinc(t/n) if |t| < n,
w(t) =
0 otherwise.

where n is a positive integer usually set to 2 or 3. The Lanczos kernels (Figures 13, 14 and 15) do not
reproduce constants exactly, but can be normalized to fix this as described in the section on Kernel
Normalization.

Figure 13: The Lanczos-3 kernel.

Figure 14: Surface plot of the Lanczos-3 kernel. Yellow circles indicate local maxima.

There are many other possibilities for the window, for example Hamming, Kaiser, and DolphCheby-
shev windows to name a few, each making different tradeoffs in frequency characteristics.

Figure 15: Comparison of the normalized Lanczos kernels and the sinc.

247
 Pascal Getreuer

12 Splines
Define the B-splines
j+1  
j 1X j+1 j+1
j
β (t) = (−1)k (t + − k)+ .
j! k=0 k 2

The B-spline functions (see example in Figure 16) are optimal in the sense that, among all
piecewise polynomials with uniformly spaced knots, the B-splines have the maximal approximation
order and are maximally continuous for a given support.
The B-spline β J−1 has approximation order J. For J > 2, β J−1 is not interpolating, so prefiltering
must be applied as described in the section on Two-Step Interpolation where β J−1 takes the role of
ϕ.
As J → ∞, B-spline interpolation converges to Whittaker–Shannon interpolation in a strong
sense: the associated interpolation kernel K1 converges to the sinc both in 1 ≤ p < ∞, see [4],[6].
This convergence is illustrated Figure 17 with K1 and its Fourier transform for degrees 1, 3, 5, and
7.

Figure 16: The cubic B-spline.

Figure 17: Comparison of the B-spline interpolation kernels and the sinc.

Aside from the B-splines, there are numerous other piecewise polynomial methods for interpola-
tion in the literature. One example is the class of Maximum Order and Minimal Support (Moms)
functions introduced by Blu, Thévenaz, and Unser [8], which are linear combinations of β J−1 and its
derivatives
J−1
X dk
ϕ(t) = ck k β J−1 (t).
k=0
dt
The B-splines are a special case within the class of Moms. The B-splines are the Moms with
maximum regularity, they are J − 2 times continuously differentiable.

248
 Linear Methods for Image Interpolation

Within the Moms, the “o-Moms functions” are the Moms defined by minimizing the quantity
s
1 X (
|ϕ̂ J)(k)|2 .
J! k∈Z
∗

The o-Moms functions have the same approximation order and support as the B-splines, but
the Cint constant is smaller. So compared to the B-splines, the advantage of o-Moms is that they
have lower asymptotic L2 interpolation error. On the other hand, o-Moms are less regular than the
B-splines.
The splines proposed by Schaum [5] are also within the class of Moms. Schaum splines have the
property that they are interpolating, so prefiltering is not needed. However, for a given support size,
the approximation constant Cint is worse than with the o-Moms of the same support.
The first few even-order o-Moms are
Order J ϕ(t)
2 β 1 (t)
1 d2
4 (1 + 42 dx2
)β 3 (t)
1 d2 1 d4
6 (1 + 33 dx2
+ 7920 dx4
)β 5 (t)
1 d2 1 d4 1 d6 7
8 (1 + 30 dx2
+ 4680 dx4
+ 3603600 β
(t)
2 d2 7 d4 1 d6 1 d8
10 (1 + 57 dx2 + 25840 dx4 + 1627920 dx6 + 3047466240 dx8
)β 9 (t)
5 d2 1 d4 1 d6 1 d8 1 d10
12 (1 + 138 dx2
+ 3220 dx4
+ 1101240 dx6
+ 1078334208 dx8
+ 4151586700800 dx10
)β 11 (t)

A note on numbering: Usually, with a piecewise polynomial method, its approximation order is
L but its highest degree is L − 1. We refer to methods by degree, for instance “o-Moms 3” and “β 3 ”
are methods that are locally cubic polynomial and have approximation order 4.

13 Radial Basis Functions

A radial basis function (RBF) is a function ϕ such that
p
ϕ(x, y) = ϕ(r), r := x2 + y 2 .

RBF interpolation is useful for scattered data interpolation. Let (xn , yn ) denote the location of
the nth sample, then the interpolant is
X
u(x, y) = cn ϕ(x − xn , y − yn ),
n

where the coefficients cn are solved such that u(xn , yn ) = vn . If the samples are uniformly spaced,
then the cn can be found as described in the section on Two-Step Interpolation.
Popular choices for ϕ include

• Thin plate splines: ϕ(r) = |εr|n log |εr|, n even

• Bessel: ϕ(r) = J0 (εr)

• Gaussian: ϕ(r) = exp(−(εr)2 )

1
• Multiquadric: ϕ(r) = (1 + (εr)2 ) 2
1
• Inverse multiquadratic: ϕ(r) = ϕ(r) = (1 + (εr)2 )− 2

249
 Pascal Getreuer

• Inverse quadratic: ϕ(r) = ϕ(r) = (1 + (εr)2 )−1

where ε is a positive parameter controlling the shape. A challenging problem is that smaller ε
improves interpolation quality but worsens the numerical conditioning of solving the cn . A topic of
interest beyond the scope of this article is RBF interpolation in the limit ε → 0. For example, [11]
investigates numerically stable interpolation for small ε with Gaussian ϕ.

14 Methodology
14.1 Boundary handling
A technicality is how to handle the image boundaries. The interpolation formulas may refer to
samples that are outside of the domain of definition, especially when interpolating at points near or
on the image boundaries.
The usual approach is to extrapolate (pad) the input image. Several methods for doing this are

• Constant extension, . . . aaaabcdeeee. . .

• Half-sample symmetric extension, . . . cbaabcdeedc. . .

• Whole-sample symmetric extension, . . . dcbabcdedcb. . .

14.2 Image scaling

Suppose v is an M × N image with samples vm,n , m = 0, . . . M − 1, n = 0, . . . N − 1 which are located
on the integer grid Z2 .
For scaling v by a (possibly non-integer) factor d, a choice for where to resample the function u
is on a top-left-anchored grid of points (Figure 18)

1 1
( m0 , n0 ), m0 = 0, . . . , M 0 − 1, ; n0 = 0, . . . , N 0 − 1,
d d
with M 0 = [dM ], N 0 = [dN ], where [.] is a rounding convention.

Figure 18: Example top-left-anchored grids. Open circles denote input samples and filled circles
denote interpolation samples.

Another choice is the centered grid of points (Figure 19)

1 1
( m0 + sd,M,M 0 , n0 + sd,N,N 0 ), m0 = 0, . . . , M 0 − 1, ; n0 = 0, . . . , N 0 − 1,
d d
250
 Linear Methods for Image Interpolation

Figure 19: Example centered grids. Open circles denote input samples and filled circles denote
interpolation samples.

where sd,M,M 0 = (1/d − 1 + M − M 0 /d)/2. If d is integer, the offsets simplify to s = (1/d − 1)/2.
Although more complicated, the centered grid has the advantage that interpolation with a symmetric
kernel on this grid commutes with flipping the image.
For an odd integer scale factor and interpolation with a symmetric kernel, the two grids are
related in a simple way through adding and removing border samples. Interpolation performed on
the top-left-anchored grid is converted to an interpolation on the centered grid by removing (d − 1)/2
rows and columns from the bottom and right borders and extrapolating (according to the boundary
extension) the same number of rows and columns on the top and left.

14.3 Domain mapping

Aside from changing image resolution, another application is to deform or map the domain of the
image. That is, the mapped coordinates (x0 , y 0 ) are related to the original coordinates (x, y) by
x = f (x0 , y 0 ), y = g(x0 , y 0 ).
Let u(x, y) be an interpolation of v, then the deformed image is obtained by resampling as


ṽm,n = u f (m, n), g(m, n) .
For example, two simple mappings are
• Sub-pixel translation
x = x0 − ∆x
y = y 0 − ∆y

• Rotation
x = x0 cos θ − y 0 sin θ
y = x0 sin θ − y 0 cos θ

Other applications include parallax correction, lens distortion correction, image registration, and
texture mapping.

15 Algorithm
Linear interpolation amounts to evaluating the sum
X
u(x, y) = vm,n K(x − m, y − n).
m,n∈Z

251
 Pascal Getreuer

Choices of algorithms to do this efficiently depend on K and the sampling locations. Keep in
mind that the sum is conceptually over the infinite integer grid, so vn,m should be replaced with an
appropriate boundary extension when m, n is beyond the domain of the image.

15.1 Nearest neighbor, bilinear, and bicubic

Suppose the interpolation method has a separable kernel K(x, y) = K1 (x)K1 (y) with compact sup-
port, K1 (x) = 0 for x ≥ R. Then interpolation at point (x, y) can be implemented as

m0 = [x − R], n0 = [y − R], L = 2R
+L−1 n0X
m0X +L−1
u(x, y) = vm,n K1 (x − m)K1 (y − n),
m=m0 n=n0

where there are L × L non zero terms in the sum. The values of K1 can be reused so that K1 only
needs to be evaluated for 2L different arguments
For image scaling, it is possible to interpolate more efficiently by exploiting the separable structure
of the sampling points. Let vm,n , m = 0, . . . M − 1, n = 0, . . . N − 1 be the given image, which is to
be interpolated on the grid of points (x0n , yn0 ) for m0 = 0, . . . M 0 − 1,n’ = 0,. . . N’-1. Interpolation is
performed by first interpolating along the y direction,
n0X
+L−1
wm,n0 = vm,n K1 (yn0 − n), n0 = dyn0 − Re, for m = 0, . . . M − 1, n0 = 0, . . . N 0 − 1,
n=n0

and then along the x direction,

m0X
+L−1
u(xm0 , yn0 ) = wm,n K1 (xm0 − m), m0 = dxm0 − Re, for m0 = 0, . . . M 0 − 1, n0 = 0, . . . N 0 − 1.
m=m0

An efficient way to implement the one dimensional interpolations is to express them as multipli-
cation with a sparse matrix
XN
wm,n =
0 An0 ,n vm,n ,
n=1
M
X
u(xm0 , yn0 ) = Bm0 ,m wm,n0 .
m=1

A is a sparse matrix of size N 0 × N and B is a sparse matrix of size M 0 × M . Away from the
boundaries, the matrix entries are

An0 ,n = K1 (yn0 − n), Bm0 ,m = K1 (xm0 − m).

Near the boundaries, the matrix entries need to be adjusted according to the boundary extension.
Define

Half-sample symmetric: E(n) = min n mod 2N, (2N − 1 − n) mod 2N

Whole-sample symmetric: E(n) = min n mod 2N − 2, (2N − 2 − n) mod 2N − 2

 0 if n < 0,
Constant extension: E(n) = n if 0 ≤ n ≤ N − 1,
N − 1 if N − 1 < n,


252
 Linear Methods for Image Interpolation

then the matrix entries near the boundaries are computed as

X
An0 ,n = K1 (yn0 − k),
k∈E −1 (n)

and similarly for B. This sparse matrix approach is used by the libswscale library in FFmpeg 2 for
scaling video frames. It is especially well-suited for video, as A and B only need to be constructed
once to interpolate any number of frames.
If the interpolation locations are uniformly spaced with a rational period, tn0 = t0 + n0 a/b, then
another approach is to compute one-dimensional interpolation through convolution. Given n0 , let
s = bbn0 /bc and r be such that n0 = bs + r, then
X X a a
fn K1 (tn0 − n) = fn K1 (t0 + r + as − n) = (f ∗ hr n )as , hrn = K1 (t0 + r + n).
n n
b b

15.2 Lanczos
Interpolation with the Lanczos kernels does not exactly reproduce constants. To fix this, the Lanczos
kernels should be normalized as described in the section on Kernel Normalization.
Kernel normalization can be incorporated efficiently into the algorithms discussed in the previous
section. For the interpolation of a single point (x, y), the normalized interpolation is
X
vm,n K1 (x − m)K1 (y − n)
m,n
u(x, y) = X .
K1 (x − m)K1 (y − n)
m,n

For the sparse matrix approach, kernel normalization is achieved by scaling each matrix row so
that it sums to one. Similarly for the convolution approach, hr should be scaled so that it sums to
one.

15.3 Splines
For spline methods, implementation is as described in the section on Two-Step Interpolation. The
method is determined by the choice of basis function ϕ, for example, ϕ may be a B-spline or an
o-Moms function.
Given a basis function ϕ, define pm = ϕ(m). For the prefiltering step, the convolution inverse
(p)−1 is needed. We look first at the cubic B-spline as an example. For the cubic B-spline,
6 1 −r
p(z) = (z + 4 + z −1 )/6, and −1 = 6 −1 ,
z+4+z 1 − rz 1 − rz
where r = 31/2 − 2.. The first factor corresponds to the first-order causal recursive filter
c0 n = fn + rc0 n−1 , n = 1, . . . , N − 1.
The left endpoint c0 0 can be computed depending on the boundary extension,
∞
X
Half-sample symmetric: 0
c 0 = f0 + r r n fn ,
n=0
∞
X
Whole-sample symmetric: c0 0 = r n fn .
n=0
2
http://www.ffmpeg.org.

253
 Pascal Getreuer

Since |r| < 1, the terms of the sum are decaying, so in practice we only evaluate as many terms
as are needed for the desired accuracy.
The second factor corresponds to the first-order anti-causal recursive filter

c00 n = r(c00 n+1 − c0 n ), n = N − 2, . . . , 0.

The right endpoint c00 N −1 can be computed according to the boundary extension as

Half-sample symmetric: r c0
c00 N −1 = r − 1 N −1

Whole-sample symmetric c00 N −1 = r (c0 0

N −1 + rc N −2 ).
r2 − 1
Finally, the constant scale factor is applied cn = 6c00 n . The cost of prefiltering is linear in the
number of pixels. Prefiltering can be computed in-place so that the prefiltered values overwrite the
memory used by the input. For prefiltering in two-dimensions, the image is first prefiltered along
each column, then the column-prefiltered image is prefiltered along each row.
More generally, if ϕ is symmetric and compactly supported, then p(z) has the form
J
X
p(z) = a0 + aj (z j + z −j ),
j=1

J
−J
Y 1
p(z) = aJ z (z − rj )(z − ),
j=1
rj

where the rj and 1/rj are the roots of the polynomial z −J p(z). We enumerate the roots so that
|rj | < 1. The Z-transform of (p)−1 is
J
1 1 Y 1 −rj
= −1
p(z) aJ j=1 1 − rj z 1 − rj z

Therefore, prefiltering can be performed as a cascade of first-order recursive filters using the same
algorithm as for the cubic B-spline. Table 1 lists the roots and the constant scale factor for several
B-splines and o-Moms.
Once the image has been prefiltered c = (p)−1 ∗ v, interpolation at a point (x,y) is computed as
X
u(x, y) = cm,n ϕ(x − m)ϕ(y − n).
m,n∈Z

This second step is essentially the same formula as in the section on nearest neighbor, bilinear,
and bicubic, and the same algorithms can be used to perform this step. The only differences are that
the prefiltered image c is used in place of v and ϕ is used in place of K1 .

15.4 Sinc
For sinc interpolation with an integer scale factor, interpolation can be implemented using the fast
Fourier transform (FFT). This approach follows from observing that the sinc interpolant is the unique
image that is bandlimited and agrees with the input data.
The input image is first padded to twice its size in each dimension with half-symmetric extension
and transformed with the FFT. The interpolation is then constructed in the Fourier domain by

254
 Linear Methods for Image Interpolation

Method 1/aJ rj . Method 1/aJ rj .

β2 8 −3 + 81/2 o-Moms 21/4 (1051/2 − 13)/8
β3 3! 31/2 − 2 3
β5 5! −4.309628820326465 × 10−2 , o-Moms 7920/107 −7.092571896868541 × 10−2 ,
−4.305753470999738 × 10−1 5 −4.758127100084396 × 10−1
β7 7! −9.148694809608277 × 10−3 , o-Moms 675675/346 −1.976842538386140 × 10−2 ,
−1.225546151923267 × 10−1 , 7 −1.557007746773578 × 10−1 ,
−5.352804307964382 × 10−1 −5.685376180022930 × 10−1
β9 9! −2.121306903180818 × 10−3 ,
−4.322260854048175 × 10−2 ,
−2.017505201931532 × 10−1 ,
−6.079973891686259 × 10−1
β 11 11! −5.105575344465021 × 10−4 ,
−1.666962736623466 × 10−2 ,
−8.975959979371331 × 10−2 ,
−2.721803492947859 × 10−1 ,
−6.612660689007345 × 10−1

Table 1: Roots and constant scale factor for several B-splines and o-Moms.

copying the transform coefficients of the padded input image for the lower frequencies and filling
zeros for the higher frequencies. The final interpolation is obtained by inverse FFT and removal
of the symmetric padding. For real-valued input data, some computational savings can be made
by using a real-to-complex FFT and complex-to-real inverse FFT to exploit the complex conjugate
redundancy in the transform. The complexity of the interpolation is O(N log N ) for N output pixels.

16 Examples
16.1 Scaling smooth data
The smooth function cos((x2 +y 2 )/10) is sampled on the grid {0.5, 1.5, . . . , 15.5}×{−15.5, −14.5, . . . , 15.5}
and interpolated on the centered grid with scale factor 4. Figure 20 compares the interpolation re-
sult using various linear methods, along with the root mean squared errors (RMSE) between the
interpolation and the exact image.
For this example, higher-order methods tend to perform better. The o-Moms perform slightly
better than the B-splines of the same order. Sinc interpolation yields the best result.

16.2 Scaling piecewise smooth data

We repeat the previous experiment with piecewise smooth data (Figure 21). The image is discon-
tinuous at the edges, so the bandlimited hypothesis of sinc interpolation is severely violated. Sinc
interpolation and other higher-order methods produce strong artifacts near edges. The bicubic and
Lanczos-2 interpolations have the lowest RMSE in this case.

16.3 Texture mapping

A 3D scene of a plane and a sphere is texture mapped with an image (Figure 22). Texture mapping
is an example of domain mapping, where in this case the image domain is mapped to the projected
plane and sphere surfaces.

255
 Pascal Getreuer

Exact Nearest Bilinear Bicubic Lancozs Lancozs Lancozs Sinc

RMSE 59.2 RMSE 47.2 RMSE 39.0 RMSE 38.8 RMSE 33.8 RMSE 31.9 RMSE 27.2

β3 o-Moms 3 β5 o-Moms 5 β7 o-Moms 7 β9 β 11

RMSE 33.7 RMSE 31.8 RMSE 30.9 RMSE 30.4 RMSE 29.7 RMSE 29.4 RMSE 29.0 RMSE 28.5

Figure 20: Interpolation examples with smooth data.

Exact Nearest Bilinear Bicubic Lancozs Lancozs Lancozs Sinc

RMSE 32.2 RMSE 25.6 RMSE 24.5 RMSE 24.5 RMSE 24.8 RMSE 25.1 RMSE 26.3

β3 o-Moms 3 β5 o-Moms 5 β7 o-Moms 7 β9 β 11

RMSE 24.7 RMSE 24.9 RMSE 25.1 RMSE 25.2 RMSE 25.4 RMSE 25.5 RMSE 25.6 RMSE 25.7

Figure 21: Interpolation examples with piecewise smooth data.

This mapping is derived through reverse ray tracing (see Figure 23). For each pixel of the
rendering of the 3D scene, a ray of light is virtually traced in reverse. The light ray begins from the
eye of the observer, passes through a pixel of the screen, and continues into the 3D scene. The ray
may then intersect the plane or the sphere (or nothing, if the ray continues into the background).
The ray’s intersection point (X, Y, ) with the 3D scene determines the color of its pixel. The plane
is texture mapped as x = XmodM , y = ZmodN. The sphere is texture mapped as x = θ, y = ϕ,
where θ and ϕ are the spherical angles of the intersection.

16.4 Image rotation

Image rotation is another application that for a general rotation angle requires interpolation. The
choice of interpolation method significantly affects the quality of the output image. In this exper-

256
 Linear Methods for Image Interpolation

Texture Nearest Bicubic

Figure 22: Example of texture mapping.

Figure 23: Reverse ray tracing. A ray of light is traced from the observer into the 3D scene.

iment, an image is successively rotated by 5 degrees increments using several linear methods. The
initial image is rotated by 5 degrees, then the rotated image is rotated by another 5 degrees, and so
on (see results in Figure 24).

Acknowledgment
This material is based upon work supported by the National Science Foundation under Award
No.DMS-1004694. Any opinions, findings, and conclusions or recommendations expressed in this
material are those of the author(s) and do not necessarily reflect the views of the National Science
Foundation. Work partially supported by the MISS project of Centre National d’Etudes Spatiales,
the Office of Naval Research under grant N00014-97-1-0839 and by the European Research Council,
advanced grant “Twelve labours”.

Image Credits

Pascal Getreuer.

Standard test image.

257
 Pascal Getreuer

Nearest Bilinear Bicubic β 11 Nearest Bilinear Bicubic β 11

Figure 24: Rotated versions of the same image. Rotation angle increases from top to bottom and
from left to right. Rotation using nearest neighbor produces jagged artifacts along edges. Rotation
with bilinear is much better but blurs the image. Rotation with bicubic is sharper, but blurring is
still noticeable. Finally, the sharpest rotation among these results is with the 11th order B-spline β 11 .
By interpolating with a high-order B-spline, it closely approximates rotation via sinc interpolation.

References
[1] C.E. Shannon, “A Mathematical Theory of Communication”, Bell System Technical Journal,
vol. 27, pp. 379–423, 623–656, 1948.

[2] G. Strang and G. Fix, “A Fourier analysis of the finite element variational method”, Constructive
Aspects of Functional Analysis, Edizioni Cremonese, Rome, pp. 795–840, 1973.

[3] R. Keys, “Cubic Convolution Interpolation for Digital Image Processing”, IEEE Transactions
on Acoustics, Speech, and Signal Processing, 29(6), 1981. http://dx.doi.org/10.1109/TASSP.
1981.1163711.

[4] A. Aldroubi, M. Unser, and M. Eden, “Cardinal spline filters: stability and convergence to the
ideal sinc interpolator,” Signal Processing, vol. 28, no. 2, pp. 127–138, 1992.

258
 Linear Methods for Image Interpolation

[5] A. Schaum, “Theory and design of local interpolators”, CVGIP: Graphical Models and Image
Processing, vol. 55, no. 6, pp. 464–481, 1993. http://dx.doi.org/10.1006/cgip.1993.1035.

[6] M. Unser, “Splines: A Perfect Fit for Signal/Image Processing”, IEEE Signal Processing Maga-
zine, 16(6), pp. 22–38, 1999. http://dx.doi.org/10.1109/79.799930.

[7] P. Thévenaz, T. Blu, and M. Unser, “Interpolation Revisited,” IEEE Transactions on Medical
Imaging, vol. 19, no. 7, pp. 739–758, 2000.

[8] T. Blu, P. Thévenaz, and M. Unser, “MOMS: Maximal-Order Interpolation of Minimal Support”,
IEEE Transactions on Image Processing, vol. 10, no. 7, pp. 1069–1080, 2001. http://dx.doi.
org/10.1109/83.931101.

[9] E. Meijering, “A Chronology of Interpolation: From Ancient Astronomy to Modern Signal and
Image Processing”, In Proceedings of the IEEE, 90, pp. 319–342, 2002. http://dx.doi.org/10.
1109/5.993400.

[10] F. Malgouyres and F. Guichard, “Edge direction preserving image zooming: A mathematical
and numerical analysis”, SIAM Journal on Numerical Analysis, 39(1), pp. 1–37, 2002. http:
//dx.doi.org/10.1137/S0036142999362286.

[11] B. Fornberg, E. Larsson, N. Flyer, “Stable computations with Gaussian radial basis functions”,
SIAM Journal on Scientific Computing 33, pp. 869–892, 2011. http://dx.doi.org/10.1137/
09076756X.

259