Image Transforms

Image transform
Definition: It is the operation to change the default
representation space of a digital image (spatial
domain -> another domain) so that:

• all the information present in the image is

preserved in the transformed domain, but
represented differently;

• the transform is reversible, i.e., we can revert to

the spatial domain
1-D DFT as basis expansion

Forward transform
real(A) imag(A)

u=0
Inverse transform

basis

u=7

n n
1-D DFT in matrix notations

real(A) imag(A)

u=0

N=8

u=7

n n
1-D DFT of different lengths

real(A) imag(A)
n
u

N=32
N=8

N=16 N=64
performing 1D DFT

real-valued input

Note: the coefficients in x and y on this slide are only meant for illustration purposes, which are not numerically accurate.
another illustration of 1D-DFT

real-valued input

Note: the coefficients in x and y are not numerically accurate

from 1D to 2D
1D 2D
Computing 2D-DFT

DFT

IDFT
2-D Fourier basis
real imag

real( ) imag( )
2-D FT illustrated
real-valued

real

imag
 Unitary Transform

A unitary transform of the image U is a rotation of

the MN-dimensional space (basis coordinates),
defined by a rotation matrix A in MN-dimensions.
Properties of unitary transform
 Given v  Au (A is a unitary transform)
v  u
2 2
 , i.e., energy-conserved
 a unitary transformation is a rotation of the basis
coordinates, or a rotation of the vector u in N-
Dimensional vector space
 Unitary transform tends to concentrate most of the
signal energy on a small set of transform
coefficients
 Properties of unitary transform
(cont)

 The transform coefficients tends to be decorrelated,

i.e., the off-diagonal elements of R v are approaching
zero

 The entropy of a random vector is preserved under a

unitary transformation (i.e., information preserved)
 Properties of unitary transform:
 v  E[v]  E[Au]  A E[u]  A u
R v  E[( v   v )( v   v )*T ]  A( E[( u   u )( u   u )*T ])A *T  A  R u  A *T

 2v ( k )  [R v ]k ,k  [AR u A*T ]k ,k
W 1 N 1

  v ( k )     v    A  A u    u ( n)
2 *T *T *T 2
v u
k 0 n 0

N 1 N 1


k 0
2
v ( k )  Tr[AR u A ]  Tr[R u ]    2u ( n )
*T

n 0

   
N 1 N 1

 E v( k )   E u( n )
2 2

k 0 n 0
2-D orthogonal and unitary
transforms
 Orthogonal series expansion for an NN image
u(m, n)
N 1 N 1

v(k, l)   u(m, n)a

m0 n0
k,l
(m, n) 0  k, l  N  1

N 1 N 1

u(m, n)   v(k,l)a
k 0 l0
*
k,l
(m, n) 0  m, n  N  1

 v(k, l)’s are the transform coefficients, V {v(k,l)}

represents the transformed image
 {a k ,l (m, n)} is a
set of orthonormal
functions, representing the image
transform
5-1
Orthonormality and
completeness
 {a (m, n)}must satisfies
k ,l

N 1 N 1

orthonormality :  a
m0 n0
k ,l
(m,n)a k*,l (m, n)   (k  k ,l  l)
N 1 N1

completeness : 
k 0 l0
ak ,l (m,n)a k*,l (m , n )   (m  m , n  n )

5-2
 Properties of Unitary Transform
transform
Consider v  Au, then R v  AR u (A* )t
Covariance
A 2  1 zero mean vector u is trasformed matrix
1 3 1  1 ρ 
v  u where Ru   
2 1 3 ρ 1 
1  3 ρ ρ 
then R v   2 2 
 ρ 1 3 ρ 
 2 2
 σ 2u (0)  1  σ 2u (1) ; total energy is equally distributed
but σ 2v (0)  1  3 ρ  σ 2v (1)  1  3 ρ
2 2
if ρ  0.95 , then 91% energy in v(0)
and the correlation between v(0) and v(1) is
Ε[v(0)v(1)]
ρ v (0,1)   0.83
σ v (0) σ v (1)
 energy compaction and decorrelation
Matrix representation of image
transform
N 1 N 1 N 1 N 1
 v(k, l)   u(m,n)a
m0 n0
k,l
(m, n) U  v(k,l)A
k 0 l0
*
k,l

N 1 N 1

u(m, n)  
k 0 l0
v(k, l)a k,l* (m, n) v(k,l)  U, A*k ,l 
N 1 N 1

  F,G   f (m, n)g (m, n)

m0 n0
*
is the matrix inner product
 Image U can be described as a linear combination
*
2 A
of N matrix k,l , k, l = 0,...,N-1
 A*k ,l are called the basis images
 v(k, l) can be considered as the projection of u(m, n) on
the (k, l)th basis image
5-3
 Basis images

5-4
2-D Separable image
transformation
 {a (m, n)} is separable
k ,l

a k ,l (m,n)  ak (m)bl (n)  a(k ,m)b(l, n)

N 1 N 1
v(k ,l)   a(k ,m)u(m, n)a(l, n)  V  AUAT
m0 n0
N 1 N 1
u(m, n)   a *
(k ,m)v(k ,l)a *
(l, n)  U  A *T
VA *

k 0 l 0

 A {a(k,m)} B  {b(l, n)} are unitary matrices

 Consider the above as “transforming columns of U by
A, then transforming rows of the result by AT
5-5
 Properties of DFT
 Symmetry F  F T  F 1  F *T (unitary)
 FFT needs O(N log N ) operations (DFT needs O(N 2 ) )
2

 Real DFT is conjugate symmetrical about N/2

N N
v* (  k)  v(  k)
2 2
 x 2 (n) is the circular convolution between h(n) and x1 (n)

DFT{x 2 (n)} N  DFT{h(n)} N DFT {x1 (n)} N

 Extend the length of h(n) ( N ) and x (n) (N) with zeros to have the
1

same length ( M  N   N  1 ), the above equation can be used to

compute linear convolution
1-D Discrete Cosine Transform
(DCT)
 N 1
  ( 2n  1) k 


Z ( k )  
n 0
z ( n )   ( k ) cos
 2N 


 z ( n) 
N 1
  ( 2n  1) k 
  Z ( k )   ( k ) cos 
 k 0 2N
1 2
 (0)  ,  (k ) 
N N
Transform matrix A
a(k,n) = (0) for k = 0
a(k,n) = (k) cos[(2n+1)/2N] for k > 0

• A is real and orthogonal

• rows of A form orthonormal basis
• A is not symmetric!
• DCT is not the real part of unitary DFT!
1-D DCT

1.0 1.0

0.0 0.0

-1.0 -1.0

1.0 1.0
z(n) 0.0 0.0 Z(k)

-1.0 -1.0
n
k
1.0 1.0
Original signal
0.0 0.0 Transform coeff.

-1.0 -1.0

1.0 1.0

0.0 0.0

-1.0 -1.0

Basis vectors
DFT and DCT in Matrix Notations
Matrix notation for 1D transform

1D-DCT 1D-DFT

N=32 A real(A) imag(A)

From 1D-DCT to 2D-DCT

u=0

u=7

n=7

 Rows of A form a set of orthonormal basis

 A is not symmetric!
 DCT is not the real part of unitary DFT!
basis images: DFT (real) vs DCT
Periodicity Implied by DFT and
DCT
Properties of DCT
 DCT is real and orthogonal
 C  C and C  C
* 1 T

 DCT ≠Real {DFT}

 For highly correlated data, DCT has good energy

compaction.
Sine transform
 DST (discrete sine transform)
2  (k  1)(n  1) ,
 (k, n)  sin 0  k, n  N  1
N 1 N 1
 Properties
 It is real, symmetric, and orthogonal.

 DST and DST-1 are the same in the form of DCT

 DST≠Imagery {DFT}
Walsh Transform
Here we calculate the matrix of Walsh coefficients
Here we calculate the matrix of Walsh coefficients
Here we calculate the matrix of Walsh coefficients
Symmetry of Walsh
Two-Dimensional Walsh Transform
Properties of Walsh Transforms
Here is the separable 2-Dim Inverse Walsh
Example for N=4
 odd

even
Discuss the importance of this figure
Hadamard Transform
separabl
Example of calculating Hadamard coefficients –
Standard Trivial Functions for Hadamard

One change

two changes
Discrete Walsh-Hadamard transform

1) Hadamard transform
1 1 1
H1  1  1
2  
H H n 1 
1  n 1 
H n  H n 1  H1 
2 H 
 n 1  H n 1 
# of sign changes
 sequency
1 1 1 1 0
 
1 1  1 1  1 3
H2 
2 1 1  1  1 1
 
1  1  1 1 2
 sequency
1 1 1 1 1 1 1 1 0
1 1 1 1 1 1 1  1 7

1 1 1 1 1 1  1  1 3
H 2 H2   
1    1 1 1 1 1 1 1  1 1 4
H3 
8 H  8 1 1 1 1 1 1  1  1 1
 2  H2   
1 1 1 1 1 1  1 1 6
1 1 1 1 1 1 1 1 2
 
1 1 1 1 1 1 1  1 5
 natural or Hadamard order
 also can be generated by sampling the Walsh function (1923, Walsh)
 Walsh - Hadamard transform
sequency
1 1 1 1 1 1 1 1 0
1 1 1 1 1 1 1  1 1

1 1 1 1 1 1 1 1 2
 
1 1 1 1 1 1 1 1  1 3
H3 
8 1  1 1 1 1 1 1 1 4
 
1  1 1 1 1 1 1  1 5
1  1 1 1 1 1 1 1 6
 
1  1 1 1 1 1 1  1 7
 sequency or Walsh order
Properties of Hadamard
transform
 Real, symmetric, and orthogonal  H  H *
 H T
 H 1

 Fast computation (only addition is needed)

 Can be decomposed into product of sparse matrix

 For highly correlated images, Hadamard transform also has

good energy compaction
Comparision

 DFT
• Fast transform; very useful in digital signal processing, convolution, filtering,
image analysis
• Good energy compaction; however – requires complex computations.
 DCT
• Fast transform and requires only real number operations
• The optimal alternative to the KLT for highly correlated images
• Used in compression and coding
• Excellent energy compaction
 Hadamard
• Faster than sinusoidal transforms since it only implies sums and subtractions
• Used for hardware implementation of some digital image processing algorithms
• Applied in image compression, filtering, coding
• Good energy compaction
KL transform of 1-D vector
 For a real N×1 random vector u with autocorrelation matrix
R, KL transform of u is defined as
v  Φ*T u u  Φ v
where Rφ k  k φk , 0  k  N  1 , i.e., φ k’s are orthonormalized
eigenvectors of R and φ k is the k-th column of 
  reduces R to its diagonal form, i.e.,
Φ RΦΛ  Diag{ }
*T
k

 If we replace R with R 0  R µ µ T, then eigenmatrix of R 0

is the KL transform of u-
 KL transform is independent of the image data themselves,
but related to its 2nd-order statistics
KL transform of 2-D image
 FFT, DCT, DST all are special cases of KL transform
 For an image u(m, n) of N×N pixels, its autocorrelation
function denoted as r(m, n;m,n) (or an N  N Rmatrix), 2 2

the basis images of the KL transform are k ,l (m, n) :

N 1
0  k, l  N  1

N1

r(m, n;m ,n 

) (m , n )  k ,l k ,l (m,n)
m0n0
k ,l
0  m,n  N  1

or Rψi  ψ
i i i  0,..., N 2  1
ψ i is the N 1 vector representation of  k ,l (m,n)
2
KL transform of 2-D image
(cont.)
 If R matrix is separable, then Ψ  {ψ } of N  N is also
i
2 2

separable
Kronecker
R  R1  R 2 Ψ Φ1 Φ2 operator

where Φj R jΦ*Tj  Λ j , j  1,2

 KL transform v  Ψ u or V  Φ1*T UΦ*2
*T

 :Advantages of this separability : reduce the eigenvalue

problem of R matrix ( N  N ) into two eigenvalue
2 2

problems of R1, R2 ( N  N )
Properties of the KL transform
 Decorrelation
{v(k ),k  0,..., N  1}are uncorrelated and have zero mean
  is not unique, not necessarily be the eigenmatrix of R

 Minimum basis restriction error

u A v Im w B z
NN 1≦m≦N NN

 Signal u is transformed via A to be v, clip the first m values by Im

to form w, and transformed back to z via B。The basis restriction
error is defined as

J m  E u(n)  z(n) 2 
N 1


1 1
 Tr[E{(u  z)(u  z)*T }]
N n0  N
Properties of the KL transform
(cont.)
 Find A KL and B KL (KL transform) such that J m is minimized for
each value of m [1, N ] . (KL is effective in compacting the energy)
 Select A  Φ*T and B  Φ , AB  I , and columns of  are sorted
according to their corresponding eigenvalues  0  1  2  .....   N 1
 Minimum transmission rate at a given distortion
 In a noisy channel, given a distortion specification,
1
D E[(u  u. )*T (u  u. )]
N
KL transform can, among all unitary transformation A (include A
= I ), achieve the minimum transmission rate 
R(Φ*T )  R(A)
u A v v. u.
(unitary) channel A*T
 An example of KL-transform
 A 21 vector u, whose covariance matrix is
1  
R ,  1
 1 
The KL transform is
1 1 1 
Φ Φ
*T
 
2 1 1
v  Φ u E{[v(0)] 2
}  0  1  , E{[v(1)]2 }  1  
1 1 1   )  max(0, 1 log 1  
R(Φ)  [max(0, log )]
2 2  2 
comparing R(I)  1 [2log ], 0   1
4
CCU, Taiwan
R(Φ)  R(I) Let  be small, say   1  
Wen-Nung Lie 5-26
Comparison of energy
distribution between transforms
 For first-order stationary Markov process with
large , DCT is comparable to KL
 With small , DST is comparable to KL
 For any , we can find a faster sinusoidal
transform (DCT or DST) to substitute for the
optimal KL transform which needs covariance
matrix to compute transform basis vectors