Digital Image Processing

COMP3317 Computer Vision

Dr. Kenneth Wong
 Image as a 2D Function
— Consider now only monochrome images
— An image can be regarded as a (continuous) 2D light-intensity function,
denoted by I(x, y)
o x
I

y x
— The value of I at spatial coordinates (x, y) gives the intensity (or gray
level) of the image at that point, which depends on
— The amount of source light incident on the scene being viewed
— The amount of light reflected by the objects in the scene

2
 Digitizing an Image
— When both the spatial coordinates (x, y) and the intensity values are all
finite and discrete quantities, the image is called a digital image
— To convert an image into digital form, the function I(x, y) has to be
sampled in both the coordinate and intensity values
— Digitizing the coordinate values is called sampling
— Digitizing the intensity values is called quantization

3
 Digital Image Representation
— A digital image can be represented as a N ´ M matrix I(i, j) of intensity
values
é I 0, 0 I 0,1  I 0, M -1 ù
ê I I1,1  I1, M -1 úú
I(i, j ) = ê
1, 0

ê     ú
ê ú
ë I N -1,0 I N -1,1  I N -1, M -1 û

— The image is said to have a spatial resolution of M´N

— Each element of the matrix is called a picture element or pixel
— Typically, each pixel is represented by an 8-bit value, giving rise to
28 = 256 different gray levels (i.e., 0, 1, 2, …, 255)
— The image is said to have a gray-level resolution of 256, and is called an
8-bit image

4
 Quality of a Digital Image
— The quality of a digital image is determined to a large degree by
— the number of samples used in sampling (i.e., spatial resolution) and
— the number of discrete gray levels used in quantization (i.e., gray-level
resolution)
— When there is an insufficient number of samples, there will be the
problem of sampling checkerboards
— When there is an insufficient number of gray levels, there will be the
problem of false contouring

5
 Quality of a Digital Image
— The problem of sampling checkerboards:

1024×1024 512×512 256×256

128×128 64×64 32×32

6
 Quality of a Digital Image
— The problem of false contouring:

8 bits 7 bits 6 bits 5 bits

4 bits 3 bits 2 bits 1 bit

7
 Neighbors of a Pixel
— Consider a pixel p at coordinates (x, y)

— The 4-neighbors of p, denoted by N4(p), are the

pixels at coordinates (x+1, y), (x–1, y), (x, y+1), and p
(x, y–1) respectively
N4(p)

— The 4 diagonal neighbors of p, denoted by ND(p),

are the pixels at coordinates (x–1, y–1), (x+1, y–1), p
(x+1, y+1), and (x–1, y+1) respectively
ND(p)

— The 8-neighbors of p, denoted by N8(p), are given

by the union of the 4-neighbors and the diagonal p
neighbors of p, i.e., N8(p) = N4(p) È ND(p)
N8(p)
8
 Adjacency
— Let V be the set of gray-level values used to define adjacency
V = {0,1}
— 2 pixels p and q with values from V are 4-adjacent 3 1 0
if q is in the set N4(p) 2 0 3
2 3 1
— 2 pixels p and q with values from V are 8-adjacent 3 1 0
if q is in the set N8(p) 2 0 3
2 3 1
— 2 pixels p and q with values from V are m-adjacent 3 1 0
(mixed-adjacent) if
2 0 3
— q is in the set N4(p), or
2 3 1
— q is in set ND(p) and the set N4(p) Ç N4(q) has no
pixels whose values are from V

9
 Adjacency
— 2 image subsets S1 and S2 are adjacent if some pixel in S1 is adjacent to
some pixel in S2

S1 S2
2 2 3 3 3 4 5 0 1 0 4-adjacent?
0 2 4 1 5 5 1 2 3 1 8-adjacent?
1 3 5 1 5 0 1 4 2 0
m-adjacent?
4 3 1 1 1 3 4 5 6 0
3 2 1 0 0 1 1 1 1 0

V = {0, 1}

10
 Path
— A path from pixel p with coordinates (x, y) to pixel q with coordinates
(s, t) is a sequence of distinct pixels with coordinates (x0, y0), (x1, y1), …,
(xn, yn) where (x0, y0) = (x, y), (xn, yn) = (s, t), and pixels (xi, yi) and
(xi-1, yi-1) are adjacent for 1 £ i £ n
— The length of the path is n
— The path is said to be closed if (x0, y0) = (xn, yn)

11
 Path
— Example: Find the shortest path from pixel p to pixel q

3 1 2 1 q shortest 4-path?
shortest 8-path?
2 2 0 2
shortest m-path?
1 2 1 1

p 1 0 1 2

V = {0, 1}

12
 Connectivity
— Let S represents a subset of pixels in an image
— 2 pixels p and q are said to be connected in S if there exists a path
between them consisting entirely of pixels in S
— For any pixel p in S, the set of pixels that are connected to it in S is called
a connected component of S
— If S has only one connected component, it is called a connected set

13
 Connectivity
— Example: Find the number of connected components in S

S
2 2 3 3 3 4 5 0 1 0 4-adjacency?
0 2 4 1 5 5 1 2 3 1 8-adjacency?
1 3 5 1 5 0 1 4 2 0
m-adjacency?
4 3 1 1 1 3 4 5 6 0
3 2 1 0 0 1 1 1 1 0

V = {0, 1}

14
 Distance Measures
— Consider pixels p, q, and z, with coordinates (x, y), (s, t), and (v, w),
respectively z

p
— D is a distance function (or metric) if
— D(p, q) ³ 0 (D(p, q) = 0 iff p = q),
— D(p, q) = D(q, p), and
— D(p, z) £ D(p, q) + D(q, z)

15
 Distance Measures
— Euclidean distance:
De(p, q) = sqrt[(x – s)2 + (y – t)2]
— City-block distance (D4 distance):
D4(p, q) = |x – s| + |y – t|
— Chessboard distance (D8 distance ):
D8(p, q) = max(|x – s| , |y – t|)
— Dm distance
— length of the shortest m-path between 2 pixels

2 2 2 2 2 2
2 1 2 2 1 1 1 2
2 1 0 1 2 2 1 0 1 2
2 1 2 2 1 1 1 2
2 2 2 2 2 2
city-block distance chessboard distance
16
 Logic Operations
AND OR NOT
0 0 0 0 0 0 0 1
0 1 0 0 1 1 1 0
1 0 0 1 0 1
1 1 1 1 1 1

— Only need to consider AND, OR and NOT logic operators, as any other
operator can be implemented by using only these three basic functions
— Operate on a pixel-by-pixel basis
— For grayscale images, pixel values are processed as strings of binary
numbers

17
 Negative Transformation using NOT
— NOT operator performs the same function as the negative
transformation
— Example: consider an 8-bit pixel with a gray level of 127(x), its
corresponding binary string is thus 01111111(ii), and the result of the
NOT operator will be 10000000(ii) = 128(x)

0 1 2 253 254 255

ß ß ß … … … ß ß ß
NOT 255 254 253 2 1 0

0 1
1 0
NOT

18
 Masking using AND/OR
— AND and OR operators are used for masking
— Masking is the process of selecting sub-images in an image, sometimes
is referred to as region of interest (ROI) processing
— Examples:
AND
0 0 0
0 1 0 AND =
1 0 0
1 1 1

OR
0 0 0
0 1 1 OR =
1 0 1
1 1 1

19
 Arithmetic Operations
— Addition, Subtraction, Multiplication & Division
— Performed on a pixel-by-pixel basis between 2 or more images
— Useful for image enhancement and segmentation

20
 Image Averaging using Addition and Division
— Commonly used for noise reduction by observing the same scene over a
long period of time
— Effective in removing zero-mean uncorrelated noise from an image
K -1
1
g ( x, y ) =
K
å g ( x, y )
i =0
i

K -1
1
=
K
å [ f ( x, y ) + n ( x, y ) ]
i =0
i

1 K -1 1 K -1
= å f ( x, y ) + å n ( x, y )
i
K i =0 K i =0

» f ( x, y )

21
 Image Subtraction
— To enhance the difference between 2 images
— Commonly used for change detection
— Rescaling may be needed to ensure all pixel values are within the
interval [0, L- 1]
L -1
g ' ( x, y) = {g ( x, y) - min[ g ( x, y)]}´
max[g ( x, y)] - min[ g ( x, y)]
— Example: background subtraction

- =
rescaling

— To extract the original object, create a mask using the difference image
and use AND/OR operator
22
 Gray-Level Masks using Multiplication
— Division can be considered as multiplying one image by the reciprocal of
the other
— Multiplying an image by a constant can increase / decrease the average
gray level of the image
— Multiplication can also be used to implement gray-level masks (as
compare to binary masks using AND and OR operators)

´ =

23
 Convolution
— The convolution of two functions f(x) and g(x), denoted by f(x)*g(x), is
defined as ¥
f ( x) * g ( x) = ò f (u ) g ( x - u )du
-¥
— Example:

f(x) g(x) f(x)*g(x)

1
* = 1
1/2 1/2
x x x
1 1 1 2

24
 Convolution
¥

— Example: f ( x) * g ( x) = ò f (u ) g ( x - u )du
-¥
g(u) f(u)

1
1/2
u u
1 1

g(-u) g(x-u)

1/2 1/2
u u
-1 1 -1 x 1

f(u) f(x)*g(x)

1 1
g(x-u)
1/2
u x
1 1 2
x 25
x
 Convolution
— Convolution in 2D:
¥ ¥
f ( x, y ) * g ( x, y ) = ò ò f (u, v) g ( x - u, y - v)dudv
- ¥- ¥

— Discrete cases:
M -1
1
f ( x) * g ( x) = å f (u) g ( x - u) where x = 0, 1, …, M-1
M u =0

M -1 N -1
1
f ( x , y ) * g ( x, y ) =
MN
åå f (u, v) g ( x - u, y - v)
u =0 v =0
where x = 0, 1, …, M-1
y = 0, 1, …, N-1
— Convolution is symmetric, i.e.,
f ( x, y) * g ( x, y) = g ( x, y ) * f ( x, y )
— Convolution is associative, i.e.,
[ f ( x, y) * g ( x, y)]* h( x, y) = f ( x, y) * [ g ( x, y ) * h( x, y)]

26
 Linear Spatial Filtering
— Linear spatial filtering in image processing
w(-1,-1) w(0,-1) w(1,-1)
is based on convolution, and is often referred
to as convolving a mask with an image w(-1,0) w(0,0) w(1,0)

— A mask (also known as filter or kernel) can be w(-1,1) w(0,1) w(1,1)

considered as a sub-image with some weighting
coefficients a 3´3 mask

— The center of the mask is moved from pixel

to pixel in an image and the response is 12 16 18 12

given by the sum of products of the mask 13 11 30 12

coefficients with the corresponding pixels
in the area spanned by the mask 15 15 16 14

a b
I 2 ( x, y ) = å å w(u, v) I ( x - u, y - v)
u = - av = - b
1
14 13 13 12

27
 Linear Spatial Filtering
— Problem occurs near the border of the
image when part of the mask is located
outside the image 12 16 18 12

— This problem can be avoided by limiting 13 11 30 12

the excursions of the center of the mask
15 15 16 14
so that the mask always stays completely
within the image, but this will result in a 14 13 13 12
smaller filtered image
— Alternative approaches in handling this problem include
— padding the image by zeros or replicating rows and columns; and
— employing a partial filter mask

28
 Smoothing Filters
— Smoothing filters are used for blurring and for noise reduction
— The response of a linear smoothing filter is simply the weighted average
of the pixel intensities contained in the neighborhood of the mask
— Examples: Linear smoothing filters

1 1 1 1 2 1

1/9 ´ 1 1 1 1/16 ´ 2 4 2
1 1 1 1 2 1
a 3´3 box filter a 3´3 weighted average filter

13 16.67 16.5 18 12 16 18 12 13 15.8 17.6 15.3

13.7 16.2 16 17 13 11 30 12 13.3 15.8 18.2 16.2

13.5 15.6 15.1 16.2 15 15 16 14 14 15 16 14.9

14.3 14.3 13.8 13.8 14 13 13 12 14.1 13.9 13.6 13.1

29
 Sharpening Filters
— Sharpening filters are used to highlight fine details in an image or to
enhance details that have been blurred
— Sharpening is accomplished by spatial differentiation
— Derivatives of a digital function can be
obtained by finite differences
𝜕𝐼
𝐼 𝑥 + 1, 𝑦 − 𝐼(𝑥, 𝑦) 𝐼 𝑥, 𝑦 − 𝐼(𝑥 − 1, 𝑦)
𝜕𝑥
𝜕!𝐼
𝐼 𝑥 + 1, 𝑦 + 𝐼(𝑥 − 1, 𝑦) − 2𝐼(𝑥, 𝑦)
𝜕𝑥 !
𝜕𝐼
𝐼 𝑥, 𝑦 + 1 − 𝐼(𝑥, 𝑦) 𝐼 𝑥, 𝑦 − 𝐼(𝑥, 𝑦 − 1)
𝜕𝑦
𝜕!𝐼
𝐼 𝑥, 𝑦 + 1 + 𝐼(𝑥, 𝑦 − 1) − 2𝐼(𝑥, 𝑦)
𝜕𝑦 !
30
 Sharpening Filters
— Example: Laplacian filter
𝜕!𝐼 𝜕!𝐼
∇! 𝐼 = !+ !
𝜕𝑥 𝜕𝑦
= [𝐼 𝑥 + 1, 𝑦 + 𝐼 𝑥 − 1, 𝑦 − 2𝐼 𝑥, 𝑦 ] +
[𝐼 𝑥, 𝑦 + 1 + 𝐼 𝑥, 𝑦 − 1 − 2𝐼 𝑥, 𝑦 ]
= 𝐼 𝑥 + 1, 𝑦 + 𝐼 𝑥 − 1, 𝑦 +
𝐼 𝑥, 𝑦 + 1 + 𝐼 𝑥, 𝑦 − 1 − 4𝐼(𝑥, 𝑦)

isotropic mask isotropic mask

for rotations in for rotations in
increments increments
of 90o of 45o

31
 Composite Laplacian Masks
— To preserve the background features in the image, the Laplacian image is
subtracted from / added to the original image to produce the enhanced
image:
If the center coefficient of the
ì I ( x, y) - Ñ 2 I ( x, y) Laplacian mask is negative
g ( x, y) = í
îI ( x, y) + Ñ I ( x, y)
2 If the center coefficient of the
Laplacian mask is positive

— Example:
0 0 0 1 1 1 -1 -1 -1
0 1 0 1 -8 1 -1 9 -1
0 0 0 1 1 1 -1 -1 -1

- =

32
 Nonlinear Smoothing Spatial Filters
— Median filters replace the value of a pixel by the median of the pixel
intensities in the neighborhood of that pixel
— Particularly effective in the presence of impulse noise, also called salt-
and-pepper noise because of its appearance as white (salt) and black
(pepper) dots superimposed on an image
— Produce less blurring than linear smoothing filters of similar size
— Example:

input output of an output of a

averaging filter median filter 33
 Order-Statistics Filters
— Median filter is a kind of order-statistics filters whose response is based
on the ordering or ranking of the pixel intensities contained in the image
area spanned by the filter
— Other examples:
— Max filter: fˆ ( x, y) = max {g (s, t )}
( s ,t )ÎS
xy
— Useful for finding the brightest points in an image, and for reducing
pepper noise
— Min filter: fˆ ( x, y ) = min {g ( s, t )}
( s ,t )ÎS xy
— Useful for finding the darkest points in an image, and for reducing salt
noise
1
— Midpoint filter: fˆ ( x, y ) = éê max {g ( s, t )} + min {g ( s, t )}ùú
2 ë( s ,t )ÎS xy ( s ,t )ÎS xy û

— Works best for randomly distributed noise, e.g., Gaussian noise or

uniform noise

34
 Color Models
— Human eye perceive all colors as variable
combinations of the 3 primary colors:
Red (R), Green (G), and Blue (B)
— The primary colors can be added to produce the
secondary colors
— Green + Blue → Cyan (C)
— Blue + Red → Magenta (M)
— Red + Green → Yellow (Y)
— Commonly used color models:
— RGB (for color monitors)
— CMY (for printers)
— YIQ (for color TV broadcast)
— HSI, HSV, …, etc.

35
 RGB Model
— Each color appears in its primary spectral components of red, green and
blue, i.e., (R,G,B)
— All values of R, G, and B are assumed to
be in the range [0, 1]
— Examples:
— (1, 0, 0) = pure red
— (0, 1, 0) = pure green
— (0, 0, 1) = pure blue
— Typically, each component is represented by 8 bits
— Hence each pixel can be presented by 8´3 = 24 bits (pixel depth), giving
rise to 224 = 16 million different colors
— A 24-bit RGB color image is called a full-color image
— The grayscale is given by equal portions of R, G, and B
36
 YIQ Model
— Used in commercial TV broadcasting
— RGB to YIQ conversion:
éY ù é0.299 0.587 0.114 ù é R ù
ê I ú = ê0.596 - 0.275 - 0.321ú êG ú
ê ú ê úê ú Converting a color image into
êëQ úû êë0.212 - 0.523 0.311 úû êë B úû
a monochrome image
— The Y component gives the luminance, whereas the I and Q
components provide the color information
— The Y component provides all the video information required by a
monochrome television set
— A color image can be easily converted into a monochrome image by
taking only the Y component of the above conversion
I(i, j) = 0.299´R(i, j) + 0.587´G(i, j) + 0.114´B(i, j)

37