IDC410

A course on Image Processing and

Machine Learning
(Lecture 04)

Shashikant Dugad,
IISER Mohali

Shashikant R Dugad, IISER Mohali 1

 rs
lt e
F i

Shashikant R Dugad, IISER Mohali 2

 Filters

● Purpose: To reduce the noise in the image

● Improve the visibility of all features of the image
○ Brightness
○ Contrast

● Types of filters:
○ Mean Filter, Median Filter, Gaussian Filter, Convolution Filter ….

● Concepts of filters used neural network

● Purpose: Reduce noise, extract features, create images that are rich
in feature of interest, reduce size etc.

Shashikant R Dugad, IISER Mohali 3

 Noise types

● Images are often corrupted by random variations in intensity values, called

noise.
● Some common types of noise are salt and pepper noise, impulse noise, and
Gaussian noise.
○ Salt and pepper noise contains random occurrences of both black and white
intensity values.

○ Impulse noise contains only random occurrences of white intensity values.

○ Gaussian noise contains variations in intensity that are drawn from gaussian or
normal distribution and is a very good model for many kinds of sensor noise, such
as the noise due to camera electronics

Shashikant R Dugad, IISER Mohali 4

 (a)

(c) (d)
Images
Figure corrupted by salt of
4.5: Examples and pepper,
images impulse,byand
corrupted saltGaussian noise. impulse,
and pepper, (a) & (b)
Original images. (c) Salt and pepper noise. (d) Impulse noise. (e) Gaussian noise.
and Gaussian noise. (a) & (b) Original images. (c) Salt and pepper noise.
Shashikantnoise.
R Dugad, IISER Mohali 5
(d) Impulse noise. (e) Gaussian
 Mean Filter
● The mean filter is implemented by a local averaging operation where the value of
each pixel is replaced by the average of all the values in the local neighbourhood
where M is the total number of pixels in the neighbourhood.

● If f(i,j) is an initial image and h(i, j) is the final image after applying mean filter of
say 3 x 3 neighbourhood then, h(i, j) can be obtained as,
#&% (&%
1
ℎ 𝑖, 𝑗 = ( ( 𝑓(𝑘, 𝑙)
9
!"#$% '"($%
Mean Filter Smoothening Filter
1/9 1/9 1/9 1/16 1/8 1/16
1/9 1/9 1/9 1/8 1/4 1/8
1/9 1/9 1/9 1/16 1/8 1/16

Shashikant R Dugad, IISER Mohali 6

 t

r r
t

I I
L Mean
L Filter IResult
r
4.3. LINEAR FILTERS 121
I L L
I ___ .._ ___.._ 1 1

L L
I ___.._ 1

Original Image

I
MF: 3 x 3 L MF: 7 x 7
L
I ___.._ 1

I I
I
, ,
,
J J J
MF: 5 x 5
Figure Figure
4.7: The Figure
4.7:results
The of4.7:
a 3 The
results xof3,aresults3, of
53 xx 5, 5 xa 35,
and 7 xxand
3,7 5mean
7x x5,7and 7 on
filter
mean xfilter
7the
mean filter
noisy
on on the noisy
the noisy
Shashikant R Dugad, IISER Mohali 7
imagesimages fromimages
from Figure 4.5. from
Figure 4.5. Figure 4.5.
 Median Filter

● Local averaging operations tend to blur sharp discontinuities in intensity values

in an image.

● An alternative approach is to replace each pixel value with the median of the gray
values in the local neighbourhood.

● Filters using this technique are called median filters.

● Median filters effective in removing salt and pepper and impulse noise while
retaining image details because they do not depend on values which are
significantly different from typical values in the neighbourhood.

Shashikant R Dugad, IISER Mohali 8

 Median Filter Implementation
124 CHAPTER 4. IMAGE FILTERING

● Median filters approach is more like

Sorted by pixel value
mean filter .... " 10
-"-
75 99 36 - . .... .... '- .... .... -', 19
● For example, take a 3 x 3 window 75 99 36 22
38 49 10 38 9 rm 36
and compute the median of the .... ,-
19 \9g,l'12
"-
38 -+- Median value
22 ......
19 98 ..... .... '"
pixels in each window cantered
"\1\
.... ,-
'\"'
49
"

\ \
'" 75
around [i, j] f\
\ 98
38 99
● Sort the pixels into ascending order \

by gray level. New pixel value

● Select the value of the middle pixel Figure 4.9: An example illustrating the median filter using a 3 x 3 neighbor-
hood.
as the new value for pixel [i,j].
where the Gaussian spread parameter (J determines the width of the Gaus-
sian. For image processing, the two-dimensional zero-mean discrete Gaussian
function,
(4.12)
Shashikant R Dugad, 9
is used asIISER Mohalifilter. A plot of this function is shown in Figure 4.11.
a smoothing
 --. --. --Median Filter Results
.,-.,-.,-
4.3. LINEAR FILTERS 121

. . II' II' II'

j
--. :--. :
j
j
" "
"
. .
..
--.
.. ..
:
.. ..
.. .. ..
..
.." .." .." Original Image

I
MF: 3 x 3 L MF: 7 x 7
L
I ___.._ 1

MF: 5 x 5
Figure
Figure Figure
4.10: 4.10:
The The
4.10:results
The of aresults
results 3,a of
53 xa5,
3 xofShashikant3 xand
3, 5 3, 57 xxand
x 5, 5, and
7 x 77 median
7 median x filter
7 median filter
filter
on the on on the
the
noisy noisy
noisy 10
R Dugad, IISER Mohali
images
imagesimages
from fromfrom
Figure 4.5.Figure
Figure 4.5. 4.5.
 Gaussian Filter
● Gaussian filters are a class of smoothening filters where, the weights are chosen according
to the shape of a Gaussian function. The Gaussian smoothing filter is a very good filter for
removing noise drawn from a normal distribution.!

● The zero-mean Gaussian function in 1-dimension and 2-dimension are:

𝒙𝟐 𝒊𝟐& 𝒋𝟐
! !
𝒈 𝒙 = 𝒆 𝟐𝝈𝟐 , 𝒈 𝒊, 𝒋 = 𝒆 𝟐𝝈𝟐

● Gaussian filter properties: a) Rotational symmetry, b) Single lobe, c) Spread of smoothening

governed by sigma, d) Separable.
𝒎 𝒏 𝒎 𝒏
𝒌𝟐 )𝒍𝟐
(
𝒈 𝒊, 𝒋 ★𝒇 𝒊, 𝒋 = ( ( 𝒈 𝒌, 𝒍 ∗ 𝒇 𝒊 − 𝒌, 𝒋 − 𝒍 = ( ( 𝒆 𝟐𝝈𝟐 ∗ 𝒇 𝒊 − 𝒌, 𝒋 − 𝒍
𝒌#𝟏 𝒍#𝟏 𝒌#𝟏 𝒍#𝟏

𝒌𝟐 𝒍𝟐
( (
= ∑𝒎
𝒌#𝟏 𝒆 𝟐𝝈𝟐 ∑𝒏𝒍#𝟏 𝒆 𝟐𝝈𝟐 ∗ 𝒇 𝒊 − 𝒌, 𝒋 − 𝒍

Shashikant R Dugad, IISER Mohali 11

 J
f
l .r
Gaussian Filter (c)
e 4.14: The approximation of a Gaussian filter using the fifth row of
l's triangle. (a) Original noisy image. (b) Result after smoothing in the
● Pascal Tringles (coefficient of following equation) can be used for small size filters
ontal direction. (c) Final result after smoothing in the
𝒏
vertical
𝒏
direction.
𝒏 𝒏(𝒓 𝒓
(𝒙 + 𝒚) = ∑𝒓#𝟎 𝑪𝒓 𝒙 𝒚

● For n=1: [1, 1], n=2: [1,2,1], n=4: [1, 4, 6, 4, 1]

● Seperatibility property can be used to apply two dimensional filter using 1-dimensional
choosing a value 134 CHAPTER 4. IMAGE FILTERIN
filter for (72,we can evaluate it over an n x n window to
n a kernel, or mask, for which the value at [0,0] equals 1. For example,
ng (72 =●2 and
Fornlarge
= 7, n,
thegaussian function yields
above expression can
Now,bebyused:
the n=7, σthe
array:
multiplying
2=2
rest of the weights by k, we obtain
[i,j] -3 -2 -1 o 1 2 3 [i,j] -3 -2 -1 0 1 2 3
-3 .011 .039 .082 .105 .082 .039 .011 -3 1 4 7 10 7 4 1
-2 .039 .135 .287 .368 .287 .135 .039 -2 4 12 26 33 26 12 4
-1 .082 .287 .606 .779 .606 .287 .082 -1 7 26 55 71 55 26 7
o .105 .368 .779 1.000 .779 .368 .105 0 10 33 71 91 71 33 10
1 .082 .287 .606 .779 .606 .287 .082 1 7 26 55 71 55 26 7
2 .039 .135 .287 .368 .287 .135 .039 2 4 12 26 33 26 12 4
3 .011 .039 .082 .105 .082 .039 .011 3 1 4 7 10 7 4 1
Shashikant R Dugad, IISER Mohali 12
ever, we desire the filter weights to be integerThis is the
values forresulting convolution mask for the Gaussian filter (also shown
ease in compu-
 Result of Gaussian Filter
~~J'
I ~
~
!
~~

....

t
I

(a) (b)
The results
Figure 4.16: of
Thesmoothing
results ofusing the 7x7 using
smoothing Gaussian
the mask.
7 x 7 (a) Originalmask.
Gaussian image(a)
corruptedimage
Original by Gaussian noise.by
corrupted (b)Gaussian
Smoothed noise.
image. (b) Smoothed image.
Shashikant R Dugad, IISER Mohali 13
 Linear linear
• One simple version: Filtering
filtering
(cross-correlation, convolution)
● Linear Filtering: Replace each pixel by a linear combination of its neighbours and
self – Replace each pixel by a linear combination of its
neighborsconvolution
○ cross-correlation,

● • The prescription
The prescription for the linear
for the linear combination combination
is called the “kernel” (oris“mask”,
“filter”) called the “kernel” (or “mask”, “filter”)

10 5 3 0 0 0
4 6 1 0 0.5 0 8
1 1 8 0 1 0.5

Local image data kernel Modified image data

● Same Kernel applied across entire image are referred as Space Invariant Filters
Source: L. Zhang
Shashikant R Dugad, IISER Mohali 14
 Cross-correlation
Cross-correlation
●
Let be the image, be the kernel (of
Let F be the input image and H be the kernel (of size 2k+1 x 2k+1), and G
Cross-correlation
be thesize 2k+1
output x 2k+1),
image and be the output
image
Let be the image, be the kernel (of
size 2k+1 x 2k+1), and be the output
image

● This is This isacalled

called a cross-correlation
cross-correlation operation:operation:
This is called a cross-correlation operation:

Shashikant R Dugad, IISER Mohali 15

 Convolution
Convolution
Convolution
● • Same as cross-correlation,
Same as cross-correlation, except
except that the kernelthat the
is “flipped” (horizontally
• Same
and vertically) as cross-correlation,
kernel except and
is “flipped” (horizontally thatvertically)
the
kernel is “flipped” (horizontally and vertically)

This is called a convolution operation:

● This is called aThis
convolution
is called aoperation:
convolution operation:

• Convolution is commutative and associative

• Convolution is commutative and associative
● Convolution is commutative and associative

Shashikant R Dugad, IISER Mohali 16

 Convolution

0 0 0

*
Linear filters: =
0 1 0 examples
0 0 0

Original Identical image

0 0 0

* 1
0
0
0
0
0
=
Source: D. Lowe

Original Shifted left

Shashikant R Dugad, IISER Mohali By 1 pixel 17
 Convolution

1 1 1

*
Linear filters:
1 1 1 =
examples
1 1 1

Original Blur (with a mean filter)

-
0 0 0 1 1 1

* 0
0
2
0
0
0
1
1
1
1
1
1
=
Source: D. Lowe

Sharpening filter
Original (accentuates edges)

Shashikant R Dugad, IISER Mohali 18

 Properties of Convolution Filter
● Notation: b = c ★ a
● Convolution is a multiplication-like operation
○ Commutative: a ★ b = b ★ a
○ Associative: a ★ (b ★ c) = (a ★ b) ★ c
○ Distributes over addition: a ★ (b + c) = (a ★ b) + (a ★ c)
○ Scalars factor out: αa ★ b = a ★ αb = α(a ★ b)
○ Identity: unit impulse e = […, 0, 0, 1, 0, 0, …]: a ★ e = a

● Usefulness of associativity
○ Often apply several filters one after another: (((a ★ b1) ★ b2) ★ b3)
○ This is equivalent to applying one filter: a * (b1 ★ b2 ★ b3)

Shashikant R Dugad, IISER Mohali 19

 Padding and Strides
● In Convolutional Neural Networks (CNNs), stride refers to the step size by which the
filter/kernel moves across the input image during the convolution operation in
horizontal and vertical direction
○ Stride defines how big of steps filters should take (i.e., how many pixels our filters should
skip ) while sliding over the image
○ Minimum value of stride = 1
○ Smaller strides (1 or 2) offer detailed feature extraction, while larger strides (3+) aid in
down-sampling.

● CNN is like a collection of small, overlapping magnifying glasses

called filters. These filters scan over different parts of a image to find interesting
features, like edges, shapes, or colours. These filters slide or convolve over the
entire image as defined by the stride.

● The kernel size, stride value can substantially reduce the size of output im the
output size and computational efficiency of the network, influencing feature
extraction and spatial dimensions of image.
Shashikant R Dugad, IISER Mohali 20
Movement of Filter

Stride = 1

Stride = 2

Shashikant R Dugad, IISER Mohali 21

 Image Padding
Output Image (7 x 7) [Pad = 1]
Input Image (5 x 5)
0 0 0 0 0 0 0
22 145 23 167 67 0 22 145 23 167 67 0
● Padding means adding extra columns
45 110 45 119 29 Pad = 1 0 45 110 45 119 29 0 and rows with ZERO pixel intensity
78 99 78 88 112 0 78 99 78 88 112 0
before doing any operations.
145 100 99 38 164 0 145 100 99 38 164 0
223 110 23 45 29 0 223 110 23 45 29 0
0 0 0 0 0 0 0 ● Helps in keeping the spatial info
intact, particularly prevents data loss
Output Image (9 x 9) [Pad = 2] at the edges hence stabilises training
Input Image (5 x 5) 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
● It also helps to keep the output size
22 145 23 167 67 0 0 22 145 23 167 67 0 0
consistent with the input and makes
45 110 45 119 29 Pad = 2 0 0 45 110 45 119 29 0 0
78 99 78 88 112 0 0 78 99 78 88 112 0 0 training more stable.
145 100 99 38 164 0 0 145 100 99 38 164 0 0
223 110 23 45 29 0 0 223 110 23 45 29 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0

Shashikant R Dugad, IISER Mohali 22

 Padding and Strides
Padding & Stride in CNN
● Image Dimension: Nrow X Mcol
● Padding: P Strides: Srow, Scol. Dilation: Drow, Dcol

● Kernal Size: Krow, Kcol

𝑵𝒊𝒏
𝒓𝒐𝒘 4 𝟐 𝑿 𝑷 9 𝑫𝒓𝒐𝒘 𝑿 (𝑲𝒓𝒐𝒘 9𝟏) 9𝟏
𝑵𝒐𝒖𝒕
𝒓𝒐𝒘 = 𝑺𝒓𝒐𝒘
+1

𝑴𝒊𝒏
𝒄𝒐𝒍 4 𝟐 𝑿 𝑷 9 𝑫𝒄𝒐𝒍 𝑿 (𝑲𝒄𝒐𝒍 9𝟏) 9𝟏
𝑴𝒐𝒖𝒕
𝒄𝒐𝒍 = 𝑺𝒄𝒐𝒍
+1

padding=1, stride=2

Shashikant R Dugad, IISER Mohali 23