Linear Filters
Hybrid Images, Oliva et al., [Link]
Image filtering
Hybrid Images, Oliva et al., [Link]
Announcements
• Assignment 1 will be uploaded this weekend you need to
submit it on teams by next Saturday 12:00 am.
Image transformations
• As with any function, we can apply operators to an image
g (x,y) = f (x,y) + 20 g (x,y) = f (-x,y)
• Today we’ll talk about a special kind of operator, convolution
(linear filtering)
Filters
• Filtering
– Form a new image whose pixel values are a combination of the original
pixel values
• Why?
– To get useful information from images
• E.g., extract edges or contours (to understand shape)
– To enhance the image
• E.g., to remove noise
• E.g., to sharpen and “enhance image”
– A key operator in Convolutional Neural Networks
Question: Noise reduction
• Given a camera and a still scene, how can you reduce noise?
Source: S. Seitz
Question: Noise reduction
• Given a camera and a still scene, how can you reduce noise?
Take lots of images and average them!
What’s the next best thing?
Source: S. Seitz
Image filtering
• Modify the pixels in an image based on some function of a
local neighborhood of each pixel
10 5 3 Some function
4 5 1 7
1 1 7
Local image data Modified image data
Source: L. Zhang
Linear filtering
• One simple version of filtering: linear filtering (cross-correlation,
convolution)
– Replace each pixel by a linear combination (a weighted sum) of its neighbors
• The prescription for the linear combination is called the “kernel” (or
“mask”, “filter”)
10 5 3 0 0 0
4 6 1 0 0.5 0 8
1 1 8 0 1 0.5
Local image data kernel Modified image data
Source: L. Zhang
Cross-correlation
Let be the image, be the kernel (of size 2k+1 x
2k+1), and be the output image
This is called a cross-correlation operation:
• Can think of as a “dot product” between local
neighborhood and kernel for each pixel
Convolution
• Same as cross-correlation, except that the kernel is “flipped”
(horizontally and vertically)
This is called a convolution operation:
• Convolution is commutative and associative
Convolution
Adapted from F. Durand
Mean filtering
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 20 10
0 0 0 90 90 90 90 90 0 0 0 20 40 60 60 60 40 20
=
0 0 0 90 90 90 90 90 0 0 0 30 60 90 90 90 60 30
*
1 1 1
1 1 1 0 0 0 90 90 90 90 90 0 0 0 30 50 80 80 90 60 30
0 0 0 90 0 90 90 90 0 0 0 30 50 80 80 90 60 30
1 1 1
0 0 0 90 90 90 90 90 0 0 0 20 30 50 50 60 40 20
0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 30 20 10
0 0 90 0 0 0 0 0 0 0 10 10 10 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
Mean filtering/Moving average
Mean filtering/Moving average
Mean filtering/Moving average
Mean filtering/Moving average
Mean filtering/Moving average
Mean filtering/Moving average
Linear filters: examples
0 0 0
* 0
0
1
0
0
0
Original
Source: D. Lowe
What image operation does filtering with this kernel
perform?
([0 0 0; 0 1 0; 0 0 0])
ⓘ Start presenting to display the poll results on this slide.
Linear filters: examples
0 0 0
=
* 0
0
1
0
0
0
Original Identical image
Source: D. Lowe
Linear filters: examples
0 0 0
* 1
0
0
0
0
0
Original
Source: D. Lowe
What image operation does filtering with this
kernel perform? ([0 0 0; 1 0 0; 0 0 0])
ⓘ Start presenting to display the poll results on this slide.
Linear filters: examples
0 0 0
=
* 1
0
0
0
0
0
Original Shifted left by 1 pixel
Source: D. Lowe
Linear filters: examples
1 1 1
=
* 1
1
1
1
1
1
Original Blur (with a mean filter)
Source: D. Lowe
Linear filters: examples
-
0 0 0 1 1 1
* 0
0
2
0
0
0
1
1
1
1
1
1
=
Sharpening filter
Original
(accentuates edges)
Source: D. Lowe
Sharpening
Source: D. Lowe
Smoothing with box filter revisited
Source: D. Forsyth
Gaussian kernel
Source: C. Rasmussen
Gaussian filters
= 1 pixel = 5 pixels = 10 pixels = 30 pixels
Mean vs. Gaussian filtering
Gaussian filter
• Removes “high-frequency” components from the image
(low-pass filter)
• Convolution with self is another Gaussian
* =
– Convolving twice with Gaussian kernel of width
= convolving once with kernel of width
Source: K. Grauman
Sharpening revisited
• What does blurring take away?
– =
original smoothed (5x5) detail
(This “detail extraction” operation is also
Let’s add it back: called a high-pass filter)
+α =
original detail sharpened
Photo credit: [Link]
Sharpen filter
+
blurred
image unit impulse
image
(identity kernel
with single 1 in
center, zeros
elsewhere)
scaled impulse Gaussian Sharpen filter
Sharpen filter
unfiltered
filtered
“Optical” convolution
Camera shake
= *
Source: Fergus, et al. “Removing Camera Shake from a Single Photograph”, SIGGRAPH 2006
Bokeh: Blur in out-of-focus regions of an image.
Source: [Link]
Filters: Thresholding
Linear filters
• Can thresholding be implemented with a linear filter?
Can thresholding be implemented with a
linear filter?
ⓘ Start presenting to display the poll results on this slide.
Questions?
