Intensity Transformation and Spatial Filtering

Spatial domain

• Image plane
• Image processing methods based on direct manipulation of pixels
• Two principal image processing technique classifications
1. Intensity transformation methods
2. Spatial filtering methods

Background

• Spatial domain
– Aggregate pixels composing an image
– Computationally more efficient and require less processing resources for implementation
• Spatial domain processes denoted by the expression

g(x, y) = T [f (x, y)]

– f (x, y) is input image

– g(x, y) is output image
– T is an operator on f , defined over some neighborhood of (x, y)
– T may also operate on a set of images (adding two images)
• Neighborhood of a point (x, y)
– Square or rectangular subimage area centered at (x, y) (Figure 3.1)
∗ Typically, the neighborhood is much smaller than the image
– Center moves over each pixel in the image
– T is applied at each point to get g at that location
∗ Compute the average intensity of the neighborhood
– Also possible to have neighborhood approximations in the form of a circle
– The above application is also called spatial filtering
∗ Neighborhood may be extracted by a spatial mask, or kernel, or template, or window
– Handling pixels at image border
∗ Part of the neighborhood is outside the image frame
∗ Outside pixels can be ignored; replaced by a uniform gray scale; replaced by 0; or inner pixels can be reflected
outside
• Single pixel neighborhood, or point processing techniques
– Simplest form of T
– Smallest possible neighborhood of size 1 × 1
∗ g depends only on the value of f at a single point (r, c)
– Gray-level (or intensity) transformation function of the form

s = T (r)
Intensity Transformation and Spatial Filtering 2

∗ r and s denote the gray level of f (x, y) and g(x, y)

– Contrast stretching and thresholding (Figure 3.2)
• Larger pixel neighborhoods (mask processing or filtering)
– Use a neighborhood around (x, y) to determine the value of g(x, y)
– Neighborhood defined by masks, filters, kernels, templates, or windows (all refer to the same thing)
– A kernel is a small 2D array whose coefficients determine the nature of the process

Basic gray level transformations

• Pixel values denoted by r and s before and after transformation

• Transform denoted by s = T (r)
• Mapping performed by lookup tables (LUTs)
• Figure 3.3 – Basic transformations
• Image negatives
– Equivalent of a photographic negative
– Input gray level range: r = [0, L − 1]
– Transformation is given by s = (L − 1) − r
– Suited for enhancing white or gray detail in dark areas of image, specially when dark areas are dominant in size
(Figure 3.4)

• Flipping and flopping

– Simplest geometric transformation
– Reflect the image about a horizontal axis (flip) or a vertical axis (flop)

– Assume an image of size w columns and h rows

– Flip
x0 = x; y 0 = h − 1 − y
Intensity Transformation and Spatial Filtering 3

∗ Flip, forward mapping

g(h − 1 − r, c) = f (r, c)
∗ Flip, reverse mapping
g(r, c) = f (h − 1 − r, c)
– Flop
x0 = w − 1 − x; y 0 = y
∗ Flop, forward mapping
g(r, w − 1 − c) = f (r, c)
∗ Flop, reverse mapping
g(r, c) = f (r, w − 1 − c)
– Flip-flop is a flip operation followed by a flop
• Log transformations

– General form given by: s = c log(1 + r) (Figure 3-3)

∗ c is a constant
∗ r≥0
∗ Expand the range of dark pixels in the image
– Maps a narrow range of gray level values in input image to a wider range of output levels, or the other way round
with inverse log transform
– Log function compresses the dynamic range of images with large variation in pixel values
– Easiest way to generate log transforms is by using a lookup table, and scaling the input to the range [0,1]
– Example given by Fourier spectrum (Figure 3-5; input values in the range [0, 1.5 × 106 ])
• Power-law or gamma transformations

– General form given by: s = crγ (Figure 3-6)

∗ c and γ are positive constants
– General form may also be written as s = c(r + )γ to account for an offset
∗ Offset is useful for some measurable output when the input is zero
∗ Typically, an issue of display calibration and hence, normally ignored
– Fractional value of γ map a narrow range of dark input values into a wider range of output values; opposite is true
for higher values of input levels
– The transformation reduces to identity values when c = γ = 1
– Gamma-correction
∗ Different devices respond differently to pixel values according to power law
∗ Typical values of γ for CRTs is between 1.8 and 2.5
∗ Important if displaying the image accurately on a computer screen is of concern
· Images not properly corrected can look bleached out or too dark
– General-purpose contrast manipulation
∗ Playing with the gamma to enhance detail in a desired region (Figure 3.9)
• Piecewise-linear transformation functions
– Form of function can be arbitrarily complex
– Specification requires considerable user input
– Contrast stretching
Intensity Transformation and Spatial Filtering 4

∗ Image contrast could be low due to poor illumination, lack of dynamic range in the sensor, or wrong setting of
lens aperture during image acquisition
∗ Increase the dynamic range of gray levels in the image being processed to the full intensity range of recording
medium or display device
∗ Figure 3.10
∗ Thresholding function
– Gray-level slicing
∗ Figure 3-11
∗ Two approaches
1. Display in one value (white) all the values in the range of interest, and change to black everything else
2. Brighten or darken desired range of intensities leaving all other intensity levels unchanged
3. Figure 3-12
– Bit-plane slicing
∗ Figure 3-13
∗ Most changes can be captured in significant bits
∗ You may not be able to perceive minute changes reflected in low bits
∗ Figure 3-14
· See how the border is black in some cases and white in others (pixel value 194, or 1100 0010)
∗ Decomposition useful for image compression
∗ Combine some of the bit planes
∗ Figure 3-15

• Relationship between pixel intensities

– Most of the gray level transformations are monotonically increasing
– For the transformation s = T (r), we have
r1 < r2 ⇒ s1 < s2

– The rule has exceptions, such as the negative transformation where s = L − 1 − r

– Saturation: If the result of a transformation exceeds the maximum possible intensity level, the intensity is clamped
at the maximum (same for minimum being capped at 0)

s = max(0, min(L, T (r)))

– The image can be made brighter by multiplying each pixel by a small amount (and made darker by dividing each
pixel by a constant)
∗ Notice that the use of saturation arithmetic makes the operation irreversible
∗ Multiplying each pixel by 2 and then, dividing it by 2 will not yield the original pixel intensity
– Gain and bias
∗ Multiplication by a constant is known as gain (γ) while addition by a constant is adjustment of bias (β)

g(x, y) = γf (x, y) + β

∗ In older monitors, gain was used to adjust contrast while bias was used to adjust brightness or blacklevel
∗ If γ = 1, the operation reduced to just addition of a constant
∗ If β = 0, the operation reduces to just multiplication by a constant
∗ If γ = −1 and β = L − 1, the operation reduced to grayscale inversion (or image negative)
– Linear contrast stretch
Intensity Transformation and Spatial Filtering 5

∗ Maps gray levels between [g1 , g2 ] in the input image to [g10 , g20 ]

g20 − g10
g(x, y) = (f (x, y) − g1 ) + g10
g2 − g1

Histogram processing

• Discrete function h(rk ) = nk

– rk is the kth gray level
– nk is the number of pixels in the image at gray level rk
• Normalized histogram
– Normalize a histogram by dividing each value by total number of pixels in the image M N
– Given by: p(rk ) = nk /M N , for k = 0, 1, . . . , L − 1
– p(rk ) gives the probability of occurrence of gray level rk
– Sum of all components of a normalized histogram is 1
– If you have to create a plot, the histogram to be created is given by:
nk
p(k) =
max(n0 , n1 , . . . , nL−1 )
for k = 0, 1, . . . , L − 1
• Basis for many spatial processing techniques
– Simple to calculate in software and economic hardware implementations
– Can be used for image enhancement and to look at image statistics
– Figure 3-16
• Histogram equalization
– Method to increase the global contrast in images by better distributing the intensities in the histogram
∗ Areas of lower local contrast gain higher contrast
∗ Achieved by spreading out most frequent intensity values
– Consider a continuous function, with r being the gray levels of the image to be enhanced
– Let r be in the interval [0, L − 1] such that r = 0 represents black and r = L − 1 represents white
– Consider the transformation of the form

s = T (r) 0≤r ≤L−1

to produce a level s for every pixel value r in the original image

– Let T (r) satisfy the following conditions
1. T (r) is single-valued and monotonically increasing in the interval 0 ≤ r ≤ L − 1
2. 0 ≤ T (r) ≤ L − 1 for 0 ≤ r ≤ L − 1
3. Figure 3-17
∗ Monotonic vs strictly monotonic
∗ Strictly monotonic allows for the existence of inverse
– Single-valued requirement guarantees the existence of inverse transform
– Monotonicity preserves the increasing order of black to white in output image, preventing artifacts generated by
reversals of intensity
Intensity Transformation and Spatial Filtering 6

– Second condition guarantees that gray levels are preserved in output image
– Inverse transform from s back to r is denoted

r = T −1 (s) 0≤s≤L−1

∗ T −1 may fail to be single valued unless we require T (r) to be strictly monotonically increasing in the interval
[0,L-1]
– Gray values in an image may be viewed as random variables in the interval [0, L − 1]
– Probability density function
∗ Used to describe a random variable
∗ Let pr (r) and ps (s) denote the probability density functions of random variables r and s, respectively
· pr and ps are different functions
∗ If pr (r) and T (r) are known, and T (r) is continuous and differentiable over the range of values of interest, the
PDF of the transformed variable s can be obtained using the simple formula

dr
ps (s) = pr (r)
ds
∗ A transformation function of importance in image processing is:
Z r
s = T (r) = (L − 1) pr (w)dw
0

where w is a dummy variable of integration

· The integral gives the cumulative distribution function (CDF) of random variable r
· Since PDFs are positive, the area under the function cannot decrease as r increases
· At r = L − 1, the integral evalutes to 1 (the area under a PDF curve is always 1)
∗ Finding ps (s) corresponding to the transformation
· Leibniz’s rule: Derivative of a definite integral with respect to its upper limit (L − 1) is the integrand
evaluated at the limit
ds dT (r)
=
dr dr Z r 
d
= (L − 1) pr (w)dw
dr 0
= (L − 1)pr (r)

· Substituting the result in the equation for the PDF of transformed variable, we get
dr
ps (s) = pr (r)
ds
1
= pr (r)
(L − 1)pr (r)
1
= 0≤s≤L−1
L−1
· The form of ps (s) is a uniform PDF, or performing the intensity transformation yields a random variable
s, characterized by a uniform PDF
· T (r) depends on pr (r) but the resulting ps (s) always is uniform, regardless of the form of pr (r)
· Figure 3.18
– For discrete values for an image of size n = M × N pixels, we have
nk
pr (rk ) = k = 0, 1, 2, . . . , L − 1
n
Intensity Transformation and Spatial Filtering 7

– The discrete transformation function is given by

k
X
sk = T (rk ) = (L − 1) pr (rj )
j=0
k
X nj
= (L − 1) k = 0, 1, 2, . . . , L − 1
j=0
n

– Processed image obtained by mapping each pixel with level rk into a corresponding pixel with level sk
∗ Histogram equalization or histogram linearization
– Transform spreads the histogram of the input image so that the levels of histogram-equalized image span a fuller
range of gray scales
– Discrete form does not guarantee uniform distribution of probability density function
– Advantages
∗ More dynamic range of gray level distribution
∗ Fully automatic algorithm; no parameters need be specified
∗ Simple to compute
– Example: Consider a 3-bit image (8 intensity levels) of size 64 × 64 pixels with the intensity distribution as follows:
rk nk pr (rk ) = nk /n
0 790 0.19
1 1023 0.25
2 850 0.21
3 656 0.16
4 329 0.08
5 245 0.06
6 122 0.03
7 81 0.02
∗ Histogram in Figure 3.19a
∗ Values of histogram transformation are obtained by (last column shows the value rounded to nearest integer)
P0
s0 = T (r0 ) 7 j=0 pr (rj ) = 7pr (r0 ) 1.33 1
P1
s1 = T (r1 ) 7 j=0 pr (rj ) = 7(pr (r0 ) + pr (r1 )) 3.08 3
s2 = T (r2 ) 4.55 5
s3 = T (r3 ) 5.67 6
s4 = T (r4 ) 6.23 6
s5 = T (r5 ) 6.65 7
s6 = T (r6 ) 6.86 7
s7 = T (r7 ) 7.00 7
∗ Transformation function and equalized histogram in Figure 3.19 b and c
∗ Some intensity levels are mapped to a single intensity level, hence fewer intensity levels in the equalized
histogram
∗ Intensity levels of the equalized histogram span a wider range of intensity scale, leading to contrast enhance-
ment
∗ Perfectly flat histograms
· Not possible because of two reasons
1. Histograms are approximations to PDFs
2. We cannot create new intensity levels when we perform histogram equalization because of discrete
values
· Spreading the histogram of input image results in a wider span of intensities for equalized image
Intensity Transformation and Spatial Filtering 8

– Inverse transform from s to r is denoted by

rk = T −1 (sk ) k = 0, 1, 2, . . . , L − 1

∗ Satisfies the two conditions outlined earlier only if none of the levels rk , k = 0, 1, 2, . . . , L − 1 are missing
from input image (none of the components of the equalized histogram are zero)
∗ Plays a central role in histogram matching scheme
– Figure 3.20
∗ Significant improvement in low contrast images (first three)
∗ Transformation functions (Figure 3.21)
· Transformation 4 as nearly linear shape, indicating that the inputs were mapped to nearly equal outputs
∗ Transformation function changes contrast by redistributing pixel gray levels but does not alter the number
(content given by probability) of matched gray levels
• Histogram matching/Histogram specification
– Histogram equalization gives an image with uniformly distributed gray levels automatically
– Histogram matching allows us to specify the shape of the histogram of new image
∗ Useful when you have to create two images with same contrast and brightness
– Development of method
∗ Consider continuous gray levels r and z (continuous random variables)
· r is the intensity level of input image
· z is the intensity level of processed image
∗ Let pr (r) and pz (z) denote their continuous probability density functions
∗ pr (r) is estimated from input image
∗ pz (z) is the specified probability density function desired in output image
∗ Let s be a random variable with the property
Z r
s = T (r) = (L − 1) pr (w)dw
0

This expression is a continuous version of histogram equalization

∗ Define a random variable z with the property
Z z
G(z) = (L − 1) pz (t)dt = s
0

using t as a dummy variable of integration

∗ It follows that G(z) = T (r) and z must satisfy the following condition:

z = G−1 (s) = G−1 [T (r)]

Rr
· T (r) can be obtained by estimating pr (r) from the input image using T (r) = 0 pr (w)dw
Rz
· G(z) can be computed from a given pz (z) using G(z) = 0 pz (t)dt
∗ An image with specified probability density function can be obtained from a given image using the procedure:
1. Compute pr (r) from the input image and obtain the values of T (r) or s
Rz
2. Compute G(z) using the specified PDF in G(z) = 0 pz (t)dt
3. Obtain the inverse transformation z = G−1 (s) by mapping from s to z
4. Get the output image by first equalizing the input image; pixel values in this image are s values. For each
pixel in the equalized image, perform the inverse mapping G−1 (s) to get the corresponding pixel in output
image.
Intensity Transformation and Spatial Filtering 9

– Example: Let an image have the intensity PDF given by

 2r
(L−1)2 0 ≤ r ≤ (L − 1)
pr (r) =
0 otherwise

Find the transformation function that will produce an image whose intensity PDF is
(
3z 2
(L−1) 3 0 ≤ z ≤ (L − 1)
pz (z) =
0 otherwise

∗ Find the histogram equalization transformation for the interval [0, L − 1]

s = T (r)
Z r
= (L − 1) pr (w) dw
Z r0
2
= w dw
L−1 0
r2
=
L−1
This gives a uniform PDF due to histogram equalization transform
∗ Since we are interested in an image with a specified histogram, we compute
Z z
G(z) = (L − 1) pz (w) dw
0
Z z
3
= w2 dw
(L − 1)2 0
z3
=
(L − 1)2

over the interval [0, L − 1]

∗ Finally, require that G(z) = s leading to z 3 /(L − 1)2 = s and
1/3
z = (L − 1)2 s


∗ Multiply every histogram-equalized pixel by (L − 1)2 and compute its cube root giving an image whose
intensities z have the PDF pz (z) = 3z 2 /(L − 1)2 in the interval [0, L − 1]
∗ Since s = r2 /(L − 1), we can generate z directly from r, the input intensity by
1/3
(L − 1)2 s

z =
1/3
r2

2
= (L − 1)
(L − 1)
2 1/3
 
= (L − 1)r

– Discrete formulation of histogram equalization transformation

∗ Discrete formulation removes the ability to get a result given by an exact specified histogram
∗ Mapping from input levels in original image into corresponding sk based on the histogram of original image is
given by

sk = T (rk )
k
X
= (L − 1) pr (rj )
j=0
Intensity Transformation and Spatial Filtering 10

k
X nj
= (L − 1) k = 0, 1, 2, . . . , L − 1
j=0
n
k
(L − 1) X
= nj
M N j=0

∗ Given a specific value of sk , the discrete formulation for G(z) is based on computing the transformation
function
Xq
G(zq ) = (L − 1) pz (zi )
i=0

for a value of q, so that

G(zq ) = sk
where pz (zi ) is the ith value of the specified histogram
∗ Find the desired value zq by the inverse transformation

zq = G−1 (sk )

∗ This gives a value of z for each value of s, performing a mapping from s to z

∗ In practice, compute all the possible values of G and store in an LUT
– Summary of histogram specification process
1. Compute the histogram pr (r) of given image and use it to find histogram equalization transform
k
(L − 1) X
sk = nj
M N j=0

Round the resulting values sk to the integer range [0, L − 1]

2. Compute all values of transformation function G using
q
X
G(zq ) = (L − 1) pz (zi )
i=0

for q = 0, 1, 2, . . . , L − 1 where pz (zi ) are the values of the specified histogram; Round the values of G to
integers in the range [0, L − 1] into an LUT
3. For every value of sk ∈ [0, L − 1], use the stored value of G from step 2 to find the corresponding value of zq
so that G(zq ) is closest to sk and store these mappings from s to z
4. Histogram equalize the input image; map every histogram-equalized value and map every equalized pixel sk
of this image to corresponding value zq in the histogram-specified image using the mappings from step 3
– Example: Using the 64 × 64 pixel image from histogram equalization example
∗ Histogram in Figure 3.23(a)
∗ Specified histogram is in column 2 below
zq Specified Actual
pz (zq ) pz (zk )
z0 =0 0.00 0.00
z1 =1 0.00 0.00
z2 =2 0.00 0.00
z3 =3 0.15 0.19
z4 =4 0.20 0.25
z5 =5 0.30 0.21
z6 =6 0.20 0.24
z7 =7 0.15 0.11
Intensity Transformation and Spatial Filtering 11

∗ Obtain the scaled histogram equalized values from last example as

sk = {1, 3, 5, 6, 6, 7, 7, 7}

∗ Compute all the values of the transformation function G

P0
G(z0 ) 7 j=0 pz (zj ) 0.00
P1
G(z1 ) 7 j=0 pz (zj ) = 7[p(z0 ) + p(z1 )] 0.00
G(z2 ) 0.00
G(z3 ) 1.05
G(z4 ) 2.45
G(z5 ) 4.55
G(z6 ) 5.95
G(z7 ) 7.00
∗ The fractional values are converted to integers in our valid range [0, 7] giving

G(zi ) = {0, 0, 0, 1, 2, 5, 6, 7}

∗ The transformation function is sketched in Figure 3.23(c)

· G is not strictly monotonic and has to be handled
· Find the smallest value of zq such that G(zq ) is closest to sk
· In our example, s0 = 1 and G(z3 ) = 1; that is a perfect match, giving us s0 → G(z3 )
· Every pixel with value 1 in histogram equalized image will map to value 3 in the histogram-specified image
· Final mapping
sk → zq
1 → 3
3 → 4
5 → 5
6 → 6
7 → 7
· In the final step, use the above mappings to map every pixel in the histogram equalized image into a
corresponding pixel in the histogram-specified image
· Listed in the third column of table on top as third column (pz (zk )) (Histogram in Figure 3.23(d))
· Since s = 1 ⇒ z = 3 and there are 790 pixels in the histogram-equalized image at intensity 1, pz (z3 ) =
790/4096 = 0.19
– Example
∗ Figure 3.24 – Input image dominated by large dark areas
∗ Figure 3.25 – Histogram equalized image shows the details in dark areas
∗ Image histogram quickly rises from 0 to 190, resulting in almost all the pixels concentrated towards the upper
end of the dynamic range, giving a light, washed out appearance; the dark areas of the image are expanded
resulting in excessive noise
∗ Fixed by manual specification of the histogram
∗ Sample the function into 256 equally spaced discrete values
· Resulting transformation function G(z) in Figure 3.26
· Smoother transition of levels in the dark regions of gray scale
– Comments
∗ Manual specification of histogram is by trial-and-error
∗ Practical use is to adjust the contrast by using the histogram of a different image
• Local histogram processing
– Global methods (histogram) modify pixels by transformation functions based on entire image
Intensity Transformation and Spatial Filtering 12

∗ May not work very well when details may get obscured due to small number of pixels or smaller neighborhoods
∗ Small number of pixels have a negligible effect on computation of global transformation
– Local methods work with neighborhood of each pixel to find a transformation
– A simple approach would be to define small rectangles on the image and process the pixels in selected rectangle
using the techniques already seen
∗ May have overlapping or non-overlapping rectangles
∗ You could also define one rectangle and move its center from pixel to pixel
∗ Compute the neighborhood and obtain a histogram equalization or histogram specification transformation
– Example – Blurring to reduce noise content
∗ Figure 3-32a – Slightly noisy 512 × 512 pixel image
∗ Figure 3.32b – Global histogram equalization
∗ Figure 3.32c – Local histogram with a 3 × 3 neighborhood
∗ Local vs global histogram equalization
· Intensity values of objects too close to the intensity of large squares, and their sizes too small to influence
global histogram equalization significantly enough to show the detail
• Use of histogram statistics for image enhancement
– Let r be the discrete random variable representing intensity values in the range [0, L − 1], and p(ri ) the normalized
histogram component corresponding to ri
– Mean: Average gray scale intensity
L−1
X
m= ri p(ri )
i=0

p(ri ) is the probability of occurrence of gray level ri

– The nth moment of r about its mean is defined by
L−1
X
µn (r) = (ri − m)n p(ri )
i=0

Obviously, µ0 = 1 and µ1 = 0
– Variance (second moment): Average contrast
L−1
X
σ 2 (r) = µ2 (r) = (ri − m)2 p(ri )
i=0
√
– Standard deviation is defined simply as σ = µ2
– Mean and variance can also be measured directly from sample values (sample mean and sample variance) as
M −1 N −1
1 X X
m= f (x, y)
M N x=0 y=0

M −1 N −1
1 X X 2
σ2 = [f (x, y) − m]
M N x=0 y=0

– Example: 2-bit (L = 4) image of size 5 × 5

0 0 1 1 2
1 2 3 0 1
3 3 2 2 0
2 3 1 0 0
1 1 3 2 2
Intensity Transformation and Spatial Filtering 13

∗ Image histogram is given by: (6, 7, 7, 5), or after normalization, p(ri ) = (0.24, 0.28, 0.28, 0.20)
∗ The average value of intensities in the image is
3
X
m = ri p(ri )
i=0
= (0)(0.24) + (1)(0.28) + (2)(0.28) + (3)(0.20)
= 1.44

∗ Sample mean is computed by

4 4
1 XX
m = f (x, y)
25 x=0 y=0
= 1.44

– Enhancement can be based on measuring global mean and variance over entire image and adjusting those values to
change intensity and contrast
– These quantities can also be used for local enhancement by changing local mean and variance in predefined regions
about each pixel
∗ Consider pixel at location (x, y)
∗ Let Sxy be a subimage/neighborhood of specified size, centered at (x, y)
∗ Mean value of the neighborhood is
L−1
X
mSxy = ri pSxy (ri )
i=0

· pSxy is the histogram of region Sxy

· Many of the pi ’s will be zero, depending on the size of Sxy
∗ The gray level variance is given by
L−1
X
σS2 xy = [ri − mSxy ]2 pSxy (ri )
i=0

∗ Local mean and local variance are a measure of average intensity and average contrast, respectively in the
neighborhood

∗ Scanning electron microscope image of a tungsten filament wrapped around a support

∗ The secondary filament on the right side of the image
∗ Center image shows enhancement by global histogram equalization
∗ Right image shows enhancement by local histogram equalization
∗ Enhance using local enhancement techniques
– Enhancement of dark areas only
∗ Compare local average gray level mSxy in a neighborhood around point (x, y) to the global mean mG
∗ A pixel at point (x, y) is considered a candidate for enhancement if mSxy ≤ k0 mG , 0 < k0 < 1.0
Intensity Transformation and Spatial Filtering 14

∗ Low contrast areas are also candidates for enhancement, if σSxy ≤ k2 σG where k2 > 0 and σG is global
standard deviation
· k2 > 1.0 to enhance light areas
· k2 < 1.0 to enhance dark areas
∗ We need a lower bound on contrast so as not to enhance uniform intensity areas with standard deviation 0
· This can be achieved by requiring k1 σG ≤ σSxy , with k1 < k2
∗ A pixel meeting all the requirements can be enhanced by multiplying it by a specific constant E to in-
crease/decrease its gray level
∗ Let f (x, y) and g(x, y) be the input and enhanced values of the pixel at location (x, y)

E · f (x, y) if mSxy ≤ k0 mG && k1 σG ≤ σSxy ≤ k2 σG
g(x, y) =
f (x, y) otherwise

∗ Values chosen: E = 4.0, k0 = 0.4, k1 = 0.02, k2 = 0.4

∗ Can a multiple of 4 for pixels cause overflow?
∗ A small area under Sxy preserves detail and reduces computational burden

Basics of spatial filtering

• Filtering
– Roots in the use of Fourier transform for signal processing in frequency-domain
– Accepting or rejecting certain frequency components
– Low-pass filter, high-pass filter, band-pass filter
– Filtering effects of frequency-based filters can be achieved by using spatial filters
– Filter, mask, kernel, or window
∗ Values in filter subimage are referred to as coefficients
– Spatial filters have a 1:1 correspondence to frequency filters but can also do nonlinear filtering in addition (not
possible in frequency based filters)
• Spatial filtering
– Spatial filter consists of
1. A neighborhood, typically a small square or rectangle
2. A predefined operation performed on the image pixels in the neighborhood
– Creates a new pixel at the coordinates of the neighborhood center as the result of filtering operation
∗ Generally, the result is written into a new image as the pixels in the neighborhood may still be needed for the
filtering of other pixels
– Convolution – Moving the filter mask over pixels
– Linear filtering
∗ Product of filter coefficients with the corresponding pixels
P
∗ R = wij f (x + i, y + j)
∗ Fig 3-34
∗ Response g(x, y) of the filter at point (x, y)
· Given by the sum of products of filter coefficients and corresponding image pixels
· For a 3 × 3 filter, we have
X1 X 1
g(x, y) = w(i, j)f (x + i, y + j)
i=−1 j=−1
Intensity Transformation and Spatial Filtering 15

· The center coefficient of the filter (0, 0) aligns with the pixel at location (x, y)
∗ Odd dimensions of the filter
· For a mask of size m × n, assume that m = 2a + 1 and n = 2b + 1, where a > 0 and b > 0
· For a filter of size m × n
X a Xb
g(x, y) = w(i, j)f (x + i, y + j)
i=−a j=−b

varying x and y over the entire image

• Spatial correlation and convolution
– Correlation
∗ Process of moving the filter mask over the image and computing the sum of products at each location
– Convolution
∗ Same as correlation except that the filter is first rotated by 180◦
– Figure 3.35
∗ 1D function f and filter w
– Correlation is a function of displacement of the filter
∗ First value of correlation corresponds to zero displacement of the filter
∗ Second value corresponds to one unit displacement
– Correlating a filter w with a function that contains all 0s and a single 1 (unit impulse function or discrete unit
impulse) yields a copy of w but rotated by 180◦
– Convolving a function with a unit impulse yields a copy of the function at the location of the impulse
∗ Prerotating the function by 180◦ and performing the sliding sum of products operations yields the desired result
∗ If the filter mask is symmetric, correlation and convolution yield the same result
– Issue of border crossing by the kernel
∗ Limiting the image to the kernel overlap
∗ Limiting the kernel to the image overlap
∗ Padding image by replication or reflection
– Extending the concept to images (2D)
∗ Figure 3.36
∗ If f contains a region identical to w, value of correlation is maximum when w is centered on that region of f
· This property can be used to find matches between images
∗ Correlation is given by

9 Xa Xb
w(x, y) f (x, y) = w(s, t)f (x + s, y + t)
s=−a t=−b

∗ Convolution is given by

8f (x, y) X
a b
X
w(x, y) w(s, t)f (x − s, y − t)
s=−a t=−b

• Vector representation of linear filtering

– Characteristic response R of a mask for correlation is given by
R = w1 z1 + w2 z2 + · · · + wmn zmn
mn
X
= wk zk
k=1
T
= w z
where ws are coefficients of an m × n filter and zs are corresponding image intensities
Intensity Transformation and Spatial Filtering 16

– The same equation can be used for convolution by rotating the mask by 180◦
– A general 3 × 3 mask is labeled as
w1 w2 w3
w4 w5 w6
w7 w8 w9
– Again, R is given by wT z as above, where w and z are 9-dimensional vectors formed from the coefficients of the
mask and image intensities, respectively
• Generating spatial filter masks
– Generating an m × n linear spatial filter mask requires the specification of mn mask coefficients
– The coefficients are selected based on the purpose of the filter
∗ All a filter does is to compute sum of products
– Replacing the pixels by the average intensity of a 3 × 3 neighborhood centered at the pixels
∗ Average value at location (x, y) is the sum of nine intensity values in the 3 × 3 neighborhood centered on (x, y)
divided by 9
∗ With zi , i = 1, 2, . . . , 9, the average is
9
1X
R= zi
9 i=1
∗ This is the same as
9
X
R= wi zi
i=1

with wi = 1/9
– Generating a spatial mask based on a continuous function of two variables
∗ Gaussian function of two variables has the basic form
x2 +y 2
h(x, y) = e− 2σ 2

where σ is the standard deviation and the coordinates x and y are integers
∗ The mask for a 3 × 3 neighborhood can be generated as w1 = h(−1, −1), w2 = h(−1, 0), . . ., w9 = h(1, 1)
∗ 2D Gaussian has a bell shape with standard deviation controlling the tightness of the bell
– Generating a nonlinear filter requires the size of the neighborhood and the operations to be performed on the image
pixels contained in the neighborhood
∗ Max operation

Smoothing spatial filters

• Used for blurring and noise reduction

– Noise reduction can be achieved by blurring with a linear filter or by a nonlinear filter
• Smoothing linear filters
– Averaging filter or lowpass filter
∗ Average of pixels in a neighborhood
∗ Reduction in sharp transitions in gray levels
· Random noise is characterized by sharp transitions in intensity levels
· Averaging reduces the sharp transitions
Intensity Transformation and Spatial Filtering 17

· A side effect is the blurring of edges which are also characterized by sharp transitions in intensity levels
∗ The filter can also be used to reduce false contouring
– Standard average of pixels in a 3 × 3 neighborhood
1 1 1
1
9×
1 1 1
1 1 1
P1 P1
∗ Same as g(x, y) = 19 c=−1 r=−1 f (x + c, y + r)
∗ Also called box filter (all coefficients of filter are equal)
∗ The entire image can be divided by 9 at the end of filtering
– Weighted average of pixels in a 3 × 3 neighborhood
1 2 1
1
16 ×
2 4 2
1 2 1
∗ Pixels are multiplied by different coefficients, giving more importance to some pixels at the expense of others
∗ Reduces blurring in the smoothing process
∗ Optimization by shifting right by four bits instead of dividing by 16
∗ General implementation to filter an image with a weighted averaging filter of size m × n is given by
Pa Pb
s=−a t=−b w(s, t)f (x + s, y + t)
g(x, y) = Pa Pb
s=−a t=−b w(s, t)

– Example: Figure 3.33

∗ Results with square averaging filters of size m = 3, 5, 9, 15, 35 pixels
∗ Pronounced black border with larger filters
· Result of padding the border region with 0s
– Example: Figure 3.34
∗ Blur an image to detect objects of interest
∗ Image from Hubble telescope, and application of 15 × 15 averaging mask
• Order-statistics filters

– Nonlinear spatial filters

– Response based on ordering or ranking the pixels under the kernel
– Median filter
∗ Good for filtering impulse noise (or salt-and-pepper noise), with less blurring
∗ Most useful order-statistic filter
∗ Force points with distinct gray values to be more like their neighbors
∗ Consider the following neighborhood:
15 15 15
28 21 10
1 23 20
∗ The values are sorted as:
1, 10, 15, 15, 15, 20, 21, 23, 28
giving the median as 15
∗ Consider another example
Intensity Transformation and Spatial Filtering 18

13 15 15 30 25
17 15 15 15 22
26 28 21 10 29
27 1 23 20 5
31 7 27 24 21
∗ Application of 3 × 3 median filter yields
- - - - -
- 15 15 21 -
- 21 15 20 -
- 26 21 21 -
- - - - -
∗ Isolated clusters with area less than n2 /2 are eliminated by n × n median filter
∗ Figure 3.49
– Max filter
∗ Helps with finding the brightest points in a neighborhood
∗ Erodes dark areas adjacent to light regions
– Min filter
∗ Opposite of max filter

Sharpening spatial filters

• Highlight fine detail in an image or enhance detail that has been blurred
– Averaging is same as integration
– Achieve sharpening by differentiation
∗ Strength of response of a derivative operator is proportional to the degree of discontinuity of image at the point
where the operator is applied
∗ Differentiation enhances edges and discontinuities (including noise) and deemphasizes slow varying gray scale
values
∗ High pass filtering
• Foundation
– One dimensional derivatives to simplify the discussion
– Behavior of derivatives at the beginning (step) and end (ramp) of discontinuities and along gray-level ramps
∗ Discontinuities used to model noise points, lines, and edges
– Derivative of a digital function
∗ Defined in terms of differences
∗ Definition for a first derivative must be
· Zero in flat segments (areas of constant gray level)
· Nonzero at the onset of a gray-level step or ramp
· Nonzero along ramps
∗ Definition of a second derivative must be
· Zero in flat areas
· Nonzero at the onset and end of a gray level step or ramp
· Zero along ramps of constant slope
∗ Keep in mind the finite nature of digital functions
· Finite values
Intensity Transformation and Spatial Filtering 19

· Finite maximum change

· Shortest distance for maximum change is adjacent pixels
∗ Basic definition of first-order derivative of a one-dimensional function given by the difference

∂f
= f (x + 1) − f (x)
∂x
∗ Second-order derivative is defined as the difference
∂2f
= f (x + 1) + f (x − 1) − 2f (x)
∂x2
· The two definitions satisfy the conditions laid out above
· Fig 3-50
· Scan line contains an intensity ramp, three sections of constant intensity, and an intensity step
· First-order derivatives are indicated by dots while the second-order derivatives are indicated by squares
· Circles indicate the onset or end of intensity transitions
· First-order derivative produces thick edges while second-order derivative produces fine edges
· For isolated point, second-order derivative gives a much stronger response than first-order derivative
· Second-order derivative can enhance fine detail much more than the first-order derivative
∗ Conclusions from above
· First-order derivatives produce thicker edges in an image
· Second-order derivatives have stronger response to fine detail
· First-order derivatives have stronger response to gray-level step
· Second-order derivatives produce a double response at step changes in gray level
∗ In most applications, second derivative is better suited than first derivative because of enhancement of fine
detail
∗ Zero-crossing property
· Sign of the second derivative changes at the onset and end of a step or ramp
· In a step transition, a line joining the two values crosses the horizontal axis midway between the two
extremes
· Useful to detect edges
• Use of second derivative for enhancement – the Laplacian
– Define a discrete formulation of second order derivative and construct a filter mask based on that formulation
– Preference for isotropic filters
∗ Response is independent of direction of discontinuities in image
∗ Rotationally invariant filters
· Rotating the image and then applying the filter gives the same result as applying the filter first and then
rotating the image
– Development of method
∗ Laplacian – simplest derivative operator
2 2
∗ Defined as ∇2 f = ∂∂xf2 + ∂∂yf2
· Since derivative of any order is a linear operation, Laplacian is a linear operator
∗ Discrete Laplacian operator
· Must satisfy the properties of second derivative
· Partial-order derivative in x direction
∂2f
= f (x + 1, y) + f (x − 1, y) − 2f (x, y)
∂x2
Intensity Transformation and Spatial Filtering 20

· Partial-order derivative in y direction

∂2f
= f (x, y + 1) + f (x, y − 1) − 2f (x, y)
∂y 2
· Discrete Laplacian in two dimensions is given by taking the sum of partials

∇2 f = f (x + 1, y) + f (x − 1, y) + f (x, y + 1) + f (x, y − 1) − 4f (x, y)

· The mask is given by

0 1 0
1 -4 1
0 1 0
· The mask gives isotropic result in increments of 90◦
∗ Diagonal directions can be added to Laplacian by adding two more terms to above equation resulting in the
mask as
1 1 1
1 -8 1
1 1 1
· The mask gives isotropic result in increments of 45◦
∗ Properties of Laplacian – As derivative operator
· Highlights gray level discontinuities in an image
· Deemphasizes slowly varying gray level changes
· Produces images with grayish discontinuities superimposed on a dark featureless background
· Background features can be recovered by adding original and Laplacian images, if the center is a positive
coefficient, or subtracting the Laplacian image from original if center is negative coefficient

f (x, y) − ∇2 f (x, y) if center is negative


g(x, y) =
f (x, y) + ∇2 f (x, y) if center is positive

· Figure 3.52
∗ Scaling Laplacian
· Add to the image its minimum value to bring the new minimum to zero and then, scale the result to the full
[0, L − 1] range
· Figure 3.53
– Simplifications
∗ Computation of Laplacian and subtraction can be done in a single pass of a single mask

g(x, y) = f (x, y) − [f (x + 1, y) + f (x − 1, y) + f (x, y + 1) + f (x, y − 1)] + 4f (x, y)

= 5f (x, y) − [f (x + 1, y) + f (x − 1, y) + f (x, y + 1) + f (x, y − 1)]

∗ The mask is defined as

0 -1 0
-1 5 -1
0 -1 0
• Unsharp masking and high-boost filtering
– Unsharp masking
∗ Used in publishing industry
∗ Subtract a blurred version of the picture from itself
∗ Obtain the mask as:
gmask (x, y) = f (x, y) − f¯(x, y)
Intensity Transformation and Spatial Filtering 21

∗ Add a weighted portion of the mask back to original image:

g(x, y) = f (x, y) + k × gmask (x, y)

k ≥ 0 is a weight for generality

· k = 1 leads to unsharp masking as defined above
· k > 1 leads to high-boost filtering
· k < 1 de-emphasizes the contribution of unsharp mask
∗ Figure 3-54
· Unsharp mask is similar to the second-order derivative
· The points of change of slope in intensity get emphasized
· Possible for the final result to have negative intensities, especially if input image has zero values or k is
chosen as large enough
· Negative intensities can lead to objectionable results (dark halo around edges)
∗ Figure 3-55
· Blurred with a 31 × 31 Gaussian smoothing filter with σ = 5
· Figure 3.55[def] shows result with k = 1, 2, 3
· High values of k lead to unacceptable images
– High-boost filtering
∗ Generalization of unsharp masking
∗ Defined as:
ghb (x, y) = Af (x, y) − f¯(x, y)
where A ≥ 1 and f¯ is a blurred version of f
∗ It can also be written as:

ghb (x, y) = (A − 1)f (x, y) + f (x, y) − f¯(x, y)

= (A − 1)f (x, y) + gmask (x, y)

– Using Laplacian
Af (x, y) − ∇2 f (x, y) if center is negative

g(x, y) =
Af (x, y) + ∇2 f (x, y) if center is positive
– High-boost filtering is standard Laplacian sharpening when A = 1
– If A is large, high-boost image is approximately equal to the original image multiplied by a constant
• Use of first derivative for enhancement – the gradient
– First derivative implemented using the magnitude of gradient
– Gradient of f at (x, y) is defined as the column vector
" #
  ∂f
gx ∂x
∇f = = ∂f
gy ∂y

∗ Important geometric property of the vector: it points in the direction of the greatest rate of change of f at (x, y)
– Magnitude of vector ∇f , denoted by M (x, y), is given by

M (x, y) = mag(∇f )
q
= gx2 + gy2
" 
2  2 #1/2
∂f ∂f
= +
∂x ∂y
Intensity Transformation and Spatial Filtering 22

∗ M (x, y) is the same as the vector norm ||∇f ||

∗ M (x, y) is an image of the same size as the original
∗ Commonly referred to as the gradient image
– Partial derivatives are not rotationally invariant but magnitude of gradient is
– The gradient operator is computationally expensive, and is therefore, approximated as

∇f ≈ |Gx | + |Gy |

∗ Simpler to compute and preserves relative changes in gray levels

∗ Does not preserve isotropic feature property
– Digital approximations to compute appropriate filter masks
∗ Use the following notation to denote intensities in a 3 × 3 region
z1 z2 z3
z4 z5 z6
z7 z8 z9
∗ Simplest approximations

gx = f (x, y + 1) − f (x, y)
gy = f (x + 1, y) − f (x, y)

∗ Robert’s definition, based on cross differences

gx = f (x + 1, y + 1) − f (x, y)
gy = f (x, y + 1) − f (x + 1, y)

· Compute the gradient as

p
M (x, y) = [f (x + 1, y + 1) − f (x, y)]2 + [f (x, y + 1) − f (x + 1, y)]2

· Using absolute values, the gradient is given by

M (x, y) ≈ |f (x + 1, y + 1) − f (x, y)| + |f (x, y + 1) − f (x + 1, y)|

∗ Implemented with the mask (Roberts cross-gradient operator)

-1 0 0 -1
0 1 1 0
∗ Even-sized masks are different to implement due to lack of center of symmetry
∗ An approximation using absolute values at point f (x, y) using a 3 × 3 mask is given by Sobel operators

M (x, y) ≈ |(f (x − 1, y + 1) + 2f (x, y + 1) + f (x + 1, y + 1)) −

(f (x − 1, y − 1) + 2f (x, y − 1) + f (x + 1, y − 1))| +
|(f (x + 1, y − 1) + 2f (x + 1, y) + f (x + 1, y + 1)) −
(f (x − 1, y − 1) + 2f (x − 1, y) + f (x − 1, y + 1))|

or, pictorially as
-1 -2 -1 -1 0 1
0 0 0 -2 0 2
1 2 1 -1 0 1
· Difference in first and third row in the left mask gives partial derivative in the vertical direction
· Difference in first and third column in the right mask gives partial derivative in the horizontal direction
∗ Mask gives gradient in x and y directions, and coefficients sum to zero indicating no change in constant gray
level areas
Intensity Transformation and Spatial Filtering 23

∗ Used for edge detection (Fig 3-57)

Combining spatial enhancement methods

• Use a combination of the filters to enhance the image depending on the application
• Figure 3.43
• Reading assignment