Image Segmentation

Introduction
 Image Segmentation refers to the decomposition or partition of an image into meaningful
regions.
 Segmentation subdivides an image into its constituent regions or objects.
 Most of the image analysis algorithms perform segmentation as a first step towards
producing the description.
 Applications => Identifying objects in a moving scene for object-based video compression
(MPEG4), Identifying objects which are at different distances from a sensor using dept
measurements from a laser range finder enabling path planning for a mobile robots (Stereo
Vision).
 Image segmentation algorithms are based on two basic properties of intensity values:
Discontinuity and Similarity.
 Discontinuity => To partition an image based on abrupt change in intensity such as edge in
image. ( Edge detection, Point detection, Line detection)
 Similarity => To partition an image into regions that are similar according to a set of
predefined criteria. (Thresholding, Region growing, Region splitting etc.)
7.1 Detection of Discontinuities
 The most common way to look for discontinuities is to run a mask through the image.
 For 3 x 3 mask => The response (R) of the mask at any given point in the image is given by
R = w1z1 + w2z2+……………+w9z9
R=∑
w1 w2 w3
w4 w5 w6
w7 w8 w9

 Where Zi is the gray level of the pixel associated with the mask coefficient wi.
 The response of the mask is defined with respect to its center location.
7.1.1 Point Detection
 Principle of detection of isolated points in an image.
 We can say that a point has been detected at the location on which the mask is centered if
 |R| T (Where T is a non-negative threshold and R is response of the mask).The point can
be detected using the mask shown below.

-1 -1 -1
-1 8 -1
-1 -1 -1

Figure 1 : Point Detection Mask

 This formulation measures the weighted difference between the center point and its
neighbors.
 The idea is that an isolated point ( a point whose gray level is significantly different from its
background and which is located in homogeneous area) will be quite different from its
surroundings and thus be easily detectably by this type of mask.

Fig (a) Original Image (b) Point Detection Result

7.1.2 Line Detection
 It is possible via various masks.
 The horizontal mask, +45° mask , Vertical mask and -45° mask is given below.
-1 -1 -1
2 2 2
-1 -1 -1

Figure 2 : Horizontal Line Detection Mask

-1 -1 2
-1 2 -1
2 -1 -1

Figure 3 : + 45° Line Detection Mask

-1 2 -1
-1 2 -1
-1 2 -1
Figure 4 : Vertical Line Detection Mask
2 -1 -1
-1 2 -1
-1 -1 2

Figure 5 : -45° Line Detection Mask

 Suppose that the four masks (R1 to R4) are run individually through an image.
 If at a certain point of image, |Ri| > |Rj| for all i j, that point is said to be more likely
associated with a line in the direction of mask i.
 For example, if at a point in the image, |R1| > |Rj| for j=2,3,4, that particular point is said to
be more likely associated with a horizontal line.
7.1.3 Edge Detection
 – Edge: a set of connected pixels that lie on the boundary between two regions
• ‟Local‟ concept in contrast to ‟more global‟ boundary concept
• To be measured by grey-level transitions
• Ideal and blurred edges

 It is most common approach for detecting meaningful discontinuities in gray level.

 Model of ideal digital edge, mode of a ramp digital edge
 The “thickness” of the edge is determined by the length of the ramp and its transitions from
an initial to a final gray level.
 This length is determined by the slope, which, in turn, is determined by the degree of
blurring.
 Blurred edges tend to be thick and sharp edges tend to be thin.
 First and Second derivative of Gray level profile.
 The first derivative is positive at the points of transition into and out of the ramp as we move
from left to right along the profile. It is constant for points in the ramp and is zero in areas of
constant gray level.
 The second derivative is positive at the transition associated with dark side and negative at
the transition associated with the light side of the edge. The value is zero along the ramp and
in the areas of constant gray level.
 Observations are given below :
 The magnitude of first derivative can be used to detect the presence of an edge at a point in
an image.
 The sign of second derivative can be used to determine whether an edge pixel lies on dark or
light side of an edge.

First order derivative

First order derivative => Gradient operator (Roberts, Prewitt and Sobel operators)
 A 3 X 3 region of an image is shown below.
Z1 Z2 Z3
Z4 Z5 Z6
Z7 Z8 Z9

3 x 3 Region of an Image
 Computation of the gradient of an image is based on obtaining the partial derivatives ∂f/∂x
and ∂f/∂y at every pixel location.
∂f/∂x = f(x+1,y) – f(x,y)
∂f/∂y = f(x,y+1) – f(x,y)
 For Roberts operator : Gx =(Z9-Z5) and Gy = (Z8-Z6)
 For Prewitt : Gx = (Z7+Z8+Z9) – (Z1+Z2+Z3) and Gy = (Z3+Z6+Z9) – (Z1+Z4+Z7)
 For Sobel : Gx = (Z7+2Z8+Z9) – (Z1+2Z2+Z3) and Gy = (Z3+2Z6+Z9) – (Z1+2Z4+Z7)
Second order derivative => Laplacian operator
 The Laplacian of two dimensional image f(x,y) is given by
2
f= +

 This equation needs to be expressed in discrete form.

= f(x+1,y) + f(x-1,y) – 2f(x,y) ( Partial Second order derivative in x direction )

= f(x,y+1) + f(x,y-1) – 2f(x,y) ( Partial Second order derivative in y direction)

 2
f = [ f(x+1,y) + f(x-1,y) + f(x,y+1) + f(x,y-1) – 4f(x,y) ]
2
 Construct filter mask using f.
Second-order derivative of a 2-D function
– Digital approximations by proper masks
– Complementary use for edge detection
• Cons: Laplacian is very sensible to noise; double edges
• Pros: Dark or light side of the edge; zero crossings are of better use
• Laplacian of Gaussian (LoG): preliminary smoothing to find edges through
zero crossings
7.2 Edge Linking and Boundary Detection Local processing
– Analyze pixels in a small neighborhood following predefined criteria.
– Connect similar (labeled „edge‟) points

• Both magnitude and angle criteria should be satisfied

Example: find rectangular shapes similar to license plate
– Find gradients
– Connect edge points
– Check horizontal-vertical proportion
Global processing via the Hough Transform
– Determine if points lie on a curve of specified shape
– Consider a point (xi,yi) and the general line equation yi= axi+ b
– Write the equation with respect to ab-plane (parametric space) b = −x ai + yi
– Write the equation for a second point (xj,yj) and find the intersection point (a‟,b‟) on the
parametric space
– All points on a line intersect at the same parametric point

Computational aspects of the Hough transform

– Subdivision of the parametric space into accumulator cells
Hough transform: handling the vertical lines
– Through normal representation x cosθ + y sinθ = ρ
Instead of straight lines, there are sinusoidal curves in the parameter space. The number or
intersecting sinusoids is accumulated and then the value Q in the accumulator A(i,j) shows the
number of collinear points lying on a line x cosθj + y sinθj = ρ

Example: two lines connecting three points each – Fig (d) indicates that the Hough transform
exhibits a reflective adjacency relationship

Summary of Hough transform for edge linking – Compute the gradient – Specify subdivisions in
the parametric plane – Examine the counts of the accumulator cells – Examine the continuity
relationship between pixels in a chosen cell
Global processing via Graph-Theoretic Techniques

– Graph G=(N,U): a set of nodes N and a set U of arcs (ni, nj).

• In a directed arc ni is called parent and nj is called successor• Expansion: identifying successors
of a node.• Starting (0 or root) level, last (goal) level. Cost c (ni, nj) associated with an arc;

• Path n1, n2,…,nk, with each ni, being a successor of nj-1. Cost of a path is the sum of costs of
the arcs constituting the path.

 Edge element defined between two neighboor pixels p and q (xp, yp) (xq, yq)
 Associate cost with an edge element c( p,q) = H −[ f ( p) − f (q)]
7.3 Thresholding
 Threshold can be chosen as the relation t = t(r,c , p(r,c)), where p(r,c) is the feature value at
the pixel (r,c).
 In the case of gray level thresholding => p(r,c) = g(r,c) for all (r,c) within the image domain.
 If t depends on the feature p(r,c) => Local threshold
 If t depends on the pixel position (r,c) as well as on the feature p(r,c) => Dynamic Threshold.
7.3.1 Threshold Selection Using Histogram
 Our objective is to segment an image g(r,c) of size M x N that contains two types of regions
say R1 and R2 by gray level thresholding.
 Threshold can be selected using the information contained by gray level histogram of the
image.
 Suppose gray level of pixels belonging to R2 type regions are greater than R1 and the
number of pixels contained in R2 is p% of total number of pixels in the image.
 Threshold t should be such that
t = pMN/100
 This method cannot be applied to unknown class of images where p is not known a priori.

7.4 Multi-Level Thresholding

 Suppose an image contains n ( where n > 2 ) types of regions.
 The feature (eg. graylevel) histogram of the image is expected to contain n number of peaks
each of which corresponds to distinct type of regions.
 Thus we have (n-1) valleys in the histogram and select graylevel corresponding to the bottom
of these valleys as threshold for image segmentation.

7.5 Local Thresholding

 Consider where due to variation in illumination, overall brightness of the image varies widely
from one corner to the other.
 In this case, the graylevel distribution for the object as well as for the background overlap
each other significantly.
 No Single gray level threshold can segment the image properly
 Solution : Divide the whole image into smaller ones so that the variation in illumination over
each these sub-images is negligible.
 Each sub-image is segmented independently and the segmented sub images are put together
in appropriate order to get segmented version of an original image.

7.6 Optimum Threshold

 Let us determine an optimum threshold for graylevel thresholding.
 Suppose an image contains two types of regions R1 and R2 with mean graylevels µ 1 and µ2
respectively.
 Variation in graylevel is due to additive noise with zero mean and standard deviation σ.
 The a priori probability that a pixel of the image belongs to region R1 is P1 and that for
region R2 is P2.
 Note that P1+P2 = 1
 Then optimum threshold (t) for an image segmentation is given by
t = (µ1 + µ2 )/2 + σ2 / (µ1 - µ2) ln (P2/P1)
7.7 Basic Global Thresholding
– Threshold midway between maximum and minimum gray levels
– Appropriate for industrial inspection applications with controllable illumination
– Automatic algorithm
• Segment with initial T into regions G1 and G2
• Compute the average gray level m1 and m2
• Compute new T=0.5(m1+m2)
• Repeat until reach an acceptably small change of T in successive iterations

7.4. Region Based Segmentation

7.4.1 Region Extraction
 Suppose the spatial domain on which the image is defined is denoted by V.
 The image segmentation techniques divides V into say n regions denoted by Ri (
i=1,2,……n) such that
⋃ =V
Ri Rj =
Prop (Ri) = True
Prop (Ri Rj) = False
 The first condition says that every pixel of the image domain is mapped to one region or the
other.
 The second condition ensures that a pixel is mapped to only one region.
 Third condition indicates that the regions are defined on some property.
 Finally maximality of each region is assured by the fourth condition.
7.4.2 Region Growing
 Let us pick an arbitrary pixel (r,c) from the domain of the image to be segmented.
 This pixel is called seed pixel and this pixel belongs to some other regions.
 Examine the nearest neighbors (4 or 8-neighbors depending on the connectivity) of (r,c) one
by one and a neighboring pixel is accepted to belong to the same region as (r,c) if they
together satisfy the homogeneity property of a region.
 Once a new pixel is accepted as a member of the current region, the nearest neighbors of this
new pixel are examined.
 This pixel goes on recursively until no more pixel is accepted.
 All pixels of the current regions are marked with a unique label.
 Another seed pixel is picked up and the same procedure is repeated.
 Region labeling is done until every pixel is assigned to some region or the other.
 Main problem of this approach in addition to large execution time are : Selection of property
to be satisfied and Selection of seed point.
7.4.3 Region Splitting
 Suppose we try to satisfy the homogeneity property over a rectangular region. [g(r,c) th ].
 If the gray level present in the region do not satisfy the property => Divide into four equal
quadrants.
 If the property is satisfied => Leave as it is.
 Graph theory => Region [ node ]
 Then a region is split into four children if it does not satisfy the given property.
 If the property is satisfied => Node is left unaffected.
 This method is applicable to images whose number of rows and columns are integer power of
2.
 Example : Consider 8 x 8 Image with gray level ranging from 0 to 7. Threshold value (th) =
3.
5 6 6 6 7 7 6 6
6 7 6 7 5 5 4 7
6 6 4 4 3 2 5 6
5 4 5 4 2 3 4 6
1 3 2 3 3 2 4 7
 0 0 1 0 2 2 5 6
1 1 0 1 0 3 4 4
1 0 1 0 2 3 5 6

 Perform the split operation. => Start from 4 x 4. [ Splitting process to create m/2 x m/2 ].

7.4.4 Region Merging

 This method is exactly opposite to the region splitting method.
 Region splitting method => Top-down method.
 Region merging method => Bottom-up method.
 This method is also applicable to images whose number of rows and number of columns are
some integer power of 2.
 At any level of merging, check if four adjacent regions arranged in 2 x 2 together satisfies the
homogeneity property.
 If yes => Merge regions to a single homogeneous region => Otherwise leave as they are.
 In terms of graph theory => Child nodes are removed if the parent node satisfies
homogeneity.

5 6 6 6 7 7 6 6
6 7 6 7 5 5 4 7
6 6 4 4 3 2 5 6
5 4 5 4 2 3 4 6
1 3 2 3 3 2 4 7
0 0 1 0 2 2 5 6
1 1 0 1 0 3 4 4
1 0 1 0 2 3 5 6

 Perform the merge operation => Start from 2 x 2 [ Merging process to create 2m x 2m ]
7.4.5 Split and Merge
 If most of the homogeneous regions are small => Takes more time. ( Split Operation )
 No a priori knowledge available about the size of regions => More Time
 It is an hybrid approach which combines both splitting and merging technique.
 Suppose we start with rectangular regions of size m x m pixels.
 To each region homogeneity property is tested.
 If test fails, the regions is split into four quadrants each of size m/2 x m/2.
 If the region satisfies the homogeneity property => Merging process is followed to form a
region of size (2m) x (2m).