Segmentation by Multiresolution Histogram Decomposition

Segmentation by Multiresolution Histogram Decomposition RAMANA L. RAO LAKSHMAN PRASAD Robotics Research Laboratory, Louisiana State University, Baton Rouge, LA 70803 Running head: Multiresolution Histogram Decomposition address for correspondence Ramana L. Rao Robotics Research Laboratory Department of Computer Science 298 Coates Hall Louisiana State University Baton Rouge, LA 70803 (504) 388-1249 (Phone) (504) 388-1465 (FAX) email: [email protected] 1  Abstract In recent years, multiresolution techniques are increasingly being applied to image pro- cessing problems. Global features are quickly and eciently extracted from images through these techniques. In this paper, we describe a robust image segmentation scheme that an- alyzes the histogram of a grayscale image using the idea of multiresolution. Our approach is dierent from most conventional multiresolution techniques in image processing in that, the technique is applied not to the image data but to the image histogram. The overall algorithm is linear in the size of the image. The algorithm is exible and does not require that the histogram of an image be bimodal. 2  List of Symbols 2 mathematical symbol for `is an element of'  mathematical symbol for `subset of' [ mathematical symbol for `set union' 1 mathematical symbol for `innity'  mathematical symbol for `times' (multiplies) \ mathematical symbol for `set intersection' # hash sign (pound sign)  lowercase Greek letter `phi' ; mathematical symbol for `the empty set' 6= mathematical symbol for `not equals'  uppercase Greek letter `sigma' 8 mathematical symbol for `for all'  lowercase Greek letter `alpha' short arrow pointing left ::: ellipsis 3 1 Introduction The last few years have seen an explosion of multiresolution techniques in signal and image processing research. Multiresolution oers an ecient framework for extracting informa- tion from images at various levels of scale [9, 5]. Pyramid algorithms (e.g. see [1]) use a multiresolution architecture to reduce the computational complexity of image processing operations. Gauch and Pizer [2] describe a multiresolution approach to image feature anal- ysis. Liu and Yang [4] use multiresolution for segmenting color images. Lifshitz and Pizer [3] propose a hierarchical segmentation technique using intensity extrema in images. We have used the idea of multiresolution decomposition to eciently integrate information in distributed sensor networks [6, 7]. Segmentation is an important low-level image processing task[10]. The goal of segmen- tation is the grouping of pixels in an image according to some arbitrary measure (criterion). For instance, all pixels in an image whose intensity levels lie within a specied range may be grouped together as one region. Pixels grouped together are considered to be homogeneous according to the criterion used in the segmentation process. The simplest type of image segmentation is based on the statistical distribution of intensity levels in the image and involves the selection of a threshold from the image histogram. The image is subsequently thresholded, resulting in the labeling of pixels according to whether their intensity level is greater than or less than the threshold. This method depends on the bimodal property of the histogram [8]. Often, histograms do not display bimodal properties. Only rela- tively noise-free images with a well dened background and a well dened foreground, have bimodal histograms. This limits the utility of the segmentation method just described. This paper deals with a multiresolution technique for analyzing histogram data in im- ages. Thus, multiresolution is not applied directly to the image data but to the image histogram. The most signicant advantage to this approach is the reduced computational eort resulting from processing the histogram instead of the entire image, at several levels of resolution. The number of intensity levels in an image (typically a power of 2) is usually 4 much smaller than the size of the image in pixels. However, it must be remembered that at least one pass over the entire image is required to arrive at the histogram. The other advantage to applying multiresolution decomposition to the histogram is that the multires- olution process is applied in one dimension only; multiresolution of image data involves two-dimensional analysis and decomposition. One dimensional multiresolution is simpler both in concept and in implementation. The histogram is analyzed, rst at a coarse scale and later at increasingly ner scales, for modality information. The resolution process is stopped when an acceptable segmentation of the image is obtained. Alternatively, the resolution process can be stopped when the image has been segmented into a specied number of regions. We have developed an algorithm for analyzing multimodal histograms and segmenting images hierarchically. The coarse segmentation of the image obtained by analyzing the histogram at a coarse scale is rened at higher levels of resolution. The motivation for our work stems from a multiresolution architecture for segmenting images with bimodal histograms developed by Bongiovanni et al. [1]. Their approach involves a pyramid architecture for determining the bimodality of image subregions which are later integrated to arrive at the segmented image. 2 Preliminaries For every j 2 Z let k dj = f j ; k 2 Zg (1) 2 denote the dyadic rationals at resolution level j . Thus, for any j 2 Z, dj is a set of equally spaced sampling points on the real line R. If i < j , di represents a coarser (lower resolution) sampling of R, and if i > j , di represents a ner (higher resolution) sampling of R. Note S that : : :  dj ?1  dj  dj +1 : : : and 1j =?1 dj is dense in R 5  Let f : Z  Z ! [0; g ] \ Z be an image; g is the largest intensity level occuring in the image. Let hf : [0; g ] \ Z ! Z+ , where Z+ is the set of all positive integers with zero thrown in, denote the histogram of the image. In other words, hf (k) = #f(x; y ) : f (x; y ) = kg; k 2 [0; g ] ; (2) where # denotes the cardinality operator on sets. Note that hf is a discrete function. We construct a continuous function from hf as described below and through a convenient abuse of notation, henceforth refer to this continuous function as the histogram function hf . We let hf (x) = hf (k) for x 2 [k; k + 1). Thus, hf is the union of several piecewise constant functions. Now, the domain of hf is R and further, hf has the property of compact support. For j 2 Z; hf (x) can be sampled at sample points fdj g to yield a representation of hf , at resolution level j , denoted hjf . Further, this representation can be expressed as a linear combination of a set of functions obtained by scaling and translating the characteristic function of the unit interval [0; 1] ( 1 if 0  x  1 : (x) = 0 otherwise (3)  is the well-known Haar scaling function. Consider the set of functions f(2j x ? n)g1 n=?1 ; (4) each function in this set has as its support [ 2nj ; n2+1 nk nk +1 nl nl +1 j ). Note that [ 2j ; 2j ) \ [ 2j ; 2j ) = n=?1 [ 2j ; 2j ) = R. The relationship between the dyadic rationals and ; 8 k 6= l; and [1 n n+1 the intervals of support of  is worth noting. From equation (4), a representation of hf (x) at the j th level of resolution is 1 X hjf (x) = hf (2?j n) (2j x ? n) (5) n=?1 where the summation is really over nitely many n since hf has compact support. A representation of hf at the highest resolution possible is obtained by sampling hf at fd0g. 6  (a) (b) Figure 1: A typical image and its histogram 3 Histogram Multiresolution Figure 1 shows a typical image and the associated histogram of intensity levels occuring in the image. The histogram does not display bimodal properties. Multiresolution decom- position of this histogram involves sampling the histogram at several levels of resolution, starting from a coarse level and moving towards increasingly ner levels. Note that, al- though the histogram does not show well-dened modality, the envelope of the pixel counts in the histogram does show three broad modes. Before the histogram is analyzed at dierent resolution levels, it is low pass ltered to smooth the data. Neighborhood averaging, and morphological ltering are two satisfac- tory methods of data smoothing. After smoothing, the histogram is sampled at dierent resolution levels. Figure 2 (a) shows the eect of morphological ltering with a window of size 3 on the histogram in gure 1. Figure 2 (b) shows the same histogram after neighborhood averaging the result in (a) using a sliding window of about twice the size of the window used for morphological ltering. As mentioned before, three broad modes are now visible in the histogram. Figures 3 (a) through (d) show the eect of sampling this smoothed histogram 7  (a) (b) Figure 2: Histogram from gure 1(b) smoothed at resolution levels -5, -4, -3, and -2 respectively. The number of modes in the sampled histogram increases with the level of resolution. Figure 4 shows the result of segmenting the original image (gure 1 (a)) using the modality information gathered from the multiresolution decomposition of the histogram (gure 3). No postprocessing was performed on these images to improve the quality of the segmentation. The vertical gray bands in gures 4 (c) and (d) are not the result of bad segmentation; these artifacts (presumably the head signature from a device on which the original image was printed) are present in gure 1 (a) and are easily discernible. One other point is worth mentioning at this stage. The quality of the segmented image can be improved by utilizing pixel connectivity information in the image. Postprocessing using this proximity information has been deliberately not carried out to show the results of the segmentation based solely on the multiresolution decomposition of the histogram. This holds for the images shown in section 5 also. 8 (a) j = -5 (b) j = -4 (c) j = -3 (d) j = -2 Figure 3: hjf sampled at dierent resolutions 9 (a) 2 regions (b) 3 regions (c) 4 regions (d) 5 regions Figure 4: Segmented images 10 4 Segmentation Algorithm The coarsest resolution level at which hf is sampled is dictated by modality considerations. Since at least two modes are required, at a particular resolution, for a threshold intensity level to be selected, the minimum number of sample points is 4. It is not possible to have a bimodal histogram with less than 4 sample points. If the number of intensity levels in the image is , the lowest resolution level jmin is ? log2(=4). The multiresolution algorithm starts with jmin and increases the number of sample points by a factor of two at each step. The goal of segmentation is the partitioning of the image into a set of disjoint subregions I1 ; I2; : : :; Ik such that Ii \ Ij = ; for i 6= j and [k1 Ii = I . The segmentation algorithm is given below. 1. Construct the histogram hf of the input image 2. Low pass lter the histogram to smooth out noise 3. j ? log2(=4); I ; 4. resolve hf at level j to get hjf 5. analyze the modality of hjf 6. if hjf has less than two modes, j j + 1; if j < 0 go to step 4 else stop 7. if I is empty, partition image into I1 , I2 , : : :Ik using thresholds picked from hjf , and set I [ki=1 Ik j j + 1;if j < 0 go to step 4 else stop 8. For each region Ij 2 I , segment Ij using the thresholds from hjf into Ij 1 ; Ij 2; : : :; Ijr ; remove Ij from I and add Iji ; i = 1; 2; : : :; r to I 9. if segmentation criteria not satised, j j + 1; if j < 0 go to step 4 else stop In step 2 of the above algorithm, low pass ltering is accomplished by morphological ltering followed by neighborhood averaging. For morphological ltering, a sliding one 11 dimensional window of odd size is used to replace each element of the histogram by the maximum value in the window. Similarly, neighborhood averaging is performed by sliding a similar window over the data and replacing the center of the window by the average of the data values occuring in the window. The overall eect is that of suppressing the high frequency eects and retaining the low frequency components in the data. The algorithm makes at most two passes over the image data; the rst pass is required for collecting the histogram data, and a nal pass for labeling image pixels with the thresholds detected in the sampled histogram. As a result, the algorithm is faster than segmentation methods that make multiple passes over the image. 5 Results We have applied the algorithm described in the previous section to several test images. All test images had multimodal histograms. As mentioned earlier, no post processing oper- ations were performed on the segmented images to improve the quality of segmentation. Preprocessing operations were restricted to smoothing the raw histogram. No attempt was made to select test images that were free of texture. However, images with a signicant amount of texture information would not be segmented satisfactorily using the proposed algorithm since textured images don't display well-dened modes in their histograms. Figures 5 and 6 show a few test images representative of the test set, histograms, and the results of applying the multiresolution segmentation algorithm. 6 Conclusion A robust segmentation scheme based on multiresolution decomposition of image histograms was presented. The algorithm presented is not restricted for use with images displaying primarily bimodal histograms. The performance of existing split-and-merge segmentation schemes can be improved by incorporating our algorithm in the 'split' phase. Since the 12  (a) original image (b) histogram (c) 3 regions at j = -5 (d) 5 regions at j = -4 Figure 5: Results of applying the segmentation algorithm 13  (a) original image (b) histogram (c) 3 regions at j = -5 (d) 5 regions at j = -4 Figure 6: Results of applying the segmentation algorithm 14 algorithm is hierarchical, regions created at a particular level of resolution wholly lie inside regions created at a lower level. Future research directions include extension of the general idea for segmenting color images. References [1] G. Bongiovanni, L. Cinque, S. Levialdi, and A. Rosenfeld, \Image segmentation by a multiresolution approach," Pattern Recognition Vol. 26, No. 12, pp. 1845{1854, 1993. [2] J.M. Gauch, and S.M. Pizer, \Multiresolution analysis of ridges and valleys in grey- scale images," IEEE Trans. Pattern Anal. Machine Intell., Vol. 15, No. 6, pp. 635{646, June 1993. [3] L.M. Lifshitz, and S.M. Pizer, \A multiresolution hierarchical approach to image seg- mentation based on intensity extrema," IEEE Trans. Pattern Anal. Machine Intell., Vol. 12, No. 6, pp. 529{540, June 1990. [4] J. Liu, and Y-H. Yang, \Multiresolution color image segmentation," IEEE Trans. Pat- tern Anal. Machine Intell., Vol. 16, No. 7, pp. 689{700, July 1994. [5] S. Mallat, \Multiresolution approximations and wavelet orthonormal bases of L2(R)," Trans. Amer. Math. Soc., Vol. 315, pp. 69{88, 1989. [6] L. Prasad, S.S. Iyengar, R.L. Rao, and R.L. Kashyap, \Fault-tolerant sensor integration using multiresolution decomposition," Physical Review E, Vol. 49, No. 4., pp. 3452{ 3461, April 1994. [7] L. Prasad, S.S. Iyengar, R.L. Rao, and R.L. Kashyap, \Fault-tolerant integration of abstract sensor estimates using multiresolution decomposition," Proc. IEEE Conf. Sys., Man & Cyber., Le Touquet, France, Oct. 1993 [8] T.Y. Phillips, A. Rosenfeld, and C.A. Sher, \O(log n) bimodality analysis," Pattern Recognition , Vol. 22, pp. 741{746, 1989. 15  [9] A. Rosenfeld, Ed., Multiresolution Image Processing , Springer, Berlin, 1984. [10] A. Rosenfeld, and A.C. Kak, Digital Picture Processing , Volume 2, 2nd Edition. Aca- demic Press, 1982. 16 View publication stats