Image Segmentation Using Mean Shift Clustering

Image segmentation is the process of partitioning an image into multiple segments or regions to simplify its representation and facilitate analysis. The goal is to group pixels with similar characteristics, such as color, texture, or intensity, into coherent and meaningful segments. The use of mean shift clustering (MSC) in image segmentation is a well-known method in computer vision that divides an image into meaningful zones according to color and space.

In this article, we'll discuss the implementation of image segmentation by mean shift clustering.

What is mean shift clustering?

Mean shift clustering is a method used to find clusters in data without specifying the number of clusters beforehand. Mean shift clustering is a non-parametric, reiterative algorithm that seeks to identify groups in a dataset by discovering the sharper peaks in a density function, when, unlike some clustering algorithms, it does not require a piece of prior knowledge on the number of categories.

Why to use mean shift clustering for image segmentation?

Mean shift clustering is used for image segmentation because it is a non-parametric, unsupervised method that effectively identifies the modes or high-density regions of data, which correspond to the different segments in an image. By iteratively shifting data points towards the average of points within a given window, mean shift clustering can discover arbitrarily shaped clusters without assuming a specific number of clusters beforehand. This makes it particularly suitable for image segmentation, where the goal is to partition the image into meaningful regions based on pixel intensity and color. The method's ability to handle complex structures and its robustness to noise further enhance its effectiveness in accurately segmenting images.

Steps for Image Segmentation Using Mean Shift Clustering:

1. Comprehend Mean Shift Clustering: Mean Shift is a non-parametric technique for clustering which is aimed at finding modes of density function. It essentially shifts data points to go towards an average of points situated in a particular vicinity before subsequently moving towards regions characterized by a higher density through a succession of iterations.
2. Preprocess the image: Change the color space so that the image better reflects how people perceive colors (e.g., from RGB to Lab* or HSV). It is also optional to get a read of noise by spatial smoothing (e.g., Gaussian blur).
3. Feature Space Construction: Develop a feature space combining color and spatial information. Assuming an image of size 𝑊 × 𝐻 (W×H) with 3 color channels (e.g., RGB), every pixel can be represented as a five-dimensional vector [ 𝑅, 𝐺, 𝐵, 𝑥, 𝑦 ] [R, G, B,x,y], where ( 𝑥, 𝑦 ) (x,y) are pixel coordinates.
4. Mean Shift Clustering:Apply Mean Shift clustering in space slightly limited feature space – a way of grouping similar pixels based on color and spatial proximity (Wikipedia).
5. Label Assignment: To each pixel assign the cluster center that it converged to. This second step makes it possible to divide the image into regions according to the previously designed clusters.
6. Smoothen the Image: After the segmentation has been post-processed, you may smoothen the segmented image to either reduce the number of small regions which appear to be false or refine its boundaries.

Image taken:

Code Implementation of Image Segmentation Using Mean Shift Clustering in Python

Here, is the step by step code implementation of image Segmentation Using Mean Shift Clustering in Python.

Step 1: Import Libraries

Import the necessary libraries for image processing and clustering.

import cv2
import numpy as np
from sklearn.cluster import MeanShift, estimate_bandwidth
from google.colab.patches import cv2_imshow

Step 2: Load the Image

Load the image from the specified path and check if it's loaded successfully.

# Load the image
image_path = '/content/Screenshot-2024-06-11-150857.webp'
image = cv2.imread(image_path)

# Check if the image was loaded successfully
if image is None:
    print(f"Error: Unable to load image at {image_path}")
else:
    # Convert the image to RGB
    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

Step 3: Convert Image to Lab* Color Space

Convert the image from RGB to Lab* color space for better clustering.

    # Convert image to L*a*b* color space
    lab_image = cv2.cvtColor(image, cv2.COLOR_RGB2LAB)

Step 4: Flatten the Image

Flatten the image to create a 2D array where each row represents a pixel with its Lab* values.

    # Flatten the image
    flat_image = lab_image.reshape((-1, 3))

Step 5: Create Feature Space

Create a feature space that includes the Lab* values and the (x, y) coordinates of each pixel.

    # Create the feature space [L, a, b, x, y]
    height, width, _ = image.shape
    x, y = np.meshgrid(np.arange(width), np.arange(height))
    flat_image_with_coordinates = np.column_stack([flat_image, x.flatten(), y.flatten()])

Step 6: Estimate Bandwidth for Mean Shift

Estimate the bandwidth parameter for the Mean Shift algorithm

    # Estimate bandwidth for Mean Shift
    bandwidth = estimate_bandwidth(flat_image_with_coordinates, quantile=0.2, n_samples=500)

Step 7: Perform Mean Shift Clustering

Apply the Mean Shift clustering algorithm to the feature space.

    # Perform Mean Shift clustering
    mean_shift = MeanShift(bandwidth=bandwidth, bin_seeding=True)
    mean_shift.fit(flat_image_with_coordinates)
    labels = mean_shift.labels_

Step 8: Reshape Labels to Image Shape

Reshape the cluster labels to match the original image dimensions.

    # Reshape the labels to the original image shape
    segmented_image = labels.reshape((height, width))

Step 9: Generate Colored Segmented Image

Create a colored segmented image using random colors for each cluster.

    # Generate a colored segmented image
    unique_labels = np.unique(labels)
    segmented_colors = np.random.randint(0, 255, size=(len(unique_labels), 3))
    colored_segmented_image = segmented_colors[segmented_image]

Step 10: Display the Original and Segmented Images

Display the original and the colored segmented images.

    # Display the result using cv2_imshow
    cv2_imshow(image)
    cv2_imshow(colored_segmented_image)

Output:

In this image, the mean shift segmentation algorithm appears to have identified two main clusters, likely corresponding to the green and orange areas of the background. The brush stroke may not be well-defined because mean shift segmentation is not ideal for elongated or irregular shapes.

Conclusion

Mean shift is very good at detecting closely packed regions in the feature space, thus effectively segmenting the image based on color and spatial information. However, it is non-parametric and does not allow for the specification of a fixed number of clusters. Consequently it is useful for various kinds of images. It is successful in segmenting out coherent regions that are noise-resistant. While it handles noise well and produces coherent regions, it can be computationally intensive, particularly for high-resolution images. Despite this, its ability to generate accurate and intuitive segments makes it a valuable method in image processing and computer vision.